Federated foundational models for embodied perception

Project Information

FRONTIER

Grant agreement ID: 101097822

DOI

10.3030/101097822

EC signature date 11 May 2023

Start date 1 January 2024

End date 31 December 2028

Funded under

• European Research Council (ERC)

Total cost

€ 2 499 825,00

EU contribution

€ 2 499 825,00

Coordinated by

CESKE VYSOKE UCENI TECHNICKE V PRAZE
Czechia

CORDIS provides links to public deliverables and publications of HORIZON projects.

Links to deliverables and publications from FP7 projects, as well as links to some specific result types such as dataset and software, are dynamically retrieved from OpenAIRE .

Publications

Peer reviewed articles (6)

GJK++: Leveraging Acceleration Methods for Faster Collision Detection

Author(s): Louis Montaut, Quentin Le Lidec, Vladimir Petrik, Josef Sivic, Justin Carpentier
Published in: IEEE Transactions on Robotics, Issue 40, 2024, ISSN 1552-3098
Publisher: Institute of Electrical and Electronics Engineers (IEEE)
DOI: 10.1109/TRO.2024.3386370

Temporally Consistent Object 6D Pose Estimation for Robot Control

Author(s): Kateryna Zorina, Vojtech Priban, Mederic Fourmy, Josef Sivic, Vladimir Petrik
Published in: IEEE Robotics and Automation Letters, Issue 10, 2025, ISSN 2377-3766
Publisher: Institute of Electrical and Electronics Engineers (IEEE)
DOI: 10.1109/LRA.2024.3502052

CoVAMPnet: Comparative Markov State Analysis for Studying Effects of Drug Candidates on Disordered Biomolecules

Author(s): Sérgio M. Marques; Petr Kouba; Anthony Legrand; Jiri Sedlar; Lucas Disson; Joan Planas-Iglesias; Zainab Sanusi; Antonin Kunka; Jiri Damborsky; Tomas Pajdla; Zbynek Prokop; Stanislav Mazurenko; Josef Sivic; David Bednar
Published in: JACS Au, Issue 4, 2024, ISSN 2691-3704
Publisher: American Chemical Society (ACS Publications)
DOI: 10.1101/2023.01.06.523007

Self-supervised learning of molecular representations from millions of tandem mass spectra using DreaMS

Author(s): Roman Bushuiev; Anton Bushuiev; Raman Samusevich; Corinna Brungs; Josef Sivic; Tomáš Pluskal
Published in: Nature Biotechnology, Issue 44, 2026, ISSN 1546-1696
Publisher: Springer Nature
DOI: 10.26434/CHEMRXIV-2023-KSS3R-V4

Multi-Task Learning of Object States and State-Modifying Actions From Web Videos

Author(s): Tomáš Souček, Jean-Baptiste Alayrac, Antoine Miech, Ivan Laptev, Josef Sivic
Published in: IEEE Transactions on Pattern Analysis and Machine Intelligence, Issue 46, 2024, ISSN 0162-8828
Publisher: Institute of Electrical and Electronics Engineers (IEEE)
DOI: 10.48550/ARXIV.2211.13500

FocalPose++: Focal Length and Object Pose Estimation via Render and Compare

Author(s): Martin Cífka, Georgy Ponimatkin, Yann Labbé, Bryan Russell, Mathieu Aubry, Vladimir Petrik, Josef Sivic
Published in: IEEE Transactions on Pattern Analysis and Machine Intelligence, Issue 47, 2025, ISSN 0162-8828
Publisher: Institute of Electrical and Electronics Engineers (IEEE)
DOI: 10.48550/ARXIV.2312.02985

Conference proceedings (12)

One Protein Is All You Need

Author(s): Anton Bushuiev, Roman Bushuiev, Olga Pimenova, Nikola Zadorozhny, Raman Samusevich, Elisabet Manaskova, Rachel Seongeun Kim, Hannes Stärk, Jiri Sedlar, Martin Steinegger, Tomáš Pluskal, Josef Sivic
Published in: International Conference on Learning Representations (ICLR), 2026
Publisher: OpenReview / ICLR Foundation
DOI: 10.48550/ARXIV.2411.02109

Learning to Engineer Protein Flexibility

Author(s): Petr Kouba, Joan Planas-Iglesias, Jiri Damborsky, Jiri Sedlar, Stanislav Mazurenko, Josef Sivic
Published in: International Conference on Learning Representations (ICLR), 2025
Publisher: OpenReview / ICLR Foundation
DOI: 10.48550/ARXIV.2412.18275

AlignPose: Generalizable 6D Pose Estimation via Multi-view Feature-metric Alignment

Author(s): Anna Šárová Mikeštíková, Mederic Fourmy, Martin Cífka, Josef Sivic, Vladimír Petrík
Published in: IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR, 2026
Publisher: IEEE / Computer Vision Foundation
DOI: 10.48550/ARXIV.2512.20538

Learning to Design Protein–Protein Interactions with Enhanced Generalization

Author(s): Anton Bushuiev, Roman Bushuiev, Petr Kouba, Anatolii Filkin, Marketa Gabrielova, Michal Gabriel, Jiri Sedlar, Tomas Pluskal, Jiri Damborsky, Stanislav Mazurenko, Josef Sivic
Published in: International Conference on Learning Representations (ICLR), 2024
Publisher: OpenReview / ICLR Foundation
DOI: 10.48550/ARXIV.2310.18515

GenHowTo: Learning to Generate Actions and State Transformations from Instructional Videos

Author(s): Tomáš Souček, Dima Damen, Michael Wray, Ivan Laptev, Josef Sivic
Published in: 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2024
Publisher: IEEE
DOI: 10.48550/ARXIV.2312.07322

MassSpecGym: A benchmark for the discovery and identification of molecules

Author(s): Bushuiev, Roman; Bushuiev, Anton; de Jonge, Niek F.; Young, Adamo; Kretschmer, Fleming; Samusevich, Raman; Heirman, Janne; Wang, Fei; Zhang, Luke; Dührkop, Kai; Ludwig, Marcus; Haupt, Nils A.; Kalia, Apurva; Brungs, Corinna; Schmid, Robin; Greiner, Russell; Wang, Bo; Wishart, David S.; Liu, Li Ping; Rousu, Juho; Bittremieux, Wout; Rost, Hannes; Mak, Tytus D.; Hassoun, Soha; Huber, Florian; van der Hooft, Justin J.J.; Stravs, Michael A.; Böcker, Sebastian; Sivic, Josef; Pluskal, Tomáš
Published in: Conference on Neural Information Processing Systems (NeurIPS), Datasets & Benchmarks Track, 2024
Publisher: Neural Information Processing Systems Foundation, Inc. (NeurIPS)
DOI: 10.48550/ARXIV.2410.23326

Generalization Beyond Benchmarks: Evaluating Learnable Protein–Ligand Scoring Functions on Unseen Targets

Author(s): Jakub Kopko, David Graber, Saltuk Mustafa Eyrilmez, Stanislav Mazurenko, David Bednar, Jiri Sedlar, Josef Sivic
Published in: AI for Scientific Discovery Workshop, NeurIPS, 2025
Publisher: arXiv (Cornell University)
DOI: 10.48550/ARXIV.2512.05386

De Novo Generation of Functional Terpene Synthases Using TpsGP

Author(s): Hamsini Ramanathan, Roman Bushuiev, Matouš Soldát, Jiří Kohout, Téo Hebra, Joshua David Smith, Josef Sivic, Tomáš Pluskal
Published in: NeurIPS 2025 AI for Science Workshop and Machine Learning for Structural Biology (MLSB) Workshop 2025, 2025
Publisher: OpenReview / NeurIPS workshops
DOI: 10.48550/ARXIV.2512.08772

Large-scale Pre-training for Grounded Video Caption Generation

Author(s): Evangelos Kazakos, Cordelia Schmid, Josef Sivic
Published in: IEEE/CVF International Conference on Computer Vision (ICCV), 2025
Publisher: IEEE / Computer Vision Foundatio
DOI: 10.48550/ARXIV.2503.10781

ShowHowTo: Generating Scene-Conditioned Step-by-Step Visual Instructions

Author(s): Tomás Soucek; Prajwal Gatti; Michael Wray; Ivan Laptev; Dima Damen; Josef Sivic
Published in: 2025 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2025
DOI: 10.48550/ARXIV.2412.01987

Revealing data leakage in protein interaction benchmarks

Author(s): Anton Bushuiev; Roman Bushuiev; Jirí Sedlár; Tomás Pluskal; Jirí Damborský; Stanislav Mazurenko; Josef Sivic
Published in: GEM Workshop at International Conference on Learning Representations (ICLR), 2024
Publisher: OpenReview / ICLR Foundation
DOI: 10.48550/ARXIV.2404.10457

6D Object Pose Tracking in Internet Videos for Robotic Manipulation

Author(s): Georgy Ponimatkin, Martin Cífka, Tomáš Souček, Mederic Fourmy, Yann Labbé, Vladimír Petrík, Josef Sivic
Published in: International Conference on Learning Representations (ICLR), 2025
Publisher: OpenReview / ICLR Foundation
DOI: 10.48550/ARXIV.2503.10307