Skip to main content
European Commission logo print header

Metrological Evaluation and Testing of Robots in International CompetitionS

Deliverables

METRICS website

This deliverable contains the project website which includes a page for each competition that presents news videosphoto gallery of meetings and evaluation campaigns webinars press releases related to the competition

Methodological framework for acceptability assessment in the four PAs

This deliverable provides a global approach and specific implementations for each PA for the acceptability analysis of robots participating in the various METRICS competitions including relevant evaluation criteria and a methodology to establish the link between technical metrics agricultural robot site throughput production rate of an agile industrial robot consumption of electricity and other consumables etc and costs associated with the implementation of these robots degree of supervision required staff training to be provided estimates of savings achieved cost associated with this task before and after robot deployment and the societal and ethical impact human risk impact on employment qualification environmental impact etc

Overview of test infrastructures for robotic competitions and requirements for a sustainable activity in Europe

The deliverable contains the inventory and classification of testbeds that are available in Europe for the evaluation of robots in the four PAs their modality of access geographic accessibility structure type physical or digital It concerns both the existing infrastructures and the testbeds that will be designed for the needs of the project It identifies the potential lacks

RAMI Evaluation Plan

The evaluation plan includes the definition of scenarios episode task and functional benchmarks and their associated execution and assessment procedures It takes into account scientific economic ethical legal and safety aspects as well as inputs from stakeholders

Good practice guide for the organisation of evaluation campaigns

The good practice guide for the organisation of evaluation campaigns includes the organisational process based on a dryrun followed by official evaluation campaigns instructions for the fair treatment of participants communication and awarenessraising actions to be carried out This deliverable also includes ethical guidelines for the definition of the evaluation plan and the implementation of the competition

ACRE Evaluation Plan

The evaluation plan includes the definition of scenarios episode task and functional benchmarks and their associated execution and assessment procedures It takes into account scientific economic ethical legal and safety aspects as well as inputs from stakeholders

HEART-MET Evaluation Plan

The evaluation plan includes the definition of scenarios episode task and functional benchmarks and their associated execution and assessment procedures It takes into account scientific economic ethical legal and safety aspects as well as inputs from stakeholders

ADAPT Evaluation Plan

The evaluation plan includes the definition of scenarios episode task and functional benchmarks and their associated execution and assessment procedures It takes into account scientific economic ethical legal and safety aspects as well as inputs from stakeholders

Preliminary methodological framework / Evaluation plan template

This deliverable contains an evaluation plan template for the METRICS competitions including a methodology for the evaluation protocol the evaluation tasks the metrics used the test facilities test databases participation rules and operating procedures capitalising on many campaigns organised by METRICS partners in the past

Publications

Quantitative Benchmarking in Agricultural Robotics

Author(s): Riccardo Bertoglio-Giulio Fontana-Matteo Matteucci-Davide Facchinetti-Stefano Santoro
Published in: 2020
Publisher: Second Italian Conference on Robotics and Intelligent Machines Program - I-RIM

On the Design of the Agri-Food Competition for Robot Evaluation (ACRE)

Author(s): Riccardo Bertoglio, Giulio Fontana, Matteo Matteucci, Davide Facchinetti, Michel Berducat, Daniel Boffety
Published in: 2021 IEEE International Conference on Autonomous Robot Systems and Competitions (ICARSC), 2021, Page(s) 161-166, ISBN 978-1-6654-3198-9
Publisher: IEEE
DOI: 10.1109/icarsc52212.2021.9429792

AI evaluation campaigns during robotics competitions: the METRICS paradigm

Author(s): Guillaume Avrin-Virginie Barbosa-Agnes Delaborde
Published in: 2020
Publisher: 1st International Workshop on Evaluating Progress in Artificial Intelligence - EPAI 2020 In conjunction with the 24th European Conference on Artificial Intelligence - ECAI 2020

From ERL to RAMI: Expanding Marine Robotics Competitions - Through Virtual Events

Author(s): Gabriele Ferri; Fausto Ferreira; Alessandro Faggiani; Tommaso Fabbri
Published in: OCEANS 2021: San Diego – Porto, 2022
Publisher: IEEE
DOI: 10.23919/oceans44145.2021.9706098

Automatic Dataset Generation From CAD for Vision-Based Grasping

Author(s): Saad Ahmad, Kulunu Samarawickrama, Esa Rahtu and Roel Pieters
Published in: 2021 20th International Conference on Advanced Robotics (ICAR), 2022
Publisher: IEEE
DOI: 10.1109/icar53236.2021.9659336

Validation de méthodologies d’évaluation de solutions de désherbage autonomes, dans le cadre des projets Challenge ROSE et METRICS. Validation of methodologies for the evaluation of autonomous weed in the framework of the Challenge ROSE and METRICS projects.

Author(s): Rémi Regnier ; Guillaume Avrin ; Virginie Barbosa ; Daniel Boffety ; Anne Kalouguine ; Sophie Lardy-Fontan
Published in: Revue Ouverte d’Intelligence Artificielle-ROIA Open Journal of Artificial Intelligence-ROIA, 2021
Publisher: Revue Ouverte d’Intelligence Artificielle-ROIA Open Journal of Artificial Intelligence-ROIA
DOI: 10.5802/roia.8

Benchmarking Robots by Inducing Failures in Competition Scenarios

Author(s): Santosh Thoduka, Nico Hochgeschwender
Published in: Digital Human Modeling and Applications in Health, Safety, Ergonomics and Risk Management. AI, Product and Service - 12th International Conference, DHM 2021, Held as Part of the 23rd HCI International Conference, HCII 2021, Virtual Event, July 24–29, 2021, Proceedings, Part II, Issue 12778, 2021, Page(s) 263-276, ISBN 978-3-030-77819-4
Publisher: Springer International Publishing
DOI: 10.1007/978-3-030-77820-0_20

A COTS (UHF) RFID Floor for Device-Free Ambient Assisted Living Monitoring

Author(s): Smith, R., Ding, Y., Goussetis, G., Dragone, M
Published in: Advances in intelligent system and computing - Vol 1239 - Ambiant intelligence - Software and applications - 11th international symposium of ambient intelligence, 2020
Publisher: International Symposium on Ambient Intelligence
DOI: 10.1007/978-3-030-58356-9

Assessing artificial intelligence capabilities

Author(s): Guillaume Avrin - Laboratoire national de métrologie et d'essais
Published in: Educational Research and Innovation - AI and the Future of Skills, Volume 1 - CAPABILITIES AND ASSESSMENTS, 2021
Publisher: OECD iLibrary
DOI: 10.1787/47d04fe3-en

The VISTA datasets, a combination of inertial sensors and depth cameras data for activity recognition

Author(s): Fiorini, L., Cornacchia Loizzo, F. G., Sorrentino, A., Rovini, E., Di Nuovo, A., & Cavallo, F.
Published in: Nature Scientific Data, Issue Scientific Data, 9(1), 1-14 -Article number: 218 (2022), 2022, ISSN 2052-4463
Publisher: Scientific Data (Sci Data)
DOI: 10.1038/s41597-022-01324-3

Activity Recognition for Ambient Assisted Living with Videos, Inertial Units and Ambient Sensors

Author(s): Caetano Mazzoni Ranieri, Scott MacLeod, Mauro Dragone, Patricia Amancio Vargas, Roseli Aparecida Francelin Romero 
Published in: Sensors, Issue 21/3, 2021, Page(s) 768, ISSN 1424-8220
Publisher: Multidisciplinary Digital Publishing Institute (MDPI)
DOI: 10.3390/s21030768

A Comparative Study of Fourier Transform and CycleGAN as Domain Adaptation Techniques for Weed Segmentation

Author(s): Riccardo Bertoglio; Alessio Mazzucchelli ; Nico Catalano ; Matteo Matteucci
Published in: Smart Agricultural Technology, 2023, ISSN 1872-6291
Publisher: Science direct - Elsevier
DOI: 10.1016/j.atech.2023.100188

Searching for OpenAIRE data...

There was an error trying to search data from OpenAIRE

No results available