Leveraging Benchmarking Data for Automated Machine Learning and Optimization

Project Information

AutoLearn-SI

Grant agreement ID: 101187010

DOI

10.3030/101187010

EC signature date 5 August 2024

Start date 1 March 2025

End date 28 February 2030

Funded under

• Widening participation and spreading excellence

Total cost

No data

EU contribution

€ 2 485 960,00

Coordinated by

INSTITUT JOZEF STEFAN
Slovenia

Periodic Reporting for period 1 - AutoLearn-SI (Leveraging Benchmarking Data for Automated Machine Learning and Optimization)

Reporting period: 2025-03-01 to 2026-05-31

Summary of the context and overall objectives of the project

Artificial intelligence has rapidly transformed research and industry across Europe, with the European Research Executive Agency managing over 1,000 AI-related projects totalling €1.7 billion in funding. Automated Machine Learning (AutoML) and Automated Optimization (AutoOPT) — collectively AutoAI — have emerged as critical enablers for democratized AI use, reducing the need for deep expertise and making AI accessible to small businesses and non-specialist practitioners.

Despite this global momentum, a significant gap exists in Slovenia and the wider Central and Eastern European region, where dedicated AutoAI research groups, academic programmes, and training opportunities are largely absent. AutoLearn-SI, led by the Jožef Stefan Institute (JSI) and submitted to the Horizon Europe ERA Chairs call, seeks to address this by establishing a permanent, internationally excellent AutoAI research group. The ERA Chair is Asst. Prof. Dr. Eva Tuba and the coordinator is Asst. Prof. Tome Eftimov.

The project pursues five integrated objectives: establishing the region's first dedicated AutoAI research center at JSI; generating new scientific knowledge across experimental databases, representation learning, and algorithm selection; training researchers in both technical and transferable skills; embedding AutoML and AutoOPT courses into master's and doctoral programmes; and building long-term sustainability through industrial partnerships and follow-on funding.

Expected outputs include 24 peer-reviewed publications, two FAIR benchmarking databases, and ten open-source Python implementations. Industry guidelines aligned with the EU AI Act will support transparent AI adoption, while a focus on algorithm efficiency will reduce energy consumption and promote greener AI practices. Over the longer term, AutoLearn-SI aims to become a self-standing JSI department and a regional hub for AutoAI research and innovation.

Work performed from the beginning of the project to the end of the period covered by the report and main results achieved so far

During the initial phase of AutoLearn-SI (WP1 and WP2, covering months 1–15), the project successfully established the ERA Chair within JSI's Computer Systems Department, putting in place all key governance and management structures, including the Executive Science Committee, Management Board, and Advisory Board. The ERA Chair holder and PhD candidates were recruited in line with the European Code of Conduct for the Recruitment of Researchers. Foundational project documents were delivered, including the Data Management Plan, the Dissemination and Exploitation Plan, and the ERA Chair's strategic sustainability plan. The project's public-facing presence was launched through a dedicated website, social media accounts, and a suite of promotional materials, complemented by press releases and editorial articles targeting the international AutoAI research community. On the scientific side, the three PhD students commenced work on their respective topics — covering experimental databases and meta-features for single-objective optimization, experimental databases and meta-features for multi-label classification, and automated algorithm selection and configuration — producing initial state-of-the-art knowledge reports as the foundation for future AutoML and AutoOPT research. Educational activities were also completed, including a two-day exploratory workshop bringing together leading EU researchers from associated partner institutions, a Science Communication training event, and an AI and Data Privacy & Security training event, all open to national and international students and young entrepreneurs.

Progress beyond the state of the art and expected potential impact (including the socio-economic impact and the wider societal implications of the project so far)

AutoLearn-SI has advanced the state of the art in Automated Machine Learning (AutoML) and Automated Optimisation (AutoOPT) through systematic cataloguing, structured knowledge building, and the establishment of shared scientific infrastructure. In single-objective optimisation, the project reviewed and structured existing landscape meta-feature extraction approaches, mapping key gaps including benchmark bias, generalisation limitations, and interpretability deficits. In multi-label classification, a systematic overview of meta-learning frameworks was produced, significantly extending the MLC Dataset Catalogue to 89 datasets and 26 benchmark methods as an open community resource. In algorithm selection and configuration, trajectory-based representation approaches were catalogued, identifying the lack of dynamic run-time representations and limited cross-domain transfer as critical open challenges. Underpinning all three areas is the OPTION knowledge base — a FAIR-compliant, RDF-structured repository of over 226 optimisation algorithms serving as shared semantic infrastructure for the AutoAI community. These activities have supported 15 peer-reviewed publications across 6 international research collaborations.
Looking ahead, Phase II will focus on completing two FAIR experimental databases for SOO and MLC, delivering open-source Python libraries, and consolidating the meta-feature catalogue as a shared community resource. Industry-facing AutoAI guidelines aligned with EU AI Act requirements will be developed, targeting adoption through the Slovenian AI Factory and AutoML 2026 in Ljubljana. Sustainability is secured by over €1,357,676 in confirmed external funding — including MSCA Postdoctoral Fellowships and MSCA COFUND SQUASH — ensuring continuity of the AutoAI research group at JSI beyond the ERA Chair grant. Collaboration with Slovenia's Ministry of Digital Transformation, IRCAI UNESCO, and two national SRIP networks further positions project outputs to contribute to national AI governance and broader EU standardisation efforts.