Periodic Reporting for period 2 - MultiRATE (Holistic framework for the MatUrity evaLuaTIon of ReAdiness level for security TEchnologies)
Período documentado: 2024-04-01 hasta 2025-05-31
Τhe MultiRATE project, created a Holistic RL (HRL) assessment framework to evaluate the maturity of products, systems, or processes developed in civil security R&D projects for both Law Enforcement Agencies (LEAs) and non-LEAs. MultiRATE’s RL dimensions include adapted versions of existing RL scales like Technology RL, Societal RL, Integration RL, Commercialisation RL, Manufacturing RL as well as additional RL dimensions relevant in the field, such as Security RL and Legal, Privacy and Ethics RL. All seven dimensions needed to be holistically harmonized to meet the project’s objectives. Additionally, the framework includes an investment forecasting module to estimate the costs required to increase RL maturity.
MultiRATE project elaborated on the implementation of this holistic framework and the evaluation of its feasibility by the project’s conclusion. More specifically, the contributions include:
• The MultiRATE HRL calculator that integrates the indicators and methodologies of the seven RL dimensions taking into account all their peculiarities adapted to the civil security sector. The web-based MultiRATE HRL calculator can be accessed here: https://hrl.multiratehl.iti.gr(se abrirá en una nueva ventana).
• The integration process of an Investment Forecasting module to effectively estimate the costs required to increase RL maturity in the context of the MultiRATE HRL.
• The validation process of the MultiRATE HRL and its components with the collaboration of an external network of experts following the Design Science Research
Methodology (DSRM)
The ambition of MultiRATE is to develop a holistic, homogenous and harmonised RL evolution methodology and calculator for R&D solutions in the security domain, available to the EU R&D community. More specifically MultiRATE’s objectives are:
• To conduct research in the abovementioned RL fields to create solid ground for the development of the holistic, homogenous and harmonised RL evaluation methodology.
• To accurately evaluate solutions in all RL fields and to develop unique methodology and indicators as each field measures different aspects and has different objectives.
• To develop the HRL evaluation methodology for security related solutions. The EU R&D community will take advantage of open access HRL, harmonising the approach taken from all projects, initiatives and more.
• Establish a network of projects and organisations that will support the design of the appropriate evaluation methodologies, and will support the calibration and testing of both individual RLs and the holistic.
• Evaluate accurately the RLs of both EU and National R&D solutions as well as the investment forecasting and business plan of organisations' solutions regarding the future strategic research planning.
MultiRATE's pathways to impact include the increase in literacy on the value and efficient use of maturity assessment frameworks to communicate the readiness of technology, synchronise parallel developments, forecast implementation and support decision making in the planning of investment in the area of security. Additionally, another aspect includes the improved cross-disciplinary assessment of the maturity of innovative technologies based on common harmonised frameworks for the security domain. Further to that, a comprehensive and timely updated map of the maturity of the security solutions developed through EU-funded security research and innovation programmes enabled by widely accessible assessment tools and methods. Last but not least, the pathway to impact includes the evidence-based programming of security research built on a more reliable assessment of the SOTA technologies in the field of security.
MultiRATE project elaborated on the implementation of this holistic framework and the evaluation of its feasibility by the project’s conclusion. More specifically, the contributions include:
• The MultiRATE HRL calculator that integrates the indicators and methodologies of the seven RL dimensions taking into account all their peculiarities adapted to the civil security sector. The web-based MultiRATE HRL calculator can be accessed here: https://hrl.multiratehl.iti.gr(se abrirá en una nueva ventana).
• The integration process of an Investment Forecasting module to effectively estimate the costs required to increase RL maturity in the context of the MultiRATE HRL.
• The validation process of the MultiRATE HRL and its components with the collaboration of an external network of experts following the Design Science Research
Methodology (DSRM)
The ambition of MultiRATE is to develop a holistic, homogenous and harmonised RL evolution methodology and calculator for R&D solutions in the security domain, available to the EU R&D community. More specifically MultiRATE’s objectives are:
• To conduct research in the abovementioned RL fields to create solid ground for the development of the holistic, homogenous and harmonised RL evaluation methodology.
• To accurately evaluate solutions in all RL fields and to develop unique methodology and indicators as each field measures different aspects and has different objectives.
• To develop the HRL evaluation methodology for security related solutions. The EU R&D community will take advantage of open access HRL, harmonising the approach taken from all projects, initiatives and more.
• Establish a network of projects and organisations that will support the design of the appropriate evaluation methodologies, and will support the calibration and testing of both individual RLs and the holistic.
• Evaluate accurately the RLs of both EU and National R&D solutions as well as the investment forecasting and business plan of organisations' solutions regarding the future strategic research planning.
MultiRATE's pathways to impact include the increase in literacy on the value and efficient use of maturity assessment frameworks to communicate the readiness of technology, synchronise parallel developments, forecast implementation and support decision making in the planning of investment in the area of security. Additionally, another aspect includes the improved cross-disciplinary assessment of the maturity of innovative technologies based on common harmonised frameworks for the security domain. Further to that, a comprehensive and timely updated map of the maturity of the security solutions developed through EU-funded security research and innovation programmes enabled by widely accessible assessment tools and methods. Last but not least, the pathway to impact includes the evidence-based programming of security research built on a more reliable assessment of the SOTA technologies in the field of security.
During the second reporting period the main achievements include:
• The testing and calibration of all seven RLs appropriately during the last two validation cycles held with both internal and external evaluators.
• The HRL web-based calculator was fully developed and through the last two 3rd and 4th evaluation cycles, it made several advancements.
• Through the last two evaluation cycles more than 37 evaluations were conducted counting more than 75 elements to have been evaluated.
• As regards to the network establishment 35 security R&D projects have been identified, 6 supporting entities identified, 2 EC and agencies identified, and 14 other entities and experts identified.
• The Forecasting Module and Visual Analytics tool has been fully developed and integrated into the Visual Analytics Dashboard allowing real time use. More than 150 training data were selected, to train the model.
• The testing and calibration of all seven RLs appropriately during the last two validation cycles held with both internal and external evaluators.
• The HRL web-based calculator was fully developed and through the last two 3rd and 4th evaluation cycles, it made several advancements.
• Through the last two evaluation cycles more than 37 evaluations were conducted counting more than 75 elements to have been evaluated.
• As regards to the network establishment 35 security R&D projects have been identified, 6 supporting entities identified, 2 EC and agencies identified, and 14 other entities and experts identified.
• The Forecasting Module and Visual Analytics tool has been fully developed and integrated into the Visual Analytics Dashboard allowing real time use. More than 150 training data were selected, to train the model.
During the 2nd period all milestones have been achieved. All seven RLs have produced a set of indicators and evaluation methodology, while the AI-based model of the FM has been trained based on an final dataset collected from the partners. The HRL web based calculator has been released and disseminated. Also, the 3rd and 4th evaluation cycles have been implemented. The achievements of the 2nd period are:
1 Development of the TRL indicators focused on Hardware and alignement with the Software related indicators to fit the TRL calculator.
a Development of the standalone web based TRL calculator
2 Improvements of the SocRL indicators and methodology
3 Improvements of the SecRL indicators and methodology
4 Improvements of the LPERL indicators and methodology from the ground up.
5 Improvements of the IRL indicators and methodology
6 Improvements of the CRL indicators and methodology
7 Improvements of the MRL indicators and methodology
8 Development of the HRL web based calculator
9 Development of the final version of the FM and Visual Analytics tool.
10 As regards to the established network, 35 security R&D projects have been identified, 6 supporting entities identified, 2 EC and agencies identified, and 14 other entities and experts identified. Among others, EARTO, the EU Innovation Hub, DG Home and REA have provided input and feedback.
11 The certification mechanism has beeb defined.
The HRL web based calculator will be open to use from the EU R&D community for 5 years after the end of the project.
1 Development of the TRL indicators focused on Hardware and alignement with the Software related indicators to fit the TRL calculator.
a Development of the standalone web based TRL calculator
2 Improvements of the SocRL indicators and methodology
3 Improvements of the SecRL indicators and methodology
4 Improvements of the LPERL indicators and methodology from the ground up.
5 Improvements of the IRL indicators and methodology
6 Improvements of the CRL indicators and methodology
7 Improvements of the MRL indicators and methodology
8 Development of the HRL web based calculator
9 Development of the final version of the FM and Visual Analytics tool.
10 As regards to the established network, 35 security R&D projects have been identified, 6 supporting entities identified, 2 EC and agencies identified, and 14 other entities and experts identified. Among others, EARTO, the EU Innovation Hub, DG Home and REA have provided input and feedback.
11 The certification mechanism has beeb defined.
The HRL web based calculator will be open to use from the EU R&D community for 5 years after the end of the project.