Periodic Reporting for period 1 - MultiRATE (Holistic framework for the MatUrity evaLuaTIon of ReAdiness level for security TEchnologies)
Période du rapport: 2022-12-01 au 2024-03-31
Despite the fact that a lot of effort has been made to incorporate widely used methods and frameworks, the current maturity and Readiness Level (RL) evaluation process does not lack subjectivity. MultiRATE aims to evaluate and apply the readiness of seven aspects of R&D security solutions, namely, Technology RL(TRL), Societal Readiness Level(SocRL), Security RL(SecRL), Legal, Privacy and Ethics RL(LPERL), Integration RL(IRL), Commercialisation RL(CRL) and Manufacturing RL(MRL). These RLs set the basis to the Holistic RL(HRL) framework that evaluates the readiness of all aspects of a solution. Also, MultiRATE has already started to build more reliable forecasting for future investment regarding the solutions developed through EU R&D projects utilizing the clustering, forecasting or prediction analytics and visual analytics.
The ambition of MultiRATE is to develop a holistic, homogenous and harmonised RL evolution methodology and calculator for R&D solutions in the security domain, available to the EU R&D community.
MultiRATE’s objectives are:
O1 MultiRATE will conduct research in the abovementioned fields to create solid ground for the development of the holistic, homogenous and harmonised RL evaluation methodology.
O2 To accurately evaluate solutions in all fields of O1, MultiRATE will develop unique methodology and indicators as each field measures different aspects and has different objectives.
O3 It will develop the HRL evaluation methodology for security related solutions. The EU R&D community will take advantage of open access HRL, harmonising the approach taken from all projects, initiatives and more.
O4 Establishment of a network of projects and organisations that will support the design of the appropriate evaluation methodologies, and will support the calibration and testing of both individual RLs and the holistic.
O5 Accurate evaluation of the RLs supporting both EU and National R&D solutions as well as the investment forecasting and business plan of organisations' solutions regarding the future strategic research planning.
MultiRATE's pathways to impact are:
EO1 Increased literacy on the value and efficient use of maturity assessment frameworks to communicate the readiness of technology, synchronise parallel developments, forecast implementation and support decision making in the planning of investment in the area of security.
EO2 Improved cross-disciplinary assessment of the maturity of innovative technologies based on common harmonised frameworks for the security domain.
EO3 Comprehensive and timely updated map of the maturity of the security solutions developed through EU-funded security research and innovation programmes enabled by widely accessible assessment tools and methods.
EO4 Evidence-based programming of security research built on a more reliable assessment of the SOTA technologies in the field of security.
The ambition of MultiRATE is to develop a holistic, homogenous and harmonised RL evolution methodology and calculator for R&D solutions in the security domain, available to the EU R&D community.
MultiRATE’s objectives are:
O1 MultiRATE will conduct research in the abovementioned fields to create solid ground for the development of the holistic, homogenous and harmonised RL evaluation methodology.
O2 To accurately evaluate solutions in all fields of O1, MultiRATE will develop unique methodology and indicators as each field measures different aspects and has different objectives.
O3 It will develop the HRL evaluation methodology for security related solutions. The EU R&D community will take advantage of open access HRL, harmonising the approach taken from all projects, initiatives and more.
O4 Establishment of a network of projects and organisations that will support the design of the appropriate evaluation methodologies, and will support the calibration and testing of both individual RLs and the holistic.
O5 Accurate evaluation of the RLs supporting both EU and National R&D solutions as well as the investment forecasting and business plan of organisations' solutions regarding the future strategic research planning.
MultiRATE's pathways to impact are:
EO1 Increased literacy on the value and efficient use of maturity assessment frameworks to communicate the readiness of technology, synchronise parallel developments, forecast implementation and support decision making in the planning of investment in the area of security.
EO2 Improved cross-disciplinary assessment of the maturity of innovative technologies based on common harmonised frameworks for the security domain.
EO3 Comprehensive and timely updated map of the maturity of the security solutions developed through EU-funded security research and innovation programmes enabled by widely accessible assessment tools and methods.
EO4 Evidence-based programming of security research built on a more reliable assessment of the SOTA technologies in the field of security.
During the first reporting period the main achievements include:
• A literature survey on the mechanisms underlying technology maturity was conducted. Several factors identified that contribute to maturity, innovation lifecycles were described, and success and failure factors of EU R&D projects were highlighted. Further cross-disciplinary needs for RLs were identified, outlining user requirements for each RL. Furthermore, based on the literature review it was decided that MultiRATE will use well established scales (e.g. TRL) where possible.
• A unique evaluation methodology and indicators (per level) for each RL was established through research analysis and well-known approaches in most RLs while in RLs such as the LPERL the methodology and indicators were developed all together within MultiRATE.
• The HRL evaluation and calculation methodology was designed, by incorporating all indicators and methodologies identified while considering the unique aspects of each RL domain. Taking into considerations all levels of RLs grouping/mapping them into 4 main categories: Solution concept, Solution prototype, Functional solution and Ready-to-market solution. Finally, the web-based HRL calculator encapsulates the holistic methodology.
• Several entities were identified aligning with the project’s goals. Formal approach and documentation were established (NDA/MOU) in order to reach out to potential network members requesting collaboration. A dedicated database is used to monitor progress and interactions with the network. This methodology is designed to foster effective and meaningful collaborations throughout the project lifecycle.
• Desktop research on the domain of the Forecasting Module (FM) was performed. Initially the developed model was trained on publicly available data. Then the model was trained in a dataset gathered by the MultiRATE partners. A draft visual dashboard was developed showcasing the gathered dataset and providing an insight of how the parameters of an element can affect the targeting RL or/and the allocated effort of an element.
• A literature survey on the mechanisms underlying technology maturity was conducted. Several factors identified that contribute to maturity, innovation lifecycles were described, and success and failure factors of EU R&D projects were highlighted. Further cross-disciplinary needs for RLs were identified, outlining user requirements for each RL. Furthermore, based on the literature review it was decided that MultiRATE will use well established scales (e.g. TRL) where possible.
• A unique evaluation methodology and indicators (per level) for each RL was established through research analysis and well-known approaches in most RLs while in RLs such as the LPERL the methodology and indicators were developed all together within MultiRATE.
• The HRL evaluation and calculation methodology was designed, by incorporating all indicators and methodologies identified while considering the unique aspects of each RL domain. Taking into considerations all levels of RLs grouping/mapping them into 4 main categories: Solution concept, Solution prototype, Functional solution and Ready-to-market solution. Finally, the web-based HRL calculator encapsulates the holistic methodology.
• Several entities were identified aligning with the project’s goals. Formal approach and documentation were established (NDA/MOU) in order to reach out to potential network members requesting collaboration. A dedicated database is used to monitor progress and interactions with the network. This methodology is designed to foster effective and meaningful collaborations throughout the project lifecycle.
• Desktop research on the domain of the Forecasting Module (FM) was performed. Initially the developed model was trained on publicly available data. Then the model was trained in a dataset gathered by the MultiRATE partners. A draft visual dashboard was developed showcasing the gathered dataset and providing an insight of how the parameters of an element can affect the targeting RL or/and the allocated effort of an element.
During the 1st period all milestones have been achieved, indicating that the progress is on schedule. A review on the different RL maturity evaluation methodologies has been done assisting greatly in the development of MultiRATE’s RL methodologies and indicators. All seven RLs have produced a set of indicators and evaluation methodology, while the AI-based model of the FM has been trained based on an initial dataset collected from the partners. The 1st version of the HRL has been released and disseminated. Also, two evaluation cycles have been implemented. The achievements of the 1st period are:
1 Development of the TRL indicators focused on Software
a Questionnaire for assessing the current indicators and asking the importance of each indicator based on the type of responder has been circulated
b Identification and discussion of the possible methodologies for weight-based calculation (per indicator/per type of responder) was initiated
c The design of a Web based standalone TRL was initiated (e.g. database model etc.)
2 Development of the SocRL indicators and methodology
3 Development of the SecRL indicators and methodology
4 Development of the LPERL indicators and methodology from the ground up. Furthermore, a web based LPERL calculator has been developed
5 Development of the IRL indicators and methodology
6 Development of the CRL indicators and methodology
7 Development of the MRL indicators and methodology
8 Development of the 1st version of the HRL
9 Development of the 1st version of the FM
10 The Network has 9 members, 10 has signed and 12 more have accepted the invitation. EARTO, the EU Innovation Hub, DG Home and REA have provided input and feedback.
11 Initial steps for the creation of the certification mechanism have been taken
The web based HRL will be open to use from the EU R&D community after the end of the project and during the next period of the project the sustainability framework will be discussed in details.
1 Development of the TRL indicators focused on Software
a Questionnaire for assessing the current indicators and asking the importance of each indicator based on the type of responder has been circulated
b Identification and discussion of the possible methodologies for weight-based calculation (per indicator/per type of responder) was initiated
c The design of a Web based standalone TRL was initiated (e.g. database model etc.)
2 Development of the SocRL indicators and methodology
3 Development of the SecRL indicators and methodology
4 Development of the LPERL indicators and methodology from the ground up. Furthermore, a web based LPERL calculator has been developed
5 Development of the IRL indicators and methodology
6 Development of the CRL indicators and methodology
7 Development of the MRL indicators and methodology
8 Development of the 1st version of the HRL
9 Development of the 1st version of the FM
10 The Network has 9 members, 10 has signed and 12 more have accepted the invitation. EARTO, the EU Innovation Hub, DG Home and REA have provided input and feedback.
11 Initial steps for the creation of the certification mechanism have been taken
The web based HRL will be open to use from the EU R&D community after the end of the project and during the next period of the project the sustainability framework will be discussed in details.