Community Research and Development Information Service - CORDIS


U-Test Report Summary

Project ID: 645463
Funded under: H2020-EU.

Periodic Reporting for period 1 - U-Test (Testing Cyber-Physical Systems under Uncertainty: Systematic, Extensible, and Configurable Model-based and Search-based Testing Methodologies)

Reporting period: 2015-01-01 to 2015-12-31

Summary of the context and overall objectives of the project

Cyber-Physical Systems (CPSs) are the next generation of highly connected embedded systems. These systems have applications in varied domains including industrial automation, healthcare, robotics, and maritime industry. Even in the presence of uncertainty, CPSs must be dependable, i.e., trustworthy, robust, efficient, and safe. Inappropriate handling of uncertainty in CPSs during their real operations may have devastating effects on their users and/or environment. The U-Test project aims at ensuring that CPSs are tested adequately under uncertainty using systematic and automated techniques such as model and search-based testing to guarantee their dependable operation in real environment.
The overall objective of the U-Test project is to:
“Improve the dependability of CPSs, via cost-effective model-based and search-based testing of CPSs under uncertainty, by defining an Uncertainty Taxonomy and holistic modelling and testing frameworks with considerable reliance on standards.”
The overall objective will be met by the following project objectives:
1. O1: Provide unified and precise definitions of uncertainties in CPSs and systematically classify and characterize them, by developing a comprehensive Uncertainty Taxonomy (U-Taxonomy) with the ultimate aim of enabling the reuse, potentially via standardization, of the taxonomy across a wide range of diverse domains.
2. O2: Enable systematic, holistic and standard-based modelling of uncertainties in CPSs by developing a configurable and extensible Uncertainty Modelling Framework (UMF) utilizing existing standards.
3. O3: Developing a smart, systematic and automated approach of discovering realistic unknown uncertainties relying on existing search algorithms to support testing.
4. O4: Develop a standard-based and configurable Uncertainty Testing Framework (UTF) to generate cost effective test cases (e.g., achieving high coverage and having high chances of catching faults with lowest possible cost (in terms of e.g., test case execution time, test case generation time) possible by utilizing search algorithms (e.g., genetic algorithms).

Work performed from the beginning of the project to the end of the period covered by the report and main results achieved so far

As discussed in Section 1.1 and the DoA, there are four project objectives of U-Test (O1 to O4). O1 is associated with WP1, O2 is associated with WP2, whereas O3 and O4 are associated with WP3. WP4 implements the tool supports and this is associated with all of the objectives.
During the first reporting year (2015), the first objective is achieved, i.e.,
O1: Providing a comprehensive and extensible taxonomy of uncertainties, classifying uncertainties, their properties, and their relationships.
We have achieved this by defining, we have developed an uncertainty taxonomy (U-Taxonomy) specifically CPS under Task 1.2. At the current state-of-the-art and practice, Uncertainty in CPS during their development and testing is not studied much. Due to this reason, U-Taxonomy is designed based on the existing work on uncertainty from other fields such as philosophy, physics, statistics, and healthcare. The taxonomy is mapped to the three logical levels of CPS: Application, Infrastructure, and Integration as defined in the DoA. In addition, in parallel, a set of uncertainty requirements were collected from the use case providers of U-Test, i.e., FPX and ULMA Handling systems under Task 1.1. The collected uncertainty requirements were classified into the three levels of CPS.
In Task 1.3, we carried out validation of the taxonomy and the requirements and studied with the following two aims in our mind, 1) To precisely define the requirements the requirements such that those can be manually transformed into test ready models, 2) Validating that the U-Taxonomy is sufficiently complete with respect to the two use cases. The above two aims were achieved in the following manner. First, the uncertainty requirements collected in Task 1.1 were validated in a more systematic way including inspection, reviews, and questionnaires. Second, U-Taxonomy’s initial version produced as part of Task 1.2 was validated against the uncertainty requirements collected from the case study providers to assess the completeness of the taxonomy. As the result, both requirements and taxonomy were refined and the revised versions of both were submitted with Task 1.3. The results showed that the collected uncertainty requirements can be mapped with U-Taxonomy and thus suggesting that U-Taxonomy is complete with respect to the two use cases. This demonstrates that we have achieved the G1 goal, i.e., The completeness of the developed Uncertainty Taxonomy with respect to the two use cases is 100%.
With respect to O2, WP2 started in October 2015 and thus we haven’t fully achieved the project objective. However, we have developed an initial version of Uncertainty Modeling Framework (UMF). At the core of framework is the implementation of U-Taxonomy as a UML profile, with which uncertainty can be modeled at the three levels of CPS using UML structural and behavioral models. The UMF also uses exiting standards including UML Profile for Modeling Real-Time and Embedded Systems (MARTE) and the UML Test Profile V.2. An initial set of model libraries has been developed including model libraries for uncertainty measurement, patterns, and risks. To demonstrate the G2 of the project, i.e., All the identified uncertainties for the use cases can be 100% modelled with proposed modelling methodologies, a dedicated modelling workshop is conducted with case study providers on January 19 and January 20.
Finally, first version of the evaluation plan has been developed under Task 1.4 that will be used for assessing cost-effectiveness of test cases generated with the Uncertainty Testing Framework (UTF) and is associated with O4. Notice that we believe that the evaluation plan is an evolving document and will be refined as we proceed in the project. O3 and O4 will be partially achieved during the second reporting period.

Progress beyond the state of the art and expected potential impact (including the socio-economic impact and the wider societal implications of the project so far)

WP7 aim is to ensure that the project results will effectively be exploited in the market. Hence, WP7 will bridge between the RTD activities and the project expected impact objectives. A parallel exploitation goal is to deal with the intellectual property rights’ (IPRs) background and foreground issues during the project, in order to establish a fair and successful collaborative exploitation strategy. The four key objectives for this work package are: (1) Identify U-Test’s potential value opportunities; (2) Define business oriented exploitation approaches for the value opportunities; business models and business plans basics; (3) Bring the project’s exploitable results’ paths progressively closer to the impact objectives through product-market contrasts, and; (4) Establish consortium’s collaborative exploitation strategies and foreground IPRs agreements to facilitate partners’ exploitation initiatives.
During the first period (M01-M12):
• 26 potential sources of revenue have been identified and 13 of them have been prioritized with regards to U-Test’s key predetermined results; Uncertainty Taxonomy (UTX), Uncertainty Modelling Framework (UMF) and Uncertainty Testing Framework (UTF). These potential sources of revenue, or value opportunities, take the form of products, services and collaborations. Also, key issues that can affect the potential market success of these value opportunities have been identified through the Technology, Market and Enablers-Barriers layers’ analyses. These issues will be monitored along the project development.

• The 13 preselected value opportunities have been further elaborated in the form of value propositions: Extended description of the products/services; What Customer segment(s) do they expect to reach? Who do they create value for?; What specific Value (problems or needs solved) are they creating to these customer segments?; Who are the Competitors?; Which Distinct value, with regards to the competitors, are they creating for the customer segments? (e.g. price, novelty, brand, quality, functionality, easy-to-use, customisation, risk reduction...)

• Finally, each partner has stated its Background and Foreground for the identified 13 value opportunities and related business concepts and, following MULO methodology for collaborative exploitation, they have stated initial exploitation claims: Making and selling it; Providing services; Licensing it to 3rd parties; Use it internally to make something else for sale.

Standardization plays an important role in a modern society, which is being increasingly globe. To be competitive internationally, many industries are encouraged or forced to conform to various standards. In other words, standards make an important contribution to national and international competitiveness, which is now a well-recognized fact. To be more specific, in the context of the U-Test project, making sure that the U-Test frameworks conform to existing and widely adopted standards is an important strategy to improve the competitiveness of the consortium.
SRL leads Task 6.1 of WP6, which is to make investigation and selection of standardization bodies, where the results of U-Test can be promoted. Standardization activities in the context of U-Test has impact on the development of the U-Test frameworks from several aspects: referencing to relevant standards on which the U-Test frameworks can be built, contributing to selected standards or proposing new standards based on the U-Test outcomes, and disseminating the U-Test outcomes to relevant standardization communities. In 2015, the U-Test consortium focused on the selection of relevant standardization bodies and standards (reported in Deliverable 6.2 as a result). In summary, the U-Test consortium went through major standardization bodies including ISO, IEEE, IEC, ETSI and OMG, and more than 45 standards of the selected standardization bodies. We also classified and characterized these standards according to their relevance to the U-Test project. As the results, we selected 6 standards (i.e., MARTE, SysML, OCL, UTP, ISO/IEC/IEEE 29119, and ETSI EG 201 015) from five major standardization bodies (i.e., OMG, ISO, IEC, IEEE and ETSI), to which the U-Test outcomes can potentially contribute. SRL, FF and EGM conducted various standardization activities in 2015, which includes contributing to the development of the UTP 2 revised submission, participating the definition of several other standards, and referencing and applying several existing standards. It is also worth mentioning that the U-Test consortium has started to look for opportunities to propose a new standard on one of the U-Test outcomes: U-Taxonomy in OMG.

The deployment of Cyber-Physical Systems (CPS) is expected to increase substantially over the next decades, holding great potential for novel applications and innovative product development. Digital technologies have already penetrated day-to-day life massively, affecting all kinds of interactions between humans and their environment. However, the inherent complexity of CPSs, as well as the need to achieve optimum performance and to comply with essential standards for safety, privacy and security raises many questions that are explored by the international research community. U-Test project addresses another essential aspect of CPS i.e. uncertainty and looks into Improving the dependability of CPSs, via cost-effective model-based and search-based testing.

U-Test is promoted through various channels. This includes communication of project concepts and results to a wider audience beyond academically and commercially interested stakeholders. During first period, the project results have been presented in several international conferences, scientific papers and public communication events.

Related information

Record Number: 186496 / Last updated on: 2016-07-13