The objective of the experiment Application of an Integrated, Modular, Metric Based System and Software Test Concept is to put into practice an efficient, state-of-the-art test approach and environment, which is modular and thus scalable to projects of different size and with different requirements regarding reliability, availability, maintainability, and safety. The experiment shall prove whether the application of state-of-the art system and software test methods, procedures, and tools results in higher quality software systems. Furthermore the experiment shall prove whether the application of a well defined, state-of-the-art test approach results in measurable improvements regarding reliability, availability, maintainability, and safety of the software systems produced within the framework of this experiment. Special attention shall be given to the tight integration of the test concept into the overall system and software development life cycle, considering the necessary interfaces regarding the methods, procedures, and tools utilised during the different life cycle phases. Finally, this application experiment shall lead to the integration between state-of-the-art software and system test environments and off-the-shelf toolsets performing reliability, availability, maintainability, and safety analysis.
The test approach, consisting of a test concept and a supporting test environment, shall cover all test phases starting from the coding and debugging phase, over the module, integration, and system test phase, onto the acceptance test phase. For all test phases, state-of-the-art test methods, procedures, and tools shall be defined and applied. In particular, it is foreseen to provide projects with tools supporting the test specification, preparation, performance, and evaluation of regression tests on module, component, subsystem, and system level. In addition, the tools foreseen support the measurement and analysis of the achieved test coverage. These methods and tools utilised by the different baseline projects are supplemented by static and dynamic software quality analysis activities and rams (reliability, availability, maintainability, safety) analysis activities performed centralised for all baseline projects by appropriate specialists.
The broad range of projects and users selected for this application experiment shall ensure the definition of a modular, scalable test concept and test environment. Thus, based on the evaluation of certain project criteria, the necessary components of the test concept and the test environment can be derived. Therefore, the test concept and environment can be adapted to each project involved in the application experiment, ensuring transferability to future projects within the organisation participating in the application experiment as well as to projects within organisations outside of this consortium.