Final Report Summary - ICEBERG (How to estimate costs of poor quality in a Software QA project: a novel approach to support management decisions)
The Iceberg project has the aim of identify the most effective and efficient Quality Assurance strategy in SW development, especially in Telco and Finance industry, in order to minimize the risk related to poor quality of software in production environment and to provide an efficient predictive model to management team especially on IT departments.
In order to reach the main S&T project outputs, ICEBERG fully embeds the characteristics of a challenging multidisciplinary project mobilizing human resources highly skilled on different domains, such as:
A. The 2 research centers, CINI (Italy) and UAH (Spain), provide competences on quality estimation and forecasting models of SW products\processes and of the related costs.
B. The involved SMEs, Assioma.net (Italy) and DEISER (Spain), contribute with highly qualified experiences on testing of SW projects\processes within their clients to be applied to business decision processes management in Telco and Finance sectors.
The project framework focuses and addresses on the following interdisciplinary S&T objectives:
1. By unifying the existing literature (on SW Quality assurance, business decision processes, standards, models and staff certifications), design and develop innovative and effective models to be used to i) understand the cost associated with testing activities in relation with a given quality, ii) understand the cost associated with missing, incomplete or wrong implementation of testing activities/phases and iii) guide the business decision processes on investment to be made for the software testing process. These models describe and contain all the main elements and factors that contribute to determine the cost of quality and will allow finding the right balance between cost and quality.
2. The creation of a database (DB) combining categorization of data collected from literature and from past business (SW) project provided by ANET and DEISER in the Telco and Finance fields.
It includes parameters used within the models (e.g. defect types, testing activities performance) that will be “historicized” to allow for highly accurate analysis as far as SW products are developed. In this way, models are able to take into account the different types of SW projects along with the different types of testing adopted in each SW project, with the associated cost and achieved quality(for both products and the process phases).
3. Models-based Process Definition and Best Practices.
The process for supporting the decisions-making on investment in testing activities, for scheduling and allocating the various testing
activities and effort in each phase and thus ultimately for defining the test plan, will be defined based on the model set outcomes. A set of best practices will also be collected, documented and presented in the ICEBERG handbook in order to assess the effectiveness of the defined process, and to quantify the real benefits from the usage of the models.
4. The development of the proof-of-concept ICEBERG Tool for automating the application of the models-based process, which will receive the required parameters as input and presenting, as output, the prediction results, for aiding the decision-making process.
The final results of the project provides researchers with new research skills and broad horizons in Software Quality Assurance related to models-based process, oriented to support decision-making applicable in Telco and Finance. The models-based process and the proof of concept IT was evaluated on real test cases provided by the industrial partners and extensively described in the ICEBERG handbook.
The project includes an analysis of the state of the art conducted regarding the standards related to software quality assurance; UAH provided a summary of the state of the art referred to metrics available in the literature for measuring both software product and process quality.
References to the internal and external quality of a product (e.g.,reliability functionality, usability,robustness), and regarding the process itself (schedule variance, effort, requirement stability indexes, degree of reuse, defect density) were examined to provide a clear overview of the existing information for the context. ANET provided a study of factors involved in industry decision-making process. This factors must be taken into account during decision-making process. ANET identified the following 7 main factors: Actual Quality Level, Expected Quality Level, Human Factors,Economics, Time, Resources, Process. The IT companies should acquire information about this factors in order to collect some metrics about software projects. This is a robust way to have objective information that can be used during decisional process. The definition of parameters of interest for the definition of the supportive models for decision-making has started by reviewing the current practices of measurement of the basic components (cost/effort, time/schedule and quality) of the Iron Triangle of project management in the available industrial settings. ANET and DEISER have collected data from their customers, trying to group information into useful scenarios to the definition of the model. Considering the quality attributes explored, a general view of the model-based process has been developed. Moreover, some of the key models needed in the QA decision-supporting process have been identified; these have been defined in terms of input/output (i.e. what information they work on and what is the expected output for QA decision support) and are going to be evaluated.
You can read more on website www.iceberg-sqa.eu
In order to reach the main S&T project outputs, ICEBERG fully embeds the characteristics of a challenging multidisciplinary project mobilizing human resources highly skilled on different domains, such as:
A. The 2 research centers, CINI (Italy) and UAH (Spain), provide competences on quality estimation and forecasting models of SW products\processes and of the related costs.
B. The involved SMEs, Assioma.net (Italy) and DEISER (Spain), contribute with highly qualified experiences on testing of SW projects\processes within their clients to be applied to business decision processes management in Telco and Finance sectors.
The project framework focuses and addresses on the following interdisciplinary S&T objectives:
1. By unifying the existing literature (on SW Quality assurance, business decision processes, standards, models and staff certifications), design and develop innovative and effective models to be used to i) understand the cost associated with testing activities in relation with a given quality, ii) understand the cost associated with missing, incomplete or wrong implementation of testing activities/phases and iii) guide the business decision processes on investment to be made for the software testing process. These models describe and contain all the main elements and factors that contribute to determine the cost of quality and will allow finding the right balance between cost and quality.
2. The creation of a database (DB) combining categorization of data collected from literature and from past business (SW) project provided by ANET and DEISER in the Telco and Finance fields.
It includes parameters used within the models (e.g. defect types, testing activities performance) that will be “historicized” to allow for highly accurate analysis as far as SW products are developed. In this way, models are able to take into account the different types of SW projects along with the different types of testing adopted in each SW project, with the associated cost and achieved quality(for both products and the process phases).
3. Models-based Process Definition and Best Practices.
The process for supporting the decisions-making on investment in testing activities, for scheduling and allocating the various testing
activities and effort in each phase and thus ultimately for defining the test plan, will be defined based on the model set outcomes. A set of best practices will also be collected, documented and presented in the ICEBERG handbook in order to assess the effectiveness of the defined process, and to quantify the real benefits from the usage of the models.
4. The development of the proof-of-concept ICEBERG Tool for automating the application of the models-based process, which will receive the required parameters as input and presenting, as output, the prediction results, for aiding the decision-making process.
The final results of the project provides researchers with new research skills and broad horizons in Software Quality Assurance related to models-based process, oriented to support decision-making applicable in Telco and Finance. The models-based process and the proof of concept IT was evaluated on real test cases provided by the industrial partners and extensively described in the ICEBERG handbook.
The project includes an analysis of the state of the art conducted regarding the standards related to software quality assurance; UAH provided a summary of the state of the art referred to metrics available in the literature for measuring both software product and process quality.
References to the internal and external quality of a product (e.g.,reliability functionality, usability,robustness), and regarding the process itself (schedule variance, effort, requirement stability indexes, degree of reuse, defect density) were examined to provide a clear overview of the existing information for the context. ANET provided a study of factors involved in industry decision-making process. This factors must be taken into account during decision-making process. ANET identified the following 7 main factors: Actual Quality Level, Expected Quality Level, Human Factors,Economics, Time, Resources, Process. The IT companies should acquire information about this factors in order to collect some metrics about software projects. This is a robust way to have objective information that can be used during decisional process. The definition of parameters of interest for the definition of the supportive models for decision-making has started by reviewing the current practices of measurement of the basic components (cost/effort, time/schedule and quality) of the Iron Triangle of project management in the available industrial settings. ANET and DEISER have collected data from their customers, trying to group information into useful scenarios to the definition of the model. Considering the quality attributes explored, a general view of the model-based process has been developed. Moreover, some of the key models needed in the QA decision-supporting process have been identified; these have been defined in terms of input/output (i.e. what information they work on and what is the expected output for QA decision support) and are going to be evaluated.
You can read more on website www.iceberg-sqa.eu