Harmonised Assessment of Reliability of MOdern Nuclear I&C Software

Final Report Summary - HARMONICS (Harmonised Assessment of Reliability of MOdern Nuclear I&C Software)

Executive Summary:
Nuclear energy represents an important part of the energy mix in Europe, and maintaining the strictest safety standards is paramount for the sector's sustainability. In this context, instrumentation and control (I&C) systems represent the central nervous system of a nuclear power plant, monitoring all aspects of its operation and responding with the required adjustments.

The EU-funded project 'Harmonised assessment of reliability of modern nuclear I&C software' (HARMONICS) supported the nuclear industry in exploiting and evaluating software of high-tech safety systems. Keeping in mind the global nature of nuclear energy, the project aimed to propose practical methods to verify software through tight collaboration between five EU countries and also a corresponding Chinese project.

To achieve its aims, the project team designed new software verification tools and methods. It began by outlining the needs, practices and experiences in the EU and China, developing common approaches to assessing reliability and justifying safety-critical software. This involved analysing end-user needs and conducting case studies to validate relevant software.

In the process of developing software verification methods and tools, the project team assessed justification frameworks for software-based systems and outlined approaches to quantify software failure rates. It deployed formal methods, statistical testing and complexity analysis to assess software-based systems, as well as an analytical approach to quantify software reliability. The above approaches and methods were tested in case studies and the results assessed to ensure their validity.

The results from HARMONICS are expected to enhance plant efficiency and upgrade safety through new digital I&C technologies and methods. Licensing of digital I&C systems will become more transparent and cost efficient, while harmonisation in nuclear I&C among EU countries and beyond will facilitate sharing of best practices. Lastly, the project's results are also expected to affect competitiveness in digital I&C technologies and solutions on the market.

Project Context and Objectives:
The demand of cost-effectively and reliably produced CO2 free energy is increasing. Dependence on fossil fuels and concerns over climate change are coinciding to make the case for increasing use of safe and reliable nuclear power. This means, in the near future, the construction of new nuclear power units, and the upgrade for lifetime extension of many existing units.

The reliability and safety of the computer-based systems that implement safety functions are critical issues. This is in particular due to the fact that software can usually not be proven to be completely defect-free, and that postulated residual defects could be suspected of leading to common-cause failure that could defeat redundancy and defence-in-depth. Unfortunately, the differences in current safety justification principles and methods restrict co-operation and hinder the emergence of widely accepted best practices. They also prevent cost sharing and reduction, and unnecessarily increase licensing uncertainties, thus creating a very difficult operating environment for utilities, vendors and regulatory bodies. Relevance of I&C issues is addressed in the Strategic Research Agenda (SRA) of the EC Sustainable Nuclear Energy Technology Platform (SNETP 2009).

Given the experience with nuclear-related and software-based systems worldwide, there is now the possibility of using empirical reliability data in a way that has not been feasible before. In addition, advances in computer power and formal verification techniques means that simulated experience and formal verification are becoming more practicable as forms of evidence. This evidence could have an important role in the assurance of nuclear I&C systems. Advances have also been made, and practical experience gained, in several other domains, such as defensive measures to tolerate postulated residual software faults, and safety justification frameworks.

Objectives

The overall objective of the HARMONICS project is to ensure that the nuclear industry has well founded and up-to-date methods and data for assessing software of computer-based safety systems of Gen-II and Gen-III NPPs. It has taken advantage of the aforementioned advances to propose systematic and consistent, yet realistic and practical approaches for software assessment. These approaches address critical phases of the software and system lifecycles, from requirements specification to verification and validation.

In addition to the core project team, a larger “end user group” has been constituted with other interested stakeholders (utilities, regulatory bodies, suppliers) to review and give feedback on the project work. Thus, the project has fostered an international consensus based on a sound scientific and technical approach, and hopefully provides a good basis for harmonisation.

Scope

The project addresses three key issues: software verification & validation (V&V), software safety justification, and quantitative evaluation of software reliability. The term “software reliability” is used throughout this document as a shortcut for “software-related aspects of system reliability”. The focus has been mainly on I&C systems performing category A functions (as defined by IEC 61226) which is the highest safety category in NPP. To support research activities on these three main issues, the project has investigated and developed theories, techniques and tools as necessary. In addition, the feasibility of the developed approaches has been experimented and demonstrated with selected case examples provided by the project participants and the end user group.

Regarding software V&V, the project has analysed the state of the art, proposed innovative techniques and tools, and provided practical guidelines for applying some of these techniques and tools. V&V may be used to ascertain the effective implementation of fault avoidance measures, such as compliance to complexity limits, to design and coding rules, to specified development processes and methods. V&V may also be used for fault detection (for their subsequent removal), by applying techniques such as simulation and testing, formal verification, inspection. Lastly, V&V may be used to ascertain the effective implementation of design measures taken to guarantee that certain types of postulated residual software faults will not lead to failures or common cause failures.

Regarding software safety justification, the HARMONICS project has built on current practices and on results of previous Euratom FP6 research projects, namely CEMSIS (Cost-Effective Modernisation of Systems Important to Safety) and BE-SECBS (Benchmark Exercise on Safety Evaluation of Computer Based Systems). In particular, it proposes a framework integrated into the overall system safety justification, and based on the complementarity and integration of the rule-based, the goal-based and the risk-informed approaches. In particular, the project has analysed the domain of applicability and acceptability of each approach, and provides practical guidelines based in particular on the information gathered with the proposed V&V techniques.

Regarding software reliability, the framework integrates quantitative software reliability claims in the overall software and system safety justification. In particular, the project has investigated the nature and justification for any reliability claim limit. It also proposes practical approaches to estimate the values needed for Probabilistic Safety Assessments (PSA): probabilities of failure on demand, conditional probabilities of common cause failures (so-called beta-factors), and possibly frequencies of spurious actuations that lead to initiating events. To this end, the project has analysed the current state of the art, which is usually based on holistic approaches (e.g. conformance to international standards, collection and analysis of operating experience, statistical testing and corresponding trade-offs between realism and scale of tests). It also proposes a more analytical approach that takes into consideration all the information obtained by V&V and organised by the software safety justification. This approach can be based on the identification of failure modes of interest, of the failure mechanisms that could lead to these modes, and on the effectiveness of the measures taken to prevent given mechanisms. It will also consider the implication of I&C architectures (levels of defence and diversity) and implementation technologies in the system safety justification.

Project Results:
WORK PROGRAMME & WORK PERFORMED

The overall strategy to reach the objectives is based on the following steps of activities:

• Clarification of needs, practices and experiences. Exchange of information between EU and China. Specification of the scope, target and objectives of the approach to be developed in the next step. This activity shall facilitate the co-operation between EU and China partners.
• Development of common approaches to the assessment and justification of the reliability of safety software. The project will take advantage of the experience from the recent licensing processes and research projects (including EU FP5 projects BE-SECBS and CEMSIS). The approach will be based on the Claim-Argument-Evidence approach.
• Test of the approaches in case studies.
• Assessment of results from the case studies. Critical assessment applicability of approaches and lessons learnt.
• Dissemination of results. An important part of the whole project is to get feedback from a large group of advisories and end users. The advisory and end user group will include utilities, regulators and vendors from EU and China. Advisory and end users workshops will be arranged. Contacts with international organisations like IAEA, OECD/NEA will be utilised.

The research activities were carried out by two parallel projects (the EU project HARMONICS and the corresponding Chinese project RAVONICS). For various reasons, RAVONSICS has been unable to start in due time, and cooperation between the two projects has been limited.

WP1 - Needs, practices, experiences

The objectives of the WP1 were:
• To reduce the technical gaps between Chinese and European participants on safety-critical software reliability assessment and V&V.
• To establish a common base for the project by exchanging information, knowledge, and experiences
• To clarify the needs of China and Europe for safety-critical software reliability and V&V.

A kick-off workshop was organized in Shanghai, China at the beginning of the project. After that, HARMONICS prepared a needs analysis questionnaire that was sent to over 70 nuclear I&C experts in Europe representing utilities, regulators, vendors, research organizations and TSOs. A needs analysis report was issued based on the answers received, and also on the discussions in the first End User workshop organized in Helsinki, Finland. The results of the analysis served as a basis to determine priorities for WP2.

WP2 - Methods development

Work on methods development focused on the following topics:

• Confidence in functional and timing requirements specification: experience in the nuclear industry and in other safety critical industries has shown that flaws in functional and timing requirements could result in undesirable behaviours that are detected late in the development process, or worse, during operation.
• Complexity and structural analysis of functional diagrams: the purpose here was to identify the diagrams of higher complexity than average and their relationships with the other diagrams, in order to help determine an appropriate verification and validation approach.
• Formal software verification: an approach to the formal verification of the complete software of a class 1 digital I&C system (implementing Category A functions) has been developed

• Safety justification framework, which provides:
1. A set of principles.
2. A strategy based on understanding the behaviour of the system and its interactions
3. An approach to developing, communicating and challenging the understanding based on claims, argument and evidence (CAE).
4. Guidance on the underlying concepts of CAE and an extension of them into generic “blocks” that provide the basis for domain and problem specific templates.
5. A high level process for deploying the framework and detailed guidance on specific issues.

• Formal software verification: an approach to the formal verification of the complete software of a class 1 digital I&C system (implementing Category A functions) has been developed and expressed in the format of the Safety justification framework.
• Quantification of software reliability: the approach developed provides justifiable reliability numbers for software of computer-based safety systems in nuclear power plants. These numbers may be used to represent the software-related aspects of the computer-based systems in probabilistic safety assessments (PSA).
• Stepwise Shutdown System: development of a simple case study that can serve as the basis for the public case study.

WP3 - Case studies

5 case studies were made to help develop the methods and to confirm their applicability:

• A Backup Power Supply (BPS) case study has been made to develop the proposed approach for improving confidence in functional and timing requirements.
• The complexity and structural analysis was test cased on a complete, real-life application (not of Category A).
• Formal software verification and the safety justification framework have been test cased on the complete software of a class 1 digital I&C system.
• A case study was developed to illustrate how the Claims-Argument-Evidence (CAE) structuring approach and the research on reliability can be used to provide a template and discussion of the issues that need to be addressed in justifying software reliability, using a combination of techniques (including statistical testing).
• Stepwise Shutdown System: development of a simple case study that serves as the basis for the public case study.

WP4 - Methods assessment

WP4 had two main objectives:

• To assess the results of the approaches and methods developed by WP2, based on the case studies of WP3.
• To present recommendations to improve the proposed approaches to safety justification.

The evaluation was performed in several steps during the project. The first step was made when different verification methods were reported in D2.1 Verification methods. The report contains the first impressions and already existing experiences of the applicability of the methods.

The second step was the analysis of the results of the case studies where the developed approaches were utilized in practical level to understand and demonstrate how they should be used, for what kinds of problems they can be utilized, who should use them, in what project phase they should be used and how much effort the utilization takes.

The third step was the evaluation done by the HARMONICS End User Group. The methods and the case studies were presented at the second HARMONICS End User Workshop, and each participant filled in an evaluation form. In addition to individual evaluations and feedback, several working groups consisting of the end users (workshop participants) were formed to discuss the topic and finally present questions, ideas, feedback, open issues and conclusions on the approaches.

The criteria that the end users used for the evaluation of the proposed approaches for safety justification can be divided in two categories: the evaluation of the case study itself and the evaluation of the methods and tools. The criteria are summarized below.

Case studies:

• How the case study represents the problems and challenges of the respondent?
• Does the case study illustrate the proposed method and its usage?
• Is the case study extensive enough?
• Is the case study too complex to understand and digest?

Method and tools:

• Are the presented methods and tools useful in the context of the respondent?
• Are the presented methods and tools applicable in real projects?
• What kind of improvement they would bring to the current practice?
• What kind of drivers for, or barriers to adoption of the presented methods and tools there are?

In addition to the predefined questions of the evaluation form, the end users were also asked to make suggestions and remarks to widen the perspective and to cover issues not addressed by the evaluation form.

Working in groups and discussing with other experts clarified the overall understanding on the current utilization of the safety justification approaches in the European organizations. Despite the positive development on the adoption of the presented approaches and methods during the last few years, there are still several topics that need more explanation, education and development to bring them to standard practice in European organizations.

WP5 - Dissemination

The objective of the WP5 was to inform all potential end users of the results of the project, and to promote the use of the methods and tools proposed. The dissemination actions can be divided into three main categories:

• Communication actions, where the project results are presented in professional journals, conferences, lectures, curriculum in colleges, etc. A Dissemination Plan was developed to identify and maintain a list of communication actions throughout the duration of the project. (For more information on these communication actions, see D5.6 Journal articles, conference papers, other publications.)
• The development of a Public Case Study. This action involved all project participants. Its objective was to document the complete approach proposed by the project in a report (D5.4 Public Case Study) not be restricted by any NDA.
• Organisation of End Users Workshops to inform the end user community. One has been organised early in the project, and another at the end of the project. (For more information on these workshops, see D5.3 End User Workshop 1 Proceedings and D5.5 End User Workshop 2 Proceedings.)

RESULTS

Interactions with stakeholders

The interactions with stakeholders are formalised in four HARMONICS deliverables: D1.2 D5.3 D5.5 and D4.2.

D1.2: Needs analysis report

Deliverable D1.2 gives a summary of the answers of the questionnaire sent to the identified end users in the beginning of the project to clarify the current practices and needs of European organizations for safety-critical software reliability and V&V. When appropriate, a reference to the "Common position of seven European nuclear regulators and authorised technical support organisations, ver. 2010" is made as a point of comparison. The synthesis of the answers also reflects the summary of a similar type of questionnaire from the previous Euratom FP6 project CEMSIS. This deliverable is not public. Its table of contents is as follows:

1 Introduction
2 Objectives of the questionnaire
3 Question categories
4 Synthesis of the answers
5 Analysis of the answers
6 Reflections on HARMONICS and CEMSIS questionnaires
7 Reflections on 1st HARMONICS End User Workshop
8 References

D5.3: End User Workshop 1 proceedings

This deliverable is public and can be downloaded from the HARMONICS website. Its table of contents is as follows:

1 Introduction
2 Glossary
3 Workshop Objectives
4 Participants
5 Workshop Agenda
6 List of Presentations
7 Summary of Discussions
7.1 HARMONICS - Overview
7.2 RAVONSICS - Brief Introduction
7.3 HARMONICS - Overall WP1-WP4 Presentation
7.4 HARMONICS WP1 - Needs, Practices, Experiences
7.5 HARMONICS WP4 - Assessment Criteria
7.6 Group Discussion
7.6.1 Group A
7.6.2 Group B
7.6.3 Group C
7.6.4 Group D
7.7 Round table discussion
7.8 Summary of Day 1
7.9 The EU Framework Programme for Research and Innovation 2014-2020
7.10 RAVONSICS - Introduction
7.11 HARMONICS WP2 Methods - Safety Justification Framework
7.12 HARMONICS WP2 Methods - Quantification of Failure Probabilities
7.13 Comments from a Regulator
7.14 HARMONICS WP2 Methods - Improving Functional Requirements Specification
7.15 HARMONICS WP2 Methods - Statistical Testing
7.16 HARMONICS WP2 Methods - Structure & Complexity of Digital I&C
8 Conclusion: Round Table Comments from the EUG

APPENDIX 1 - Presentation Slides

D5.5: End User Workshop 2 proceedings

This deliverable is public and can be downloaded from the HARMONICS website. Its table of contents is as follows:

1 Introduction
2 Glossary
3 Workshop Objectives
4 Participants
5 Workshop Agenda
6 List of Presentations

APPENDIX 1 - Welcome
APPENDIX 2 - Safety Justification Framework - Introduction
APPENDIX 3 - Software Reliability Quantification - Introduction
APPENDIX 4 - Confidence in Requirements Specification - Introduction
APPENDIX 5 - Formal Software Verification - Introduction
APPENDIX 6 - Stepwise Shutdown System: Introduction
APPENDIX 7 - Safety Justification Framework - Some Viewpoints and Observations
APPENDIX 8 - Safety Justification Framework
APPENDIX 9 - Confidence in Requirements Specification
APPENDIX 10 - Confidence in Requirements Specification: Formal Verification
APPENDIX 11 - Stepwise Shutdown System
APPENDIX 12 - Software Reliability Quantification
APPENDIX 13 - Formal Software Verification

D4.2: Recommendations to improve the proposed approach to safety justification

This deliverable presents the recommendations to improve the proposed approaches to safety justification received from the project's end user group consisting of various representatives of licensees, utilities and nuclear regulators. It is not public. Its table of contents is as follows:

1 Introduction
2 Abbreviations
3 Evaluation process and criteria
3.1 Process
3.2 Criteria
4 Evaluation results and recommendations
4.1 Confidence in requirements specifications
4.1.1 Approaches / methods evaluated
4.1.2 Results
4.1.3 Recommendations
4.2 Formal verification
4.2.1 Approaches / methods evaluated
4.2.2 Results
4.2.3 Recommendations
4.3 Safety justification framework
4.3.1 Overview
4.3.2 Opportunities and barriers and feedback
4.3.3 Recommendations and conclusions
4.4 Reliability quantification
4.4.1 Approaches / methods evaluated
4.4.2 Results
4.4.3 Recommendations
4.5 Stepwise shutdown system
4.5.1 Approaches / methods evaluated
4.5.2 Results
4.5.3 Recommendations
4.6 Complexity analysis
5 Conclusions

Appendix A – List of participants per session

Summary of scientific deliverables

The technical results are formalised in five HARMONICS deliverables: D2.1 D2.2 D2.3 D3.2 and D5.4.

D2.1: Verification methods

In order to understand some of the motivation and application of the deliverable, an introduction to the overall approach to safety justification is provided. The details of the approach– the safety justification framework– is the subject of deliverable, D2.2. The technical aspects of the Verification Methods are summarised and described in more detail in the Annexes. Conclusions are made and future work discussed in Section 5.

Most of the substantive technical work is described in the Annexes. Annex 1 addresses the High-Level Functional & Timing I&C System Requirements that form the basis for assessing whether the system behaves as required. Annexes 2 and 3 on Formal Analysis and Annex 4 on Statistical Testing provide details of these two approaches, which provide direct evidence of how the product behaves. Annex 5 on Complexity Analysis describes an approach that provides some information on structural properties of the software.

This deliverable is not public. Its table of contents is as follows:

1 Introduction
2 Harmonics context
3 Overall Safety Justification
3.1 Structuring a safety justification
3.2 Safety justification strategies
3.3 Construction of a safety justification
4 Assessing behaviour
4.1 High level requirements
4.2 Formal analysis
4.3 Statistical testing
4.4 Complexity analysis
5 Discussion and conclusions
5.1 Next steps
6 Definitions and Abbreviations
6.1 Definitions
6.2 Abbreviations
7 References

Annex 1 High-Level Functional & Timing I&C System Requirements
A1.1 Introduction
A1.2 Background
A1.3 Description of Method
A1.4 Comments
A1.5 Roles in Safety Justification Triangle

Annex 2 Formal Verification
A2.1 Introduction
A2.2 Background
A2.3 Description of the Method
A2.4 Comments
A2.5 Roles in Safety Justification Triangle

Annex 3 Formal verification – Model Checking
A3.1 Introduction
A3.2 Background and scope
A3.3 Description of model checking
A3.4 Role of model checking in safety justification
A3.5 Issues

Annex 4 Statistical Testing
A4.1 Introduction
A4.2 Background to statistical testing
A4.3 Method
A4.4 Roles in safety justification triangle
A4.5 Summary and conclusions
A4.6 Addendum: Dealing with non-determinism

Annex 5 Complexity analysis
A5.1 Introduction
A5.2 Background to Complexity Analysis
A5.3 Complexity Analysis / Method
A5.4 Complexity Analysis in Safety Justification
A5.5 Benefits and Limitations
A5.6 Process and Application

D2.2: Safety justification framework

This deliverable defines a framework for justifying the use of software in systems implementing Category A nuclear functions. The framework provides
• A set of principles.
• A strategy based on understanding the behaviour of the system and its interactions.
• An approach to developing, communicating and challenging the understanding based on claims, argument and evidence (CAE).
• Guidance on the underlying concepts of CAE and an extension of them into generic “blocks” that provide the basis for domain and problem specific templates.
• A high level process for deploying the framework and detailed guidance on specific issues.
Additional deliverables contain more guidance on the techniques deployed in the framework (D2.1 on verification techniques and D2.3 on software reliability). In addition, there are supporting case studies in D3.2 and a public domain case study in D5.4.

This deliverable is not public. Its table of contents is as follows:

1 Introduction
2 Objectives and overview
3 Safety principles
3.1 IAEA and UK Safety Assessment Principles
3.2 The Harmonics safety justification principles
4 Focus on behaviour - the strategy triangle
4.1 Overall strategy
4.2 Property-based approach
4.3 Vulnerability-based approach
4.4 Standards compliance
5 Evidence of behaviour
5.1 Introduction
5.2 High level requirements
5.3 Formal analysis
5.4 Statistical testing
5.5 Complexity and structural analysis
6 The role of claims, argument and evidence
6.1 Background
7 Exploring and developing justification options
8 CAE normal form and “blocks”
8.1 Types of claim
8.2 Topology and interconnection rules
8.3 The concept and definition of a CAE “block”
8.4 The collection of blocks
9 Applying the framework
10 Conclusions
11 Acknowledgements
12 Bibliography

Appendix A Introducing CAE concepts
A.1 Background
A.2 Claims
A.2.1 The basic concept
A.2.2 Discussion
A.2.3 Operations on claims
A.2.4 Summary
A.3 Evidence
A.3.1 The basic concept
A.3.2 Evidence contents
A.3.3 Examples
A.3.4 Summary
A.4 Arguments
A.4.1 The basic concept
A.4.2 What constitutes an argument?
A.4.3 Discussion
A.4.4 Summary
A.5 Summary
A.6 References and further reading

Appendix B CAE basic building blocks
B.1 Block 1 – Decomposition
B.1.1 Definition
B.1.2 Application and guidance
B.1.3 Double decomposition by object and property
B.1.4 Decomposition by architecture
B.1.5 Decomposition by system functions
B.2 Block 2 – Substitution
B.2.1 Definition
B.2.2 Application and guidance
B.3 Block 3 – Evidence incorporation
B.3.1 Definition
B.3.2 Application and guidance
B.4 Block 4 – Concretion
B.4.1 Definition
B.4.2 Guidance and examples
B.5 Block 5 – Calculation
B.5.1 Definition
B.5.2 Application and guidance
B.6 Using and selecting blocks

D2.3: Quantification of software reliability

This deliverable describes the approach developed in HARMONICS to provide justifiable reliability numbers for software of computer-based safety systems in nuclear power plants. These numbers may be used to represent the software-related aspects of the computer-based systems in probabilistic safety assessments (PSA).

This deliverable is not public. Its table of contents is as follows:

1 Introduction
2 Relationship to PSA
3 Analysis of the system architecture
4 Common cause failure modes and defensive measures
5 Software reliability quantification methods
5.1 Statistical testing
5.2 Estimating the “probability of perfection”
5.3 Analysis of prior operational experience (OPEX)
5.4 Indirect reliability estimation methods
5.4.1 Worst case bound theory
5.4.2 Infinite time bound
5.4.3 Bayesian modelling
6 Construction of reliability models
7 Dealing with uncertainty
8 Justification of quantified reliability
9 Application of the guidance
Glossary
References

Appendix 1: Reliability estimation models
1 Chain rule
2 Worst case bound estimate
3 Infinite time bound
4 Probability of perfection
5 Failure dependency based on logic difficulty
6 Beta factor
7 Statistical testing
8 Bayesian belief networks (BBN)

D2.4: Complexity and structural analysis

The overall objective of complexity analysis of the complete application software of a digital I&C system is the identification of complex Function Diagrams (FDs), according to various complexity measurements. The overall objective of structural analysis is the identification of manageable subparts of the complete application software, based on the dependencies between FDs.

Moreover, complexity analysis and its extension to structural analysis support the identification of meaningful and manageable subsets for V&V of the application specific I&C-software.

This deliverable is not public. Its table of contents is as follows:

1 Introduction
2 Complexity Analysis
2.1 Method
2.2 Tools for Complexity Analysis
3 Extensions of Complexity Analysis for Structural Analysis
3.1 Graphical Visualisation of the network of FDs of a complete system
3.2 Identification of Trains
3.3 Inclusion of I/O-Diagrams
3.4 Decomposition of a Complete System in Manageable Parts
3.5 Extended Toolset for Complexity Analysis and Structural Analysis
4 Conclusions
5 References
6 Appendix: Source code listings

D3.2: Case studies

The case studies that were implemented are described in detail in five Appendixes (A to E):
• Appendix A addresses the verification that I&C functional and timing requirements are adequate and complete. The case study is the I&C system of a Backup Electric Power Supply system.
• Appendix B addresses the issue of software complexity analysis. The case study is the complete software of a typical reactor protection system.
• Appendix C addresses the formal software verification of the complete software of a typical reactor protection system. The case presents a formal verification strategy in a Claim-Argument-Evidence structure as suggested by WP2.2 on Safety Justification Framework. Thus, the case study is also an illustration of the results of WP2.2. A number of the formal verification tools necessary to provide evidence have been developed during the HARMONICS project or were pre-existing. However, not all necessary tools are available yet, and further work still needs to be done.
• Appendix D addresses the issue of failure rate assessment for software, based on statistical testing.
• Appendix E illustrates all the HARMONICS results on a simple system, the Stepwise Shutdown System.

This deliverable is not public. Its table of contents is as follows:

1 Introduction
2 Abbreviations
3 The case studies
Summary of the case studies implemented or not implemented

Annex A: Confidence in requirements - The BPS (Backup Power Supply) case study
1 Objectives
2 Method & tools
2.1 Reminder of the HARMONICS approach
2.2 Tools
2.2.1 Simulation
2.2.2 Overview of the FORM-L language
2.2.3 Model Checking
3 Description of the case study
3.1 BPS environment
3.2 BPS requirements in natural language
3.3 BPS overall design
4 Simulation techniques
4.1 Simulation models for the BPS
4.2 Modelling of BPS requirements
4.2.1 General remarks on requirements modelling
4.2.2 Model overview
4.2.3 MPS
4.2.4 Op
4.2.5 SICS
4.2.6 tBC
4.2.7 DG
4.2.8 tComponent
4.2.9 BPS
4.3 Modelling of BPS overall design
4.3.1 General remarks on overall design modelling
4.3.2 Model overview
4.3.3 tComponent
4.3.4 DG
4.3.5 tBreaker
4.3.6 tMSensor, tVoltmeter, tFrequencyMeter
4.3.7 tPSensor
4.3.8 tStep
4.3.9 SICS
4.3.10 DGLS
5 Model checking of the BPS
6 Results
6.1 FORM-L modelling of the BPS requirements
6.1.1 MPS
6.1.2 Op
6.1.3 SICS
6.1.4 tBC and BC
6.1.5 DG
6.1.6 tComponent
6.1.7 BPS
6.1.8 Resetting the BPS requirements model
6.2 FORM-L modelling of the BPS overall design
6.2.1 tComponent
6.2.2 DG
6.2.3 tBreaker
6.2.4 tMSensor
6.2.5 tVoltmeter
6.2.6 tFrequencyMeter
6.2.7 tPSensor
6.2.8 tStep
6.2.9 SICS
6.2.10 DGLS
6.3 UPPAAL verification results
7 Conclusions
8 Bibliography
Appendix: UPPAAL model details

Appendix B: Complexity Analysis
1 Objectives
2 Method & tools
2.1 Complexity Analysis
2.2 Tools for Complexity Analysis
3 Description of the case study
3.1 General
3.2 The Complete Application Software System
4 Results
4.1 Complexity-Vector for each FD of the Complete System
4.2 Overview Information on Size of Function Diagrams
4.3 Identification of Complex Function-Diagrams
4.3.1 Complexity by Size (Number of Elementary Functions)
4.3.2 Differentiation of mid-size FDs by Complexity of Internal Interconnection
4.3.3 Complexity by Variability
4.3.4 Complexity by External Connections
4.4 Structural Information on the Network of FDs of a Complete-System
4.5 Decomposition of a Complete System in Manageable Parts
4.5.1 Concepts for Decomposition
4.5.2 Sensors and related Actuators
4.5.3 Actuator and related Sensors
4.5.4 System Environment of an individual Function Diagram
5 Conclusions
6 References

Appendix C: Formal Software Verification case study
1 Objectives
2 Method & tools
2.1 Divide and conquer
2.2 Safety justification
2.3 Formal verification tools
2.3.1 Simple tools
2.3.2 Tools uncertainties
2.3.3 FRAMA-C
2.3.4 Model checkers
2.3.5 Equivalence checkers
3 Description of the case study
3.1 Object of the case study
3.2 Overall approach
4 Results
4.1 Top level claim TC: the I&C system implements its functional and timing requirements
4.2 Decomposition into CUs and NUs
4.3 CUs and NUs collectively implement the I&C system functional and timing requirements
4.4 Individual CU (or NU) implements its functional and timing requirements
4.5 CU design implements the CU functional and timing requirements
4.6 CU software implements the software functional requirements
4.7 CU source code implements the software functional requirements
4.8 Software architecture
4.9 Scheduling and control flow
4.10 Data Flow
4.11 Each EF used implements its requirements
4.12 AS implements its requirements
4.13 CD is correct
4.14 IBH, OBH, NHC, AMH implement their requirements
4.15 OMH and EMH are not activated
4.16 Auto Tests do not interfere
4.17 CIH implements its requirements
4.18 EH implements its requirements
4.19 Init performs the necessary initialisation
4.20 Executable code is equivalent to C code
5 Conclusion

Appendix D: Justification of a Reliability Quantification Claim
1 Introduction
2 Scenario
3 Justification template for Reliability Quantification
3.1 Deriving top-level claim for reliability
3.1.1 Requirements
3.1.2 Hardware/software decomposition
3.1.3 I&C hardware requirements
3.1.4 Modes of operation
3.1.5 Software architecture decomposition
3.1.6 Summary
3.2 Reliability claim for X1 software pfd
3.3 Justification of the test approach and execution
3.3.1 Realistic model of system use
3.3.2 Adequate test environment
3.3.3 Summary
3.4 Sentencing of results
3.5 Evidence
4 Conclusions
5 References

Appendix F: Stepwise Shutdown System
1 Objectives
2 Method & tools
3 Description of the case study
4 Results
4.1 Functional requirements
4.2 Functional logic design 7
4.3 System design verification through model checking
4.3.1 Plan for model checking
4.3.2 Results of model checking
4.4 Software implementation
4.4.1 Software design plan
4.4.2 Software testing
4.5 FPGA implementation
4.5.1 Hardware
4.5.2 Architectural design
4.5.3 Behavioural description
4.5.4 FPGA simulation
4.5.5 Test coverage
5 Conclusions
6 References

D5.4: Public case study

This deliverable presents the main results of the project in the framework of a public case study that can be freely disseminated. Section 2 presents the main HARMONICS principles. Section 3 provides an overview of the public case study. Section 4 presents an application of complexity and structural analysis. Section 5 showcases the validation of functional and timing requirements specification. Section 6 showcases the formal verification of the software implementation and presents an application of the claim-argument-evidence (CAE) based safety justification framework. Section 7 showcases the quantification of the probability of failure on demand, based on statistical testing.

The deliverable is public and can be downloaded from the HARMONICS website. Its table of contents is as follows:

1 Introduction
2 HARMONICS
2.1 Objectives
2.2 Scope
2.3 Main Principles
3 Overall Presentation of the Public Case Study
3.1 The I&C System
3.2 The Stepwise Shutdown Function
3.3 Overview of the Safety Justification
4 Complexity and Structural Analysis
4.1 What it is?
4.2 Why we do it?
4.3 Case Study
4.3.1 Complexity Analysis
4.3.2 Structural Analysis
4.4 Discussion & Conclusion
5 Validation of Functional & Timing Requirements
5.1 What it is?
5.2 Why we do it?
5.3 How we do it?
5.4 Case study
5.4.1 Identification of the plant systems served by the I&C functions
5.4.2 Natural language specification of the Stepwise Shutdown System process-level
requirements
5.4.3 Formal modelling of the Stepwise Shutdown System process-level requirements
5.4.4 Natural language specification of the Stepwise Shutdown System I&C
requirements
5.4.5 Formal modelling of the Stepwise Shutdown I&C System requirements
5.5 Discussion & Conclusion
6 Formal Software Verification and CAE-Based Justification
6.1 What it is?
6.2 Why we do it?
6.3 How we do it
6.3.1 Claim
6.3.2 Justification
6.3.3 Formal Software Verification Tools
6.4 Case study
6.4.1 Stepwise Shutdown System implements its functional and timing requirements
6.4.2 Stepwise Shutdown System design implements the Stepwise Shutdown System
functional and timing requirements
6.4.3 Stepwise Shutdown System software implements its functional requirements
6.4.4 Stepwise Shutdown System source code implements the software functional
requirements
6.4.5 Etc.
6.5 Discussion & Conclusion
7 Quantification of Failure Rates - Statistical Testing
7.1 What is it?
7.2 Why do we do it?
7.3 Case study
7.3.1 Reliability claim for software pfd
7.3.2 Justification of the test approach – sources of doubt
7.3.3 Analysing the number of successful tests
7.3.4 Sentencing of results
7.3.5 Evidence
7.4 Discussion & Conclusion
8 Conclusion
9 References & Bibliography
10 Glossary

Conclusion

Harmonics has focused on justifying the software of systems implementing Category A nuclear safety functions. As part of this we have proposed a safety justification framework that is based on set of core principles. It is outlined below how the Harmonics principles have been addressed in the project.

(Principle: Contribution of Harmonics)

1. Effective understanding of the hazards and their control should be demonstrated: The development of high-level requirements about system and plant behaviour explicitly addressed in the framework (and supporting case studies in D3.2) and supported by the formal analysis techniques in D2.1. Complexity and structural analysis provides an understanding of the overall software structure and the role of the different parts of the software (D2.4). The approach to take is addressed by property and vulnerability aspects of the strategy triangle in D2.2. The systematic use of CAE can promote and record understanding of the system and its justification (D2.2).

2. Intended and unintended behaviour of the technology should be understood: Supported by the formal analysis techniques of D2.1. Addressed by property and vulnerability aspects of the strategy triangle and the CAE blocks instantiated with relevant claims, can raise the issues to be addressed (D2.2). Complexity and structural analysis provides an understanding of the overall software structure and any unintended dependencies can be identified (D2.4). CCF analysis and the techniques described in D2.3 can provide a means for modelling the risks from unintended interactions.

3. Multiple and complex interactions between technical systems and also human systems to create adverse consequences should be recognised: Formal analysis provides a technique for this (but interaction with human systems outside scope of Harmonics). Statistical testing can reveal unexpected behaviour that might be significant although not the main goal of the testing.

4. Active challenge should be part of decision making throughout the organisation. Needs of all stakeholders to understand and challenge the case should be taken into account in its structure and presentation: The justification framework (D2.2) provides a basis for implementing or reinforcing effective challenge throughout the organisation. It does not include organisational aspects as such. The use of CAE templates provides a structured way for guidance to challenge their use. The use of explicit claims and arguments makes the justification more transparent and should facilitate challenge (D2.2 and D5.4). The underlying process for using the framework includes a challenge and response cycle (D2.2). The use of the analysis techniques and formal analysis (D2.1 D2.4) provide powerful methods for confidence building and challenge.

5. Lessons learned from internal and external sources should be incorporated: Addressed by vulnerability part of triangle and in the use of CAE templates and guidance that incorporate lessons learned (D2.2). Addressed by compliance with standards as they are considered to contain “experience” (but not elaborated in Harmonics as a requirement for compliance for systems performing Cat A functions is assumed).

6. Justification should be logical, coherent, traceable, accessible, repeatable with a rigour commensurate with the degree of trust required of the system: The use of CAE supports a logical, coherent, traceable and accessible approach (D2.2). Templates support repeatability, and the introduction of blocks and templates provides a graduated approach to increasing rigour in a case as does the use of strong formal analysis techniques and statistical testing (D2.2 D3.2 D5.4).

Additional Harmonics deliverables contain more guidance on the techniques deployed in the framework (D2.1 on verification techniques and D2.3 on software reliability). In addition, supporting case studies and a public domain case study have been delivered (D5.4).

It should be born in mind that the above results are the products of research and, while some aspects are mature, the combined approach needs experimentation and validation. There are some aspects that are immature and need more reviewing and trialling.

For further information, contact HARMONICS participants. Contact information can be found at http://harmonics.vtt.fi

Potential Impact:
Results and their potential impact and use

New concepts and technologies such as digital I&C platforms, i.e. software-based systems, have to be employed in nuclear power plants. Common approaches to safety aspects at the European level are needed as cost-cutting measures in the deregulated electricity market will put pressure on utilities. Projects related to nuclear power are typically large and include several international actors having expertise from several different areas. Large scale cost savings as well as improved safety and reliability are hard to achieve on a national level but agreeing on policies internationally has stronger impact.

HARMONICS dealt with essential questions of technical development. The participating organisations represented utilities, technical support organisations, research institutes and regulators, which guaranteed a cross-fertilisation between the various views of safety justification of digital I&C. The views presented will render deeper understanding of the shared problems, which will be further enhanced by the different approaches planned in the project.

HARMONICS will facilitate achieving better efficiency in plant operation and higher level of safety by supporting the use of new digital I&C technologies and methods. Harmonised practices help introduce more consistent and uniform requirements for licensing of digital I&C systems, which will make the licensing process more transparent and cost efficient.

HARMONICS will also increase commonality in nuclear I&C within and between EU countries and also in the rapidly growing nuclear market of China via the Chinese parallel project RAVONSICS. The representatives of RAVONSICS participated in both HARMONICS End User Workshops. Sharing the effort and knowledge within a strong network of European NPP utilities, technical support organisations and regulators will cut R&D costs and contribute to the progress of best practices in verification and validation procedures. The tools and methods for verification and validation of computer-based I&C solutions is a disorganised area. Validating new approaches through case studies focusing on digital I&C technologies is therefore one of the key issues within HARMONICS. The results will increase motivation of manufacturers to promote corresponding digital I&C technologies and solutions on the market.

By developing the adjustable approach in co-operation, HARMONICS and the parallel Chinese project RAVONICS facilitate the harmonisation of software licensing practices within the EU member states and China. The whole nuclear sector will benefit from harmonisation.

Main dissemination activities and exploitation of the results

Deliverable D5.7 is the final public report of the project. It presents the objectives and scope of the project, the project work programme and the work actually performed, and an overview of the project results and deliverables.

The dissemination actions planned in deliverable D5.2 address:

• Organisation of HARMONICS workshops.
• Presentations of HARMONICS results to scientific or professional conferences.
• Papers in scientific or professional journals.
• Integration of HARMONICS results in international standards.
• Integration of HARMONICS results in academic curricula.
• Integration of HARMONICS results in training sessions and training material.

HARMONICS results have been presented in several conferences and workshops in 2012, 2013, and 2014 (see deliverable D5.6 for a complete list):

• 14th International Workshop on Nuclear Safety and simulation, (23-25 October 2012, Harbin, China)
• American Nuclear Society NPIC-HMIT conference (July 22-26, 2012, San Diego, USA)
• Enlarged Halden Programme Group meeting in March 2013.
• NUGENIA forum in March 2013 (a poster presentation).
• VeriSure Workshop on Verification and Assurance (July 14, 2013, Saint Petersburg, Russia) http://fm.csl.sri.com/VeriSure It was next to a big formal methods conference “CAV 2013 - 25th International Conference on Computer Aided Verification” (July 13–19, 2013, Saint Petersburg, Russia) http://cav2013.forsyte.at/
• 6th International Workshop on the Application of FPGAs in NPPs (October 8-11, 2013, Kirovograd, Ukraine)
• International Symposium on Future I&C for Nuclear Power Plants, ISOFIC 2014, 24 - 28 August 2014, Jeju Island, Republic of Korea

In addition, a public case study illustrating the approaches, methods and tools developed in the framework of HARMONICS was developed. It was based on a stepwise shutdown system that is a generalised example application based on a real confidential case and augmented with examples developed from other Harmonics deliverables and Safety Justification Framework.

The public case study is described in deliverable D5.4. A complexity and structural analysis made on a complete system identifies a subsystem that is well isolated from the rest and that can be analysed and justified separately. That subsystem implements a stepwise shutdown function and is therefore called the stepwise shutdown system. The deliverable then showcases the validation of the functional and timing requirements specification, the formal verification of the software implementation and an application of the claim-argument-evidence (CAE) based safety justification framework. Finally, the deliverable showcases the quantification of the probability of failure on demand, based on statistical testing.

The HARMONICS 1st End User Workshop was organized on 17-18 April 2012 in Helsinki, Finland. The workshop findings are reported in deliverable D5.3.

The HARMONICS 2nd End User Workshop was organized on 2-3 April 2014 in Chatou, France. All the participants of the first workshop and also RAVONSICS partners were invited. The workshop findings are reported in deliverable D5.5.

The HARMONICS partners intend to organise a 3rd End User Workshop on 9-11 June 2015, also in Chatou, France, to present the final results of the project and to discuss them with the end users.

The IAEA has started in May 2014 the development of a technical report on the Dependability Assessment of Software for Safety I&C Systems at NPPs. Several members of the HARMONICS project are members of the expert team. Major results from the HARMONICS project have been proposed to the team and have been accepted (approaches to improve confidence in functional requirements, role of formal software verification, safety justification framework). In Technical Meetings, the expert team presents the draft and main concepts to delegates from IAEA Member States, and collects their feedback. The draft proposal has been well-received, including in particular the HARMONICS results. To this date (December 2014), there had been:
o One Consultancy Meeting, May 12-15, 2014 in Vienna (Austria).
o One Technical Meeting, September 23-26, 2014 in Daejeon (Republic of Korea).

List of Websites:
http://harmonics.vtt.fi/

Final Report Summary - HARMONICS (Harmonised Assessment of Reliability of MOdern Nuclear I&C Software)

Related documents

Share this page

Download