Periodic Reporting for period 4 - PASS (Program Analysis for Safe and Secure Software Evolution)
Okres sprawozdawczy: 2024-04-01 do 2024-12-31
Constant evolution is an inherent property of modern software systems. Software evolves to implement new features, adapt to new hardware and platforms, fix bugs and security vulnerabilities, or improve non-functional properties such as performance and energy consumption.
If not carefully validated, many of these changes introduce bugs and security vulnerabilities, often with disastrous consequences. Well-known bugs such as Heartbleed or Shellshock did not exist in the original system, but were instead introduced by individual software changes (usually called “patches”) which were insufficiently vetted. This is not surprising, as reasoning manually about all possible new behaviours introduced by a patch is hard. As a result, developers often deploy patches without writing a single test exercising them.
The objective of the ERC Consolidator Grant "PASS: Program Analysis for Safe and Secure Software Evolution" was to develop novel program analysis techniques that allow the safe and secure evolution of software systems. Such techniques enable the comprehensive testing and validation of software changes, ensuring correct and secure software updates.
If not carefully validated, many of these changes introduce bugs and security vulnerabilities, often with disastrous consequences. Well-known bugs such as Heartbleed or Shellshock did not exist in the original system, but were instead introduced by individual software changes (usually called “patches”) which were insufficiently vetted. This is not surprising, as reasoning manually about all possible new behaviours introduced by a patch is hard. As a result, developers often deploy patches without writing a single test exercising them.
The objective of the ERC Consolidator Grant "PASS: Program Analysis for Safe and Secure Software Evolution" was to develop novel program analysis techniques that allow the safe and secure evolution of software systems. Such techniques enable the comprehensive testing and validation of software changes, ensuring correct and secure software updates.
To comprehensively test and validate software patches, our work has focused on four main directions: (1) reusing the work performed while analysing previous versions of the system; (2) targeting the analysis toward the code impacted by the patch; (3) preventing software to become immune to the testing approach as it evolves; and (4) allowing software to be reliably updated.
Since patches are relatively small in size, each version of the program is often similar to the previous version. As a result, starting program analysis from scratch every time a new version is made available would lead to the analysis of many similar paths or path prefixes, unnecessarily wasting computation time. In our work, we have designed approaches that facilitate the reuse of previous analysis effort.
Targeting the analysis toward the code impacted by the patch is of key importance for a timely validation of software patches, and we have expended significant effort in this line of research. We have designed techniques that use effective search heuristics to guide the analysis, worked on a novel approach for soundly skipping the code unrelated to the patch, experimented with combinations of complementary program analysis techniques (such as static analysis and dynamic symbolic execution) and made significant improvements to dynamic symbolic execution, a program analysis technique used in our research.
Despite the initial success of several testing techniques proposed in the past, software can become immune to some of them. We have investigated approaches for addressing this problem in the context of compiler fuzzing. Our technique relax several constraints imposed by the prior state of the art, resulting in more effective approaches that can detect new compiler bugs.
To achieve a focused and deep analysis of the behavioural differences introduced by software changes, one needs to run the old and the new versions side-by-side. PASS has developed a novel way to accomplish this important objective, by adapting the concept of product programs (which have been traditionally used to reason about hyper-properties such as non-interference) to the domain of software evolution. This product program-based analysis is enhanced by patch specifications, which relate the states of the old and new software versions.
We have also proposed a novel approach for monitoring and modifying program execution, which can be used to improve the way software updates are applied, as well as facilitate other important software engineering techniques such as fault injection and fuzzing. An important approach we have developed enables a more effective testing of network protocol implementations, which are of critical importance to our society.
Our research has been published in some of the top venues in our area, such as the International Symposium on Software Testing and Analysis, the International Conference on Automated Software Engineering, and the Joint Meeting of the European Software Engineering Conference and the Symposium on the Foundations of Software Engineering.
PASS has helped grow and maintain KLEE, a software tool used by a large number of groups from academia and industry (https://klee-se.org/(odnośnik otworzy się w nowym oknie)). We have also organised three international workshops during the grant, the 2nd, 3rd and 4th International KLEE Workshop on Symbolic Execution, which have brought together a large audience from across the globe, spanning academia, industry and government.
The research supported by this grant has also been recognised by the international community by the prestigious IEEE TCSE New Directions Award, received by the PI, Prof. Cristian Cadar (jointly with Prof. Abhik Roychoudhury from National University of Singapore). This award is given for "substantial contributions to software engineering research or practice where these contributions moved the field in a new direction."
Since patches are relatively small in size, each version of the program is often similar to the previous version. As a result, starting program analysis from scratch every time a new version is made available would lead to the analysis of many similar paths or path prefixes, unnecessarily wasting computation time. In our work, we have designed approaches that facilitate the reuse of previous analysis effort.
Targeting the analysis toward the code impacted by the patch is of key importance for a timely validation of software patches, and we have expended significant effort in this line of research. We have designed techniques that use effective search heuristics to guide the analysis, worked on a novel approach for soundly skipping the code unrelated to the patch, experimented with combinations of complementary program analysis techniques (such as static analysis and dynamic symbolic execution) and made significant improvements to dynamic symbolic execution, a program analysis technique used in our research.
Despite the initial success of several testing techniques proposed in the past, software can become immune to some of them. We have investigated approaches for addressing this problem in the context of compiler fuzzing. Our technique relax several constraints imposed by the prior state of the art, resulting in more effective approaches that can detect new compiler bugs.
To achieve a focused and deep analysis of the behavioural differences introduced by software changes, one needs to run the old and the new versions side-by-side. PASS has developed a novel way to accomplish this important objective, by adapting the concept of product programs (which have been traditionally used to reason about hyper-properties such as non-interference) to the domain of software evolution. This product program-based analysis is enhanced by patch specifications, which relate the states of the old and new software versions.
We have also proposed a novel approach for monitoring and modifying program execution, which can be used to improve the way software updates are applied, as well as facilitate other important software engineering techniques such as fault injection and fuzzing. An important approach we have developed enables a more effective testing of network protocol implementations, which are of critical importance to our society.
Our research has been published in some of the top venues in our area, such as the International Symposium on Software Testing and Analysis, the International Conference on Automated Software Engineering, and the Joint Meeting of the European Software Engineering Conference and the Symposium on the Foundations of Software Engineering.
PASS has helped grow and maintain KLEE, a software tool used by a large number of groups from academia and industry (https://klee-se.org/(odnośnik otworzy się w nowym oknie)). We have also organised three international workshops during the grant, the 2nd, 3rd and 4th International KLEE Workshop on Symbolic Execution, which have brought together a large audience from across the globe, spanning academia, industry and government.
The research supported by this grant has also been recognised by the international community by the prestigious IEEE TCSE New Directions Award, received by the PI, Prof. Cristian Cadar (jointly with Prof. Abhik Roychoudhury from National University of Singapore). This award is given for "substantial contributions to software engineering research or practice where these contributions moved the field in a new direction."
At the time when PASS started, most program analysis research such as that on symbolic execution and fuzzing focused on analysing entire programs, and as such, were often not scalable enough to keep pace with the quick evolution of modern software. PASS has contributed several novel program analysis techniques that target evolving software, making it easier to evolve software safely and securely.