Periodic Reporting for period 3 - EPIC (Evolving Program Improvement Collaborators)
Période du rapport: 2020-10-01 au 2022-03-31
The EPIC project combines computational search and machine learning to improve the quality of software and the users experience. The EPIC research team at UCL is supervised by Professor Harman and Dr Sarro, working on the scientific foundations of this approach, while Prof. Harman also works at Facebook, thereby providing avenues for impact and adoption of previously published scientific work.
EPIC is tackling the problem of software bugs, which are time-consuming and expensive to fix using human effort but which, using the team’s techniques, can be automatically identified, fixed and re-tested by a combination of computational search on machine learning. This work is building the scientific foundation for (and developing the practice of) software systems that automatically diagnose and fix their own faults.
EPIC is tackling the problem of software bugs, which are time-consuming and expensive to fix using human effort but which, using the team’s techniques, can be automatically identified, fixed and re-tested by a combination of computational search on machine learning. This work is building the scientific foundation for (and developing the practice of) software systems that automatically diagnose and fix their own faults.
The EPIC team established a rigorous scientific evaluation framework for techniques that automatically find and fix faults in software systems [Zhang et al., TSE 2020] [Guizzo et al., FSE 2020] [Kechagia et al., 2020][Hort et al., 2020][Guizzo et al., ICSE 2021][Asprone et al., ICST 2021]. As many of these automated repair approaches draw on computational search and increasingly machine learning, the team has also provided fundamental scientific foundations for the evaluation of machine learning [Zhang et al., ICML 2020] [Jimenez et. al, FSE 2019] [Zhang et al., ICSE 2021]
[Hort et al., FSE 2021].
EPIC’s work not only has already had a significant impact on society: The results of automatic repair have been applied at scale, where they have automatically found and fixed real software bugs in tens of millions of lines of production software code, improving the communications systems used by over 2.5 billion people worldwide.
The EPIC advanced Research Fellow (Prof Harman) has won many awards and prizes in recognition of this work, as well as other members of his research team, as detailed in this report.
References:
[Guizzo et al., ICSE 2021] Giovani Guizzo, Justyna Petke, Federica Sarro, and Mark Harman: Enhancing Genetic Improvement of Software with Regression Test Selection. 2021 IEEE/ACM 43rd International Conference on Software Engineering (ICSE).
[Asprone et al., ICST 2021] Dario Asprone, Jonathan Metzman, Abhishek Arya, Giovani Guizzo, Federica Sarro: Comparing Fuzzers on a Level Playing Field with FuzzBench. 15th IEEE International Conference on Software Testing, Verification and Validation (ICST'22).
[Hort et al., FSE 2021] Max Hort, Jie Zhang, Federica Sarro, and Mark Harman: Fairea: A Model Behaviour Mutation Approach to Benchmarking Bias Mitigation Methods. ESEC/FSE 2021: Proceedings of the 29th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering
[Zhang et al., ICSE 2021] Jie Zhang, Mark Harman: 'Ignorance and Prejudice' in Software Fairness. (Proceedings) 43th International Conference on Software Engineering
[Zhang et al., TSE 2020] Jie M. Zhang, Mark Harman, Lei Ma, Yang Liu: Machine Learning Testing: Survey, Landscapes and Horizons. IEEE Transactions on Software Engineering (TSE).
[Zhang et al., ICML 2020] Jie M. Zhang, Mark Harman, Benjamin Guedj, Earl T Barr, John Shawe-Taylor: Perturbation Validation: A New Heuristic to Validate Machine Learning Models. Submitted to ICML 2020.
[Zhang et al. ICSE 2020] Jie M. Zhang, Zeyu Sun, Mark Harman, Mike Papadakis, Lu Zhang: Automatic Testing and Improvement of Machine Translation. ICSE 2020 (to appear)
[Marginean ICSE 2019] Alexandru Marginean, J. Bader, S. Chandra, Mark Harman, Yue Jia, Ke Mao, Alexander Mols, A. Scott: SapFix: automated end-to-end repair at scale. ICSE (SEIP) 2019: 269-278
[Jimenez et al.,FSE 2019] Matthieu Jimenez, Renaud Rwemalika, Mike Papadakis, Federica Sarro, Yves Le Traon, Mark Harman: The importance of accounting for real-world labelling when predicting software vulnerabilities. ESEC/SIGSOFT FSE 2019: 695-705
[Kechagia et al., 2020] Maria Kechagia, Sergey Mechtaev, Federica Sarro, Mark Harman: A Comprehensive Empirical Study on Automatic API-Misure Repair. Submitted to IEEE Transactions on Software Engineering (TSE).
[Hort et al., 2020] Max Hort, Maria Kechagia, Federica Sarro, Mark Harman: A Survey of Non-functional Optimization for Mobile Applications. To be submitted to IEEE Transactions on Software Engineering (TSE).
[Colanzi et al., SSBSE 2019] Thelma E. Colanzi, Wesley K. G. Assunção, Paulo R. Farah, Silvia R. Vergilio and Giovani Guizzo: A Review of Ten Years of the Symposium on Search-Based Software Engineering. Proceedings of the International Symposium on Search-Based Software Engineering (SSBSE), pages 42-57, 2019. Tallinn, Estonia.
[Hort et al., FSE 2021].
EPIC’s work not only has already had a significant impact on society: The results of automatic repair have been applied at scale, where they have automatically found and fixed real software bugs in tens of millions of lines of production software code, improving the communications systems used by over 2.5 billion people worldwide.
The EPIC advanced Research Fellow (Prof Harman) has won many awards and prizes in recognition of this work, as well as other members of his research team, as detailed in this report.
References:
[Guizzo et al., ICSE 2021] Giovani Guizzo, Justyna Petke, Federica Sarro, and Mark Harman: Enhancing Genetic Improvement of Software with Regression Test Selection. 2021 IEEE/ACM 43rd International Conference on Software Engineering (ICSE).
[Asprone et al., ICST 2021] Dario Asprone, Jonathan Metzman, Abhishek Arya, Giovani Guizzo, Federica Sarro: Comparing Fuzzers on a Level Playing Field with FuzzBench. 15th IEEE International Conference on Software Testing, Verification and Validation (ICST'22).
[Hort et al., FSE 2021] Max Hort, Jie Zhang, Federica Sarro, and Mark Harman: Fairea: A Model Behaviour Mutation Approach to Benchmarking Bias Mitigation Methods. ESEC/FSE 2021: Proceedings of the 29th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering
[Zhang et al., ICSE 2021] Jie Zhang, Mark Harman: 'Ignorance and Prejudice' in Software Fairness. (Proceedings) 43th International Conference on Software Engineering
[Zhang et al., TSE 2020] Jie M. Zhang, Mark Harman, Lei Ma, Yang Liu: Machine Learning Testing: Survey, Landscapes and Horizons. IEEE Transactions on Software Engineering (TSE).
[Zhang et al., ICML 2020] Jie M. Zhang, Mark Harman, Benjamin Guedj, Earl T Barr, John Shawe-Taylor: Perturbation Validation: A New Heuristic to Validate Machine Learning Models. Submitted to ICML 2020.
[Zhang et al. ICSE 2020] Jie M. Zhang, Zeyu Sun, Mark Harman, Mike Papadakis, Lu Zhang: Automatic Testing and Improvement of Machine Translation. ICSE 2020 (to appear)
[Marginean ICSE 2019] Alexandru Marginean, J. Bader, S. Chandra, Mark Harman, Yue Jia, Ke Mao, Alexander Mols, A. Scott: SapFix: automated end-to-end repair at scale. ICSE (SEIP) 2019: 269-278
[Jimenez et al.,FSE 2019] Matthieu Jimenez, Renaud Rwemalika, Mike Papadakis, Federica Sarro, Yves Le Traon, Mark Harman: The importance of accounting for real-world labelling when predicting software vulnerabilities. ESEC/SIGSOFT FSE 2019: 695-705
[Kechagia et al., 2020] Maria Kechagia, Sergey Mechtaev, Federica Sarro, Mark Harman: A Comprehensive Empirical Study on Automatic API-Misure Repair. Submitted to IEEE Transactions on Software Engineering (TSE).
[Hort et al., 2020] Max Hort, Maria Kechagia, Federica Sarro, Mark Harman: A Survey of Non-functional Optimization for Mobile Applications. To be submitted to IEEE Transactions on Software Engineering (TSE).
[Colanzi et al., SSBSE 2019] Thelma E. Colanzi, Wesley K. G. Assunção, Paulo R. Farah, Silvia R. Vergilio and Giovani Guizzo: A Review of Ten Years of the Symposium on Search-Based Software Engineering. Proceedings of the International Symposium on Search-Based Software Engineering (SSBSE), pages 42-57, 2019. Tallinn, Estonia.
The work on finding and fixing bugs automatically goes considerably beyond the current state of the art. The current state of the art involves techniques for automatically finding faults, but the end to end finding and fixing these faults is a quantum leap beyond the state of the art. This is internationally recognised in the scientific awards for the PI, and the high citation counts, academic recognition of and industrial impact from the research outputs of the team he has led.