Periodic Reporting for period 2 - EAST (Using Evolutionary Algorithms to Understand and Secure Web/Enterprise Systems)
Período documentado: 2022-02-01 hasta 2023-07-31
Nowadays, software affects most parts of life, like banking, healthcare, enterprises, transportation, smartphones, entertainment systems, etc. Unfortunately, writing software is hard, and most of the time systems are shipped with bugs, i.e. functional mistakes. Software testing is used to try to find those bugs, but it is a complex, tedious task. Manual testing is often not systematic, leaving many kinds of faults undetected. Typically, software testing takes up to half of the development time and cost for a system. As of 2013, it is estimated that software testing is costing $312 billions worldwide. Due to its high cost, software testing is often left incomplete, and only applied partially. Besides not following best practices during software development, software bugs can also lead to major security vulnerabilities, which can have dire economic and legal repercussions. For example, based on a survey of 419 companies in 13 countries sponsored by IBM Security in 2017, $3.62 million is the average total cost of a data breach.
With the EAST project, we aim to improve our understanding of the intrinsic characteristics of web/enterprise systems related to their security. We will achieve it by designing novel techniques that are able to scale to automatically generate test cases for large web/enterprise systems, and that can automatically find common types of security threats. This is a necessary stepping stone before reaching the high risk / high impact goal of designing testing systems that can adapt and learn, finding classes of security threats for which currently there is no automated solution due to the oracle-problem.
We will contribute towards this goal by constructing and studying classes of co-evolutionary algorithms that evolve in competition in separate populations of test cases for graphical user interfaces (e.g. web app GUIs) and direct network calls (e.g. HTTP). The tools and techniques developed in the EAST project will be instrumental to study and broaden our understanding of what kinds of security-related mistakes do developers make in practice, and why they are made.
With the EAST project, we aim to improve our understanding of the intrinsic characteristics of web/enterprise systems related to their security. We will achieve it by designing novel techniques that are able to scale to automatically generate test cases for large web/enterprise systems, and that can automatically find common types of security threats. This is a necessary stepping stone before reaching the high risk / high impact goal of designing testing systems that can adapt and learn, finding classes of security threats for which currently there is no automated solution due to the oracle-problem.
We will contribute towards this goal by constructing and studying classes of co-evolutionary algorithms that evolve in competition in separate populations of test cases for graphical user interfaces (e.g. web app GUIs) and direct network calls (e.g. HTTP). The tools and techniques developed in the EAST project will be instrumental to study and broaden our understanding of what kinds of security-related mistakes do developers make in practice, and why they are made.
The work done so far in the project has been fundamental to create the infrastructure to be able to automatically test (e.g. fuzz) large industrial systems. Scientific work has been done to be able to automatically fuzz web services such as REST, GraphQL and RPC, as well as supporting different programming languages with different properties, such as Java and JavaScript. Novel techniques have been designed and successfully evaluated to improve white-box, source code heuristics, published in the top scientific venues in Software Engineering research (e.g. TOSEM).
All this work has been done in a single open-source tool, called EvoMaster (www.evomaster.org). So far, the project has already achieved significant practical results and shown scientific excellence. For example, an independent study comparing 10 different tools shows that our scientific work done in this project achieves the best results in the literature. Furthermore, our work has been used to test tens of industrial systems, with millions of lines of code, from our industrial partners, automatically finding thousands of faults. The tool itself has been downloaded more than 1700 times, at the time of this writing, showing an interest of practitioners in industry for this scientific work.
All this work has been done in a single open-source tool, called EvoMaster (www.evomaster.org). So far, the project has already achieved significant practical results and shown scientific excellence. For example, an independent study comparing 10 different tools shows that our scientific work done in this project achieves the best results in the literature. Furthermore, our work has been used to test tens of industrial systems, with millions of lines of code, from our industrial partners, automatically finding thousands of faults. The tool itself has been downloaded more than 1700 times, at the time of this writing, showing an interest of practitioners in industry for this scientific work.
Based on the current scientific literature, we are confident that we can boldly claim that the work done in this project represents the current state-of-the-art in automated testing of web services. The scientific work done to design scalable and adaptable search algorithms allowed us to support all major kinds of web services, providing best results in independent studies, and achieve already practical results on large industrial systems of millions of lines of code.
The work done so far has been focused on functional testing, providing the needed scaffolding to start to tackle the main scientific challenge of this project: security testing for classes of vulnerabilities for which there is no existing automated solution. Based on the success so far, there is a good chance of providing ground-breaking scientific results by the end of project.
The work done so far has been focused on functional testing, providing the needed scaffolding to start to tackle the main scientific challenge of this project: security testing for classes of vulnerabilities for which there is no existing automated solution. Based on the success so far, there is a good chance of providing ground-breaking scientific results by the end of project.