The TESTABLE project will materialize its vision by achieving seven objectivess. These objectives are grouped along three major dimensions: methodology, testing, and impact.
The work carried out during Year 1 has partially covered Objective 1. The first goal is the definition of the scope of the project by identifying the testing tools and techniques that will serve as a basis for the TESTABLE experiments. This goal was achieved by delivering the initial set of testabilitypatterns. The Year 1 goal of Objectives 2 and 3 is delivering an assessment of the testability patterns with respect to testing techniques identified further. We achieved that goal by identifying the building blocks of each technique and then evaluating the techniques against testability patterns. Objectives 2 and 4 were also partially covered. Through the definition of the scope of privacy testing, we have established a novel approach toward defining general-purpose privacy testing methodologies that can be later on translated into specific privacy testing tools to unveil, evaluate and quantify (if possible) privacy-related problems. We have defined some general-purpose privacy-testing methodologies that will serve as a basis for developing privacy-testing tools for concrete web applications during the second and third years of the project. Year 1 goal of Objective 2 and 5 is the definition of the scope of ML testing. In this context, we have delivered a comprehensive review of ML vulnerability and evaluation of tools. In addition,we have already identified a set of testability patterns in the ML context.
The work carried out during Year 1 has partially covered Objective 7. The Year 1 main goal is identifying and defining the initial set of case studies to extract requirements
for the activities reported. The identified case studies cover multiple indicators of relevance: eight industrial-size web applications, open-source applications, and live websites.
In the first project year, TESTABLE has achieved all the prefixed objectives in terms of measurable KPIs and scientific output:
- Published eight research papers in top tier academic venues (KPI=6-7/year)
- Delivered >150 testability pattern (KPI=40)
- Tested >5M live websites (KPI=5M)
- Discovered and disclosed >180 vulnerabilities (KPI=25)
- Released one consumer-privacy product (by NOR)
In addition, TESTABLE has initiated the dissemination plan, by kicking off the OWASP Testability Patterns for Web Applications project, under the umbrella of OWASP, with the
goal of creating a self-sustaining community for collecting, archiving, and sharing testability patterns. We have also launched the project website and expanded the digital presence of the project.
