Self-assessment Oracles for Anticipatory Testing

Informacje na temat projektu

PRECRIME

Identyfikator umowy o grant: 787703

DOI

10.3030/787703

Projekt został zamknięty

Data podpisania przez KE 13 Lipca 2018

Data rozpoczęcia 1 Stycznia 2019

Data zakończenia 31 Grudnia 2023

Finansowanie w ramach

• EXCELLENT SCIENCE - European Research Council (ERC)

Koszt całkowity

€ 2 313 250,00

Wkład UE

€ 2 313 250,00

Koordynowany przez

UNIVERSITA DELLA SVIZZERA ITALIANA
Switzerland

CORDIS oferuje możliwość skorzystania z odnośników do publicznie dostępnych publikacji i rezultatów projektów realizowanych w ramach programów ramowych HORYZONT.

Odnośniki do rezultatów i publikacji związanych z poszczególnymi projektami 7PR, a także odnośniki do niektórych konkretnych kategorii wyników, takich jak zbiory danych i oprogramowanie, są dynamicznie pobierane z systemu OpenAIRE .

Rezultaty

Open Research Data Pilot (1)

Data Management Plan

Open Data associated with the empirical evaluation.

Publikacje

Conference proceedings (25)

A Framework for In-Vivo Testing of Mobile Applications

Autorzy: Mariano Ceccato, Davide Corradini, Luca Gazzola, Fitsum Meshesha Kifetew, Leonardo Mariani, Matteo Orru, Paolo Tonella
Opublikowane w: 2020 IEEE 13th International Conference on Software Testing, Validation and Verification (ICST), 2020, Strona(/y) 286-296, ISBN 978-1-7281-5778-8
Wydawca: IEEE
DOI: 10.1109/icst46399.2020.00037

An Empirical Evaluation of Mutation Operators for Deep Learning Systems

Autorzy: Gunel Jahangirova, Paolo Tonella
Opublikowane w: 2020 IEEE 13th International Conference on Software Testing, Validation and Verification (ICST), 2020, Strona(/y) 74-84, ISBN 978-1-7281-5778-8
Wydawca: IEEE
DOI: 10.1109/icst46399.2020.00018

Repairing DNN Architecture: Are We There Yet?

Autorzy: Jinhan Kim, Nargiz Humbatova, Gunel Jahangirova, Paolo Tonella, Shin Yoo
Opublikowane w: Proceedings of the IEEE Conference on Software Testing, Verification and Validation, 2023, Strona(/y) pp. 234-245
Wydawca: IEEE
DOI: 10.1109/icst57152.2023.00030

Model-based exploration of the frontier of behaviours for deep learning system testing

Autorzy: Vincenzo Riccio, Paolo Tonella
Opublikowane w: Proceedings of the 28th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering, 2020, Strona(/y) 876-888, ISBN 9781450370431
Wydawca: ACM
DOI: 10.1145/3368089.3409730

Evolutionary improvement of assertion oracles

Autorzy: Valerio Terragni, Gunel Jahangirova, Paolo Tonella, Mauro Pezzè
Opublikowane w: Proceedings of the 28th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering, 2020, Strona(/y) 1178-1189, ISBN 9781450370431
Wydawca: ACM
DOI: 10.1145/3368089.3409758

Quality Metrics and Oracles for Autonomous Vehicles Testing

Autorzy: Gunel Jahangirova, Andrea Stocco, Paolo Tonella
Opublikowane w: 2021 14th IEEE Conference on Software Testing, Verification and Validation (ICST), 2021, Strona(/y) 194-204, ISBN 978-1-7281-6836-4
Wydawca: IEEE
DOI: 10.1109/icst49551.2021.00030

An Empirical Study on Low- and High-Level Explanations of Deep Learning Misbehaviours

Autorzy: Tahereh Zohdinasab, Vincenzo Riccio, Paolo Tonella
Opublikowane w: Proceedings of the ACM/IEEE International Symposium on Empirical Software Engineering and Measurement (ESEM), 2023
Wydawca: IEEE Computer Society
DOI: 10.1109/esem56168.2023.10304866

DeepHyperion: exploring the feature space of deep learning-based systems through illumination search

Autorzy: Tahereh Zohdinasab, Vincenzo Riccio, Alessio Gambi, Paolo Tonella
Opublikowane w: Proceedings of the 30th ACM SIGSOFT International Symposium on Software Testing and Analysis, 2021, Strona(/y) 79-90, ISBN 9781450384599
Wydawca: ACM
DOI: 10.1145/3460319.3464811

A Review and Refinement of Surprise Adequacy

Autorzy: Michael Weiss, Rwiddhi Chakraborty, Paolo Tonella
Opublikowane w: 2021 IEEE/ACM Third International Workshop on Deep Learning for Testing and Testing for Deep Learning (DeepTest), 2021, Strona(/y) 17-24, ISBN 978-1-6654-4565-8
Wydawca: IEEE
DOI: 10.1109/deeptest52559.2021.00009

Run Java Applications and Test Them In-Vivo Meantime

Autorzy: Antonia Bertolino, Guglielmo De Angelis, Breno Miranda, Paolo Tonella
Opublikowane w: 2020 IEEE 13th International Conference on Software Testing, Validation and Verification (ICST), 2020, Strona(/y) 454-459, ISBN 978-1-7281-5778-8
Wydawca: IEEE
DOI: 10.1109/icst46399.2020.00061

Fail-Safe Execution of Deep Learning based Systems through Uncertainty Monitoring

Autorzy: Michael Weiss, Paolo Tonella
Opublikowane w: 2021 14th IEEE Conference on Software Testing, Verification and Validation (ICST), 2021, Strona(/y) 24-35, ISBN 978-1-7281-6836-4
Wydawca: IEEE
DOI: 10.1109/icst49551.2021.00015

Uncertainty-Wizard: Fast and User-Friendly Neural Network Uncertainty Quantification

Autorzy: Michael Weiss, Paolo Tonella
Opublikowane w: 2021 14th IEEE Conference on Software Testing, Verification and Validation (ICST), 2021, Strona(/y) 436-441, ISBN 978-1-7281-6836-4
Wydawca: IEEE
DOI: 10.1109/icst49551.2021.00056

Simple Techniques Work Surprisingly Well for Neural Network Test Prioritization and Active Learning (Replicability Study)

Autorzy: Michael Weiss, Paolo Tonella
Opublikowane w: Proceedings of the 31st ACM SIGSOFT International Symposium on Software Testing and Analysis, 2022, Strona(/y) pp. 139-150
Wydawca: ACM
DOI: 10.1145/3533767.3534375

DeepCrime: mutation testing of deep learning systems based on real faults

Autorzy: Nargiz Humbatova, Gunel Jahangirova, Paolo Tonella
Opublikowane w: Proceedings of the 30th ACM SIGSOFT International Symposium on Software Testing and Analysis, 2021, Strona(/y) 67-78, ISBN 9781450384599
Wydawca: ACM
DOI: 10.1145/3460319.3464825

DeepMetis: Augmenting a Deep Learning Test Set to Increase its Mutation Score

Autorzy: Vincenzo Riccio, Nargiz Humbatova, Gunel Jahangirova, Paolo Tonella
Opublikowane w: 2021 36th IEEE/ACM International Conference on Automated Software Engineering (ASE), 2021, Strona(/y) 355-367
Wydawca: IEEE Computer Society
DOI: 10.1109/ase51524.2021.9678764

Misbehaviour prediction for autonomous driving systems

Autorzy: Andrea Stocco, Michael Weiss, Marco Calzana, Paolo Tonella
Opublikowane w: Proceedings of the ACM/IEEE 42nd International Conference on Software Engineering, 2020, Strona(/y) 359-371, ISBN 9781450371216
Wydawca: ACM
DOI: 10.1145/3377811.3380353

IFRIT: Focused Testing through Deep Reinforcement Learning

Autorzy: Andrea Romdhana, Mariano Ceccato, Alessio Merlo, Paolo Tonella
Opublikowane w: 2022 IEEE Conference on Software Testing, Verification and Validation (ICST), 2022
Wydawca: IEEE
DOI: 10.1109/icst53961.2022.00013

When and Why Test Generators for Deep Learning Produce Invalid Inputs: an Empirical Study

Autorzy: Vincenzo Riccio, Paolo Tonella
Opublikowane w: Proceedings of the IEEE/ACM 45th International Conference on Software Engineering (ICSE), 2023
Wydawca: IEEE Computer Society
DOI: 10.1109/icse48619.2023.00104

Toward In-Vivo Testing of Mobile Applications

Autorzy: Mariano Ceccato, Luca Gazzola, Fitsum Meshesha Kifetew, Leonardo Mariani, Matteo Orru, Paolo Tonella
Opublikowane w: 2019 IEEE International Symposium on Software Reliability Engineering Workshops (ISSREW), 2019, Strona(/y) 137-143, ISBN 978-1-7281-5138-0
Wydawca: IEEE
DOI: 10.1109/issrew.2019.00063

Hypertesting of Programs: Theoretical Foundation and Automated Test Generation

Autorzy: Michele Pasqua, Mariano Ceccato, Paolo Tonella
Opublikowane w: Proceedings of the IEEE/ACM 46th International Conference on Software Engineering (ICSE), 2024
Wydawca: Association for Computing Machinery
DOI: 10.1145/3597503.3640323

ThirdEye: Attention Maps for Safe Autonomous Driving Systems

Autorzy: Andrea Stocco, Paulo J. Nunes, Marcelo d'Amorim, Paolo Tonella
Opublikowane w: Proceedings of the 37th IEEE/ACM International Conference on Automated Software Engineering (ASE), 2022
Wydawca: ACM
DOI: 10.1145/3551349.3556968

DeepAtash: Focused Test Generation for Deep Learning Systems

Autorzy: Tahereh Zohdinasab, Vincenzo Riccio, Paolo Tonella
Opublikowane w: Proceedings of the 32nd ACM SIGSOFT International Symposium on Software Testing and Analysis (ISSTA), 2023
Wydawca: ACM SIGSOFT
DOI: 10.1145/3597926.3598109

Taxonomy of real faults in deep learning systems

Autorzy: Nargiz Humbatova, Gunel Jahangirova, Gabriele Bavota, Vincenzo Riccio, Andrea Stocco, Paolo Tonella
Opublikowane w: Proceedings of the ACM/IEEE 42nd International Conference on Software Engineering, 2020, Strona(/y) 1110-1121, ISBN 9781450371216
Wydawca: ACM
DOI: 10.1145/3377811.3380395

Simulation-based Test Case Generation for Unmanned Aerial Vehicles in the Neighborhood of Real Flights

Autorzy: Sajad Khatiri, Sebastiano Panichella, Paolo Tonella
Opublikowane w: Proceedings of the IEEE Conference on Software Testing, Verification and Validation, 2023, Strona(/y) pp. 281-292
Wydawca: IEEE
DOI: 10.1109/icst57152.2023.00034

GAssert: A Fully Automated Tool to Improve Assertion Oracles

Autorzy: Valerio Terragni, Gunel Jahangirova, Paolo Tonella, Mauro Pezze
Opublikowane w: 2021 IEEE/ACM 43rd International Conference on Software Engineering: Companion Proceedings (ICSE-Companion), 2021, Strona(/y) 85-88, ISBN 978-1-6654-1219-3
Wydawca: IEEE
DOI: 10.1109/icse-companion52605.2021.00042

Peer reviewed articles (16)

Two is better than one: digital siblings to improve autonomous driving testing

Autorzy: Matteo Biagiola, Andrea Stocco, Vincenzo Riccio, Paolo Tonella
Opublikowane w: Empirical Software Engineering (EMSE), Numer Vol. 29,No. 72, 2024, ISSN 1382-3256
Wydawca: Kluwer Academic Publishers
DOI: 10.1007/s10664-024-10458-4

Confidence-driven Weighted Retraining for Predicting Safety-Critical Failures in Autonomous Driving Systems

Autorzy: Andrea Stocco, Paolo Tonella
Opublikowane w: Journal of Software: Evolution and Process, 2021, ISSN 2047-7481
Wydawca: John Wiley and Sons Ltd
DOI: 10.1002/smr.2386

Model vs System Level Testing of Autonomous Driving Systems: A Replication and Extension Study

Autorzy: Andrea Stocco, Brian Pulfer, Paolo Tonella
Opublikowane w: Empirical Software Engineering, Numer vol. 28, n. 3, 2023, ISSN 1382-3256
Wydawca: Kluwer Academic Publishers
DOI: 10.1007/s10664-023-10306-x

In vivo test and rollback of Java applications as they are

Autorzy: Antonia Bertolino, Guglielmo De Angelis, Breno Miranda, Paolo Tonella
Opublikowane w: Journal of Software: Testing, Verification and Reliability (STVR), Numer Vol. 33, No. 7, 2023, ISSN 0960-0833
Wydawca: John Wiley & Sons Inc.
DOI: 10.1002/stvr.1857

Efficient and Effective Feature Space Exploration for Testing Deep Learning Systems

Autorzy: Tahereh Zohdinasab, Vincenzo Riccio, Alessio Gambi, Paolo Tonella
Opublikowane w: ACM Transactions on Software Engineering and Methodology, Numer vol. 32, n. 2, 2023, Strona(/y) 1-38, ISSN 1049-331X
Wydawca: Association for Computing Machinary, Inc.
DOI: 10.1145/3544792

Assessing the security of inter-app communications in android through reinforcement learning

Autorzy: Andrea Romdhana, Alessio Merlo, Mariano Ceccato, Paolo Tonella
Opublikowane w: Computer Security, Numer vol. 131, 2023, ISSN 0167-4048
Wydawca: Pergamon Press Ltd.
DOI: 10.1016/j.cose.2023.103311

Uncertainty quantification for deep neural networks: An empirical comparison and usage guidelines

Autorzy: Michael Weiss, Paolo Tonella
Opublikowane w: Journal of Software: Testing, Verification and Reliability, 2023, Strona(/y) 1-23, ISSN 1099-1689
Wydawca: John Wiley & Sons
DOI: 10.1002/stvr.1840

Testing the Plasticity of Reinforcement Learning-based Systems

Autorzy: Matteo Biagiola, Paolo Tonella
Opublikowane w: ACM Transactions on Software Engineering and Methodology, Numer vol. 31, n. 4, 2022, Strona(/y) 1-46, ISSN 1049-331X
Wydawca: Association for Computing Machinary, Inc.
DOI: 10.1145/3511701

Testing machine learning based systems: a systematic mapping

Autorzy: Vincenzo Riccio, Gunel Jahangirova, Andrea Stocco, Nargiz Humbatova, Michael Weiss, Paolo Tonella
Opublikowane w: Empirical Software Engineering, Numer 25/6, 2020, Strona(/y) 5193-5254, ISSN 1382-3256
Wydawca: Kluwer Academic Publishers
DOI: 10.1007/s10664-020-09881-0

Adopting Two Supervisors for Efficient Use of Large-Scale Remote Deep Neural Networks

Autorzy: Michael Weiss, Paolo Tonella
Opublikowane w: ACM Transactions on Software Engineering and Methodology (TOSEM), Numer Vol.33, No. 1, 2024, ISSN 1049-331X
Wydawca: Association for Computing Machinary, Inc.
DOI: 10.1145/3617593

Testing of Deep Reinforcement Learning Agents with Surrogate Models

Autorzy: Matteo Biagiola, Paolo Tonella
Opublikowane w: ACM Transactions on Software Engineering and Methodology (TOSEM), Numer Vol. 33, No. 3, 2024, ISSN 1049-331X
Wydawca: Association for Computing Machinary, Inc.
DOI: 10.1145/3631970

An Empirical Validation of Oracle Improvement

Autorzy: Gunel Jahangirova, David Clark, Mark Harman, Paolo Tonella
Opublikowane w: IEEE Transactions on Software Engineering, 2019, Strona(/y) 1-1, ISSN 0098-5589
Wydawca: Institute of Electrical and Electronics Engineers
DOI: 10.1109/tse.2019.2934409

A Survey of Field-based Testing Techniques

Autorzy: Antonia Bertolino, Pietro Braione, Guglielmo De Angelis, Luca Gazzola, Fitsum Kifetew, Leonardo Mariani, Matteo Orrù, Mauro Pezzè, Roberto Pietrantuono, Stefano Russo, Paolo Tonella
Opublikowane w: ACM Computing Surveys, Numer 54/5, 2021, Strona(/y) 1-39, ISSN 0360-0300
Wydawca: Association for Computing Machinary, Inc.
DOI: 10.1145/3447240

Generating and detecting true ambiguity: a forgotten danger in DNN supervision testing

Autorzy: Michael Weiss, André García Gómez, Paolo Tonella
Opublikowane w: Empirical Software Engineering (EMSE), Numer Vol. 28,No. 146, 2023, ISSN 1382-3256
Wydawca: Kluwer Academic Publishers
DOI: 10.1007/s10664-023-10393-w

Mind the Gap! A Study on the Transferability of Virtual Versus Physical-World Testing of Autonomous Driving Systems

Autorzy: Andrea Stocco, Brian Pulfer, Paolo Tonella
Opublikowane w: IEEE Transactions on Software Engineering, Numer vol. 49, n. 4, 2023, Strona(/y) pp. 1928-1940, ISSN 0098-5589
Wydawca: Institute of Electrical and Electronics Engineers
DOI: 10.1109/tse.2022.3202311

Deep Reinforcement Learning for Black-Box Testing of Android Apps

Autorzy: Andrea Romdhana, Alessio Merlo, Mariano Ceccato, Paolo Tonella
Opublikowane w: ACM Transactions on Software Engineering and Methodology, 2022, ISSN 1049-331X
Wydawca: Association for Computing Machinary, Inc.
DOI: 10.1145/3502868