Skip to main content
An official website of the European UnionAn official EU website
European Commission logo
English English
CORDIS - EU research results
CORDIS
CORDIS Web 30th anniversary CORDIS Web 30th anniversary

Self-assessment Oracles for Anticipatory Testing

CORDIS provides links to public deliverables and publications of HORIZON projects.

Links to deliverables and publications from FP7 projects, as well as links to some specific result types such as dataset and software, are dynamically retrieved from OpenAIRE .

Deliverables

Data Management Plan

Open Data associated with the empirical evaluation.

Publications

A Framework for In-Vivo Testing of Mobile Applications

Author(s): Mariano Ceccato, Davide Corradini, Luca Gazzola, Fitsum Meshesha Kifetew, Leonardo Mariani, Matteo Orru, Paolo Tonella
Published in: 2020 IEEE 13th International Conference on Software Testing, Validation and Verification (ICST), 2020, Page(s) 286-296, ISBN 978-1-7281-5778-8
Publisher: IEEE
DOI: 10.1109/icst46399.2020.00037

An Empirical Evaluation of Mutation Operators for Deep Learning Systems

Author(s): Gunel Jahangirova, Paolo Tonella
Published in: 2020 IEEE 13th International Conference on Software Testing, Validation and Verification (ICST), 2020, Page(s) 74-84, ISBN 978-1-7281-5778-8
Publisher: IEEE
DOI: 10.1109/icst46399.2020.00018

Repairing DNN Architecture: Are We There Yet?

Author(s): Jinhan Kim, Nargiz Humbatova, Gunel Jahangirova, Paolo Tonella, Shin Yoo
Published in: Proceedings of the IEEE Conference on Software Testing, Verification and Validation, 2023, Page(s) pp. 234-245
Publisher: IEEE
DOI: 10.1109/icst57152.2023.00030

Model-based exploration of the frontier of behaviours for deep learning system testing

Author(s): Vincenzo Riccio, Paolo Tonella
Published in: Proceedings of the 28th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering, 2020, Page(s) 876-888, ISBN 9781450370431
Publisher: ACM
DOI: 10.1145/3368089.3409730

Evolutionary improvement of assertion oracles

Author(s): Valerio Terragni, Gunel Jahangirova, Paolo Tonella, Mauro Pezzè
Published in: Proceedings of the 28th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering, 2020, Page(s) 1178-1189, ISBN 9781450370431
Publisher: ACM
DOI: 10.1145/3368089.3409758

Quality Metrics and Oracles for Autonomous Vehicles Testing

Author(s): Gunel Jahangirova, Andrea Stocco, Paolo Tonella
Published in: 2021 14th IEEE Conference on Software Testing, Verification and Validation (ICST), 2021, Page(s) 194-204, ISBN 978-1-7281-6836-4
Publisher: IEEE
DOI: 10.1109/icst49551.2021.00030

An Empirical Study on Low- and High-Level Explanations of Deep Learning Misbehaviours

Author(s): Tahereh Zohdinasab, Vincenzo Riccio, Paolo Tonella
Published in: Proceedings of the ACM/IEEE International Symposium on Empirical Software Engineering and Measurement (ESEM), 2023
Publisher: IEEE Computer Society
DOI: 10.1109/esem56168.2023.10304866

DeepHyperion: exploring the feature space of deep learning-based systems through illumination search

Author(s): Tahereh Zohdinasab, Vincenzo Riccio, Alessio Gambi, Paolo Tonella
Published in: Proceedings of the 30th ACM SIGSOFT International Symposium on Software Testing and Analysis, 2021, Page(s) 79-90, ISBN 9781450384599
Publisher: ACM
DOI: 10.1145/3460319.3464811

A Review and Refinement of Surprise Adequacy

Author(s): Michael Weiss, Rwiddhi Chakraborty, Paolo Tonella
Published in: 2021 IEEE/ACM Third International Workshop on Deep Learning for Testing and Testing for Deep Learning (DeepTest), 2021, Page(s) 17-24, ISBN 978-1-6654-4565-8
Publisher: IEEE
DOI: 10.1109/deeptest52559.2021.00009

Run Java Applications and Test Them In-Vivo Meantime

Author(s): Antonia Bertolino, Guglielmo De Angelis, Breno Miranda, Paolo Tonella
Published in: 2020 IEEE 13th International Conference on Software Testing, Validation and Verification (ICST), 2020, Page(s) 454-459, ISBN 978-1-7281-5778-8
Publisher: IEEE
DOI: 10.1109/icst46399.2020.00061

Fail-Safe Execution of Deep Learning based Systems through Uncertainty Monitoring

Author(s): Michael Weiss, Paolo Tonella
Published in: 2021 14th IEEE Conference on Software Testing, Verification and Validation (ICST), 2021, Page(s) 24-35, ISBN 978-1-7281-6836-4
Publisher: IEEE
DOI: 10.1109/icst49551.2021.00015

Uncertainty-Wizard: Fast and User-Friendly Neural Network Uncertainty Quantification

Author(s): Michael Weiss, Paolo Tonella
Published in: 2021 14th IEEE Conference on Software Testing, Verification and Validation (ICST), 2021, Page(s) 436-441, ISBN 978-1-7281-6836-4
Publisher: IEEE
DOI: 10.1109/icst49551.2021.00056

Simple Techniques Work Surprisingly Well for Neural Network Test Prioritization and Active Learning (Replicability Study)

Author(s): Michael Weiss, Paolo Tonella
Published in: Proceedings of the 31st ACM SIGSOFT International Symposium on Software Testing and Analysis, 2022, Page(s) pp. 139-150
Publisher: ACM
DOI: 10.1145/3533767.3534375

DeepCrime: mutation testing of deep learning systems based on real faults

Author(s): Nargiz Humbatova, Gunel Jahangirova, Paolo Tonella
Published in: Proceedings of the 30th ACM SIGSOFT International Symposium on Software Testing and Analysis, 2021, Page(s) 67-78, ISBN 9781450384599
Publisher: ACM
DOI: 10.1145/3460319.3464825

DeepMetis: Augmenting a Deep Learning Test Set to Increase its Mutation Score

Author(s): Vincenzo Riccio, Nargiz Humbatova, Gunel Jahangirova, Paolo Tonella
Published in: 2021 36th IEEE/ACM International Conference on Automated Software Engineering (ASE), 2021, Page(s) 355-367
Publisher: IEEE Computer Society
DOI: 10.1109/ase51524.2021.9678764

Misbehaviour prediction for autonomous driving systems

Author(s): Andrea Stocco, Michael Weiss, Marco Calzana, Paolo Tonella
Published in: Proceedings of the ACM/IEEE 42nd International Conference on Software Engineering, 2020, Page(s) 359-371, ISBN 9781450371216
Publisher: ACM
DOI: 10.1145/3377811.3380353

IFRIT: Focused Testing through Deep Reinforcement Learning

Author(s): Andrea Romdhana, Mariano Ceccato, Alessio Merlo, Paolo Tonella
Published in: 2022 IEEE Conference on Software Testing, Verification and Validation (ICST), 2022
Publisher: IEEE
DOI: 10.1109/icst53961.2022.00013

When and Why Test Generators for Deep Learning Produce Invalid Inputs: an Empirical Study

Author(s): Vincenzo Riccio, Paolo Tonella
Published in: Proceedings of the IEEE/ACM 45th International Conference on Software Engineering (ICSE), 2023
Publisher: IEEE Computer Society
DOI: 10.1109/icse48619.2023.00104

Toward In-Vivo Testing of Mobile Applications

Author(s): Mariano Ceccato, Luca Gazzola, Fitsum Meshesha Kifetew, Leonardo Mariani, Matteo Orru, Paolo Tonella
Published in: 2019 IEEE International Symposium on Software Reliability Engineering Workshops (ISSREW), 2019, Page(s) 137-143, ISBN 978-1-7281-5138-0
Publisher: IEEE
DOI: 10.1109/issrew.2019.00063

Hypertesting of Programs: Theoretical Foundation and Automated Test Generation

Author(s): Michele Pasqua, Mariano Ceccato, Paolo Tonella
Published in: Proceedings of the IEEE/ACM 46th International Conference on Software Engineering (ICSE), 2024
Publisher: Association for Computing Machinery
DOI: 10.1145/3597503.3640323

ThirdEye: Attention Maps for Safe Autonomous Driving Systems

Author(s): Andrea Stocco, Paulo J. Nunes, Marcelo d'Amorim, Paolo Tonella
Published in: Proceedings of the 37th IEEE/ACM International Conference on Automated Software Engineering (ASE), 2022
Publisher: ACM
DOI: 10.1145/3551349.3556968

DeepAtash: Focused Test Generation for Deep Learning Systems

Author(s): Tahereh Zohdinasab, Vincenzo Riccio, Paolo Tonella
Published in: Proceedings of the 32nd ACM SIGSOFT International Symposium on Software Testing and Analysis (ISSTA), 2023
Publisher: ACM SIGSOFT
DOI: 10.1145/3597926.3598109

Taxonomy of real faults in deep learning systems

Author(s): Nargiz Humbatova, Gunel Jahangirova, Gabriele Bavota, Vincenzo Riccio, Andrea Stocco, Paolo Tonella
Published in: Proceedings of the ACM/IEEE 42nd International Conference on Software Engineering, 2020, Page(s) 1110-1121, ISBN 9781450371216
Publisher: ACM
DOI: 10.1145/3377811.3380395

Simulation-based Test Case Generation for Unmanned Aerial Vehicles in the Neighborhood of Real Flights

Author(s): Sajad Khatiri, Sebastiano Panichella, Paolo Tonella
Published in: Proceedings of the IEEE Conference on Software Testing, Verification and Validation, 2023, Page(s) pp. 281-292
Publisher: IEEE
DOI: 10.1109/icst57152.2023.00034

GAssert: A Fully Automated Tool to Improve Assertion Oracles

Author(s): Valerio Terragni, Gunel Jahangirova, Paolo Tonella, Mauro Pezze
Published in: 2021 IEEE/ACM 43rd International Conference on Software Engineering: Companion Proceedings (ICSE-Companion), 2021, Page(s) 85-88, ISBN 978-1-6654-1219-3
Publisher: IEEE
DOI: 10.1109/icse-companion52605.2021.00042

Two is better than one: digital siblings to improve autonomous driving testing

Author(s): Matteo Biagiola, Andrea Stocco, Vincenzo Riccio, Paolo Tonella
Published in: Empirical Software Engineering (EMSE), Issue Vol. 29,No. 72, 2024, ISSN 1382-3256
Publisher: Kluwer Academic Publishers
DOI: 10.1007/s10664-024-10458-4

Confidence-driven Weighted Retraining for Predicting Safety-Critical Failures in Autonomous Driving Systems

Author(s): Andrea Stocco, Paolo Tonella
Published in: Journal of Software: Evolution and Process, 2021, ISSN 2047-7481
Publisher: John Wiley and Sons Ltd
DOI: 10.1002/smr.2386

Model vs System Level Testing of Autonomous Driving Systems: A Replication and Extension Study

Author(s): Andrea Stocco, Brian Pulfer, Paolo Tonella
Published in: Empirical Software Engineering, Issue vol. 28, n. 3, 2023, ISSN 1382-3256
Publisher: Kluwer Academic Publishers
DOI: 10.1007/s10664-023-10306-x

In vivo test and rollback of Java applications as they are

Author(s): Antonia Bertolino, Guglielmo De Angelis, Breno Miranda, Paolo Tonella
Published in: Journal of Software: Testing, Verification and Reliability (STVR), Issue Vol. 33, No. 7, 2023, ISSN 0960-0833
Publisher: John Wiley & Sons Inc.
DOI: 10.1002/stvr.1857

Efficient and Effective Feature Space Exploration for Testing Deep Learning Systems

Author(s): Tahereh Zohdinasab, Vincenzo Riccio, Alessio Gambi, Paolo Tonella
Published in: ACM Transactions on Software Engineering and Methodology, Issue vol. 32, n. 2, 2023, Page(s) 1-38, ISSN 1049-331X
Publisher: Association for Computing Machinary, Inc.
DOI: 10.1145/3544792

Assessing the security of inter-app communications in android through reinforcement learning

Author(s): Andrea Romdhana, Alessio Merlo, Mariano Ceccato, Paolo Tonella
Published in: Computer Security, Issue vol. 131, 2023, ISSN 0167-4048
Publisher: Pergamon Press Ltd.
DOI: 10.1016/j.cose.2023.103311

Uncertainty quantification for deep neural networks: An empirical comparison and usage guidelines

Author(s): Michael Weiss, Paolo Tonella
Published in: Journal of Software: Testing, Verification and Reliability, 2023, Page(s) 1-23, ISSN 1099-1689
Publisher: John Wiley & Sons
DOI: 10.1002/stvr.1840

Testing the Plasticity of Reinforcement Learning-based Systems

Author(s): Matteo Biagiola, Paolo Tonella
Published in: ACM Transactions on Software Engineering and Methodology, Issue vol. 31, n. 4, 2022, Page(s) 1-46, ISSN 1049-331X
Publisher: Association for Computing Machinary, Inc.
DOI: 10.1145/3511701

Testing machine learning based systems: a systematic mapping

Author(s): Vincenzo Riccio, Gunel Jahangirova, Andrea Stocco, Nargiz Humbatova, Michael Weiss, Paolo Tonella
Published in: Empirical Software Engineering, Issue 25/6, 2020, Page(s) 5193-5254, ISSN 1382-3256
Publisher: Kluwer Academic Publishers
DOI: 10.1007/s10664-020-09881-0

Adopting Two Supervisors for Efficient Use of Large-Scale Remote Deep Neural Networks

Author(s): Michael Weiss, Paolo Tonella
Published in: ACM Transactions on Software Engineering and Methodology (TOSEM), Issue Vol.33, No. 1, 2024, ISSN 1049-331X
Publisher: Association for Computing Machinary, Inc.
DOI: 10.1145/3617593

Testing of Deep Reinforcement Learning Agents with Surrogate Models

Author(s): Matteo Biagiola, Paolo Tonella
Published in: ACM Transactions on Software Engineering and Methodology (TOSEM), Issue Vol. 33, No. 3, 2024, ISSN 1049-331X
Publisher: Association for Computing Machinary, Inc.
DOI: 10.1145/3631970

An Empirical Validation of Oracle Improvement

Author(s): Gunel Jahangirova, David Clark, Mark Harman, Paolo Tonella
Published in: IEEE Transactions on Software Engineering, 2019, Page(s) 1-1, ISSN 0098-5589
Publisher: Institute of Electrical and Electronics Engineers
DOI: 10.1109/tse.2019.2934409

A Survey of Field-based Testing Techniques

Author(s): Antonia Bertolino, Pietro Braione, Guglielmo De Angelis, Luca Gazzola, Fitsum Kifetew, Leonardo Mariani, Matteo Orrù, Mauro Pezzè, Roberto Pietrantuono, Stefano Russo, Paolo Tonella
Published in: ACM Computing Surveys, Issue 54/5, 2021, Page(s) 1-39, ISSN 0360-0300
Publisher: Association for Computing Machinary, Inc.
DOI: 10.1145/3447240

Generating and detecting true ambiguity: a forgotten danger in DNN supervision testing

Author(s): Michael Weiss, André García Gómez, Paolo Tonella
Published in: Empirical Software Engineering (EMSE), Issue Vol. 28,No. 146, 2023, ISSN 1382-3256
Publisher: Kluwer Academic Publishers
DOI: 10.1007/s10664-023-10393-w

Mind the Gap! A Study on the Transferability of Virtual Versus Physical-World Testing of Autonomous Driving Systems

Author(s): Andrea Stocco, Brian Pulfer, Paolo Tonella
Published in: IEEE Transactions on Software Engineering, Issue vol. 49, n. 4, 2023, Page(s) pp. 1928-1940, ISSN 0098-5589
Publisher: Institute of Electrical and Electronics Engineers
DOI: 10.1109/tse.2022.3202311

Deep Reinforcement Learning for Black-Box Testing of Android Apps

Author(s): Andrea Romdhana, Alessio Merlo, Mariano Ceccato, Paolo Tonella
Published in: ACM Transactions on Software Engineering and Methodology, 2022, ISSN 1049-331X
Publisher: Association for Computing Machinary, Inc.
DOI: 10.1145/3502868

Searching for OpenAIRE data...

There was an error trying to search data from OpenAIRE

No results available