Skip to main content

Strong Modular proof Assistance: Reasoning across Theories

Periodic Reporting for period 3 - SMART (Strong Modular proof Assistance: Reasoning across Theories)

Reporting period: 2020-03-01 to 2021-08-31

Formal proof technology delivers an unparalleled level of certainty and security. Nevertheless, applying proof assistants to the verification of complex theories and designs is still extremely laborious. High profile certification projects, such as seL4, CompCert, and Flyspeck require tens of person-years. We recently demonstrated that this effort can be significantly reduced by combining reasoning and learning in so called hammer systems: 40% of the Flyspeck, HOL4, Isabelle/HOL, and Mizar top-level lemmas can be proved automatically.

Today’s early generation of hammers consists of individual systems limited to very few proof assistants. The accessible knowledge repositories are isolated, and there is no reuse of hammer components. It is possible to achieve a breakthrough in proof automation by developing new AI methods that combine reasoning knowledge and techniques into a smart hammer, that works over a very large part of today’s formalized knowledge. The main goal of the project is to develop a strong and uniform learning-reasoning system available for multiple logical foundations. To achieve this, we will develop: (a) uniform learning methods, (b) reusable ATP encoding components for different foundational aspects, (c) integration of proof reconstruction, and (d) methods for knowledge extraction, reuse and content merging. The single proof advice system will be made available for multiple proof assistants and their vast heterogeneous libraries.
The first half of the project has mostly been executed as planned. We have managed to make more progress about the objectives (b) and (c) while we did less progress when it comes to (a) than initially proposed.

When it comes to objective (a) of the proposal, we have developed a first version of the CIC0 logic combining data from the various type theoretic systems. We have investigated the various machine learning tasks for theorem proving and created various benchmarks. We started looking at the characterization of mathematical knowledge that is more appropriate for machine learning methods.

For objective (b), we have developed a first version of the CoqHammer translation are working on making it more powerful on the different Coq libraries. We have developed the Isabelle/Mizar object logic and are working on moving the Mizar knowledge to that foundation. We have developed a number of benchmarks for proof assistant ATP methods, including formalizations of category theory in Coq and game theory in Isabelle.

For objective (c), we have looked at Monte Carlo methods in theorem proving. We have also looked at learning conjecturing in proof assistants, and developed a non clausal tableaux certification mechanism.

For objective (d), we have studied alignments across 6 proof assistant and worked on statistical and deep auto-formalization. We have also developed a classification and a standard for mathematical alignments.
The expected outcome of the project remains unchanged from the proposal. We plan to further develop and integrate the techniques developed in the first half of the project in a proof advice system.
That proof advice system should be able to automatically prove a significant part of obligations in multiple modern proof assistants. We have already made progress about Coq, and set theory in Isabelle, and we expect more progress on other proof assistants as well as improvements for the other proof libraries.

We hope that the combined smart advice method together with the vast accumulated knowledge will result in a novel kind of tool, which allows working mathematicians to automatically find proofs of many simple conjectures, paving the way for the widespread use of formal proof in mathematics and computer science.