Program Intelligence, Declaratively and Symbolically

Informazioni relative al progetto

PINDESYM

ID dell’accordo di sovvenzione: 101095951

DOI

10.3030/101095951

Data della firma CE 24 Ottobre 2023

Data di avvio 1 Gennaio 2024

Data di completamento 31 Dicembre 2028

Finanziato da

• European Research Council (ERC)

Costo totale

€ 2 395 875,00

Contributo UE

€ 2 395 875,00

Coordinato da

ETHNIKO KAI KAPODISTRIAKO PANEPISTIMIO ATHINON
Greece

Periodic Reporting for period 1 - PINDESYM (Program Intelligence, Declaratively and Symbolically)

Periodo di rendicontazione: 2024-01-01 al 2026-06-30

Sintesi del contesto e degli obiettivi generali del progetto

The PINDESYM project aims to develop insightful program understanding via declarative static analysis. Such insightful understanding of program behavior captures the essence of the computation, at multiple levels of abstraction, some close to human reasoning. To address this grand challenge, the project will significantly advance the state of the art in program analysis with new concepts, algorithms, implementation techniques, and publicly available tools. The key component of the approach is declarative specifications of analysis algorithms—an already highly successful methodology. What promises to dramatically raise the impact of declarative program analysis is two innovations, matching emerging research and technology trends. The first is the idea of combining a declarative system (e.g. a Datalog fixpoint engine) and a symbolic reasoning system, such as an SMT solver or algebraic rewrite system. The second is the seamless integration of a machine learning approach, over large amounts of data (from past code bases), in the declarative inference process. The PINDESYM project leverages symbolic reasoning and learning approaches to greatly advance program analysis. The challenge is dual: not only to invent powerful new techniques and algorithms, but also to capture all the diversity in symbolic, value-flow, and learning-based reasoning in a single, unified, reusable, and extensible analysis framework—a true deep program understanding engine, far beyond current approaches.

Lavoro eseguito dall’inizio del progetto fino alla fine del periodo coperto dalla relazione e principali risultati finora ottenuti

The overall arch of the work has been improvement over declarative static program analysis techniques. The main themes and objectives as stated in the Description of the Action have been a) the integration of symbolic reasoning; b) the integration of machine learning over past code bases; c) the evolution of whole-program analysis to be more scalable and modular; d) the introduction of new analysis algorithms and/or domains. These themes have been precisely the main aspects of the work performed in the first two years of the project, although the low-level specifics have adjusted to take advantage of arising opportunities, new machinery, and interesting prospects.
Specifically, the work has evolved the main declarative analysis frameworks of my research. Below, are the most clearly-defined sub-projects of the performed work.
i. Symbolic reasoning (via an SMT solver integration) has been added to a Datalog engine.
ii. New context-sensitive analysis algorithms have been invented.
iii. A major, cross-cutting optimization technique has been introduced that makes every declarative analysis scalable, sacrificing small degrees of completeness.
iv. A declarative modeling of low-level memory (a new domain) has been proposed and found to be very effective.
v. Analysis results are currently in the process of being exported for use by LLMs, as an MCP server that the LLM can query.
vi. LLMs have been trained for program understanding tasks, such as achieving excellent performance in queries of semantic similarity over functions.

Progressi oltre lo stato dell’arte e potenziale impatto previsto (incluso l’impatto socioeconomico e le implicazioni sociali più ampie del progetto fino ad ora)

The three most prominent achievements, as documented by top-tier publications have been:

A. The invention of a context-sensitive analysis algorithm that can closely model low-level execution flow with great precision, while maintaining scalability. The algorithm is applied to the problem of decompiling low-level code, treated as a static analysis question, especially in languages such as that of the Ethereum Virtual Machine, which are effectively continuation-passing style machines. The algorithm is documented in ISSTA'25 publication "The Incredible Shrinking Context... in a Decompiler Near You" [https://dl.acm.org/doi/10.1145/3728935]. This a static analysis algorithm very different from past work (which was mostly applicable in the context of languages like Java or Python). The idea of context sensitivity remains the same, but in this work the setting is entirely dissimilar. Whereas for analyzing the value flow of a Java program we are fortunate to have analyses with a depth of 2 or 3, here we have an analysis with a context depth of at least 10 (or even much more), where much more ambitious context shaping techniques can be employed. In particular, the algorithm aggressively exploits hints for dropping context elements ("shrinking" the context). That is, the algorithm truncates the context not just by eliminating the last context element when adding a new one, but by preemptively cutting multiple context elements that are not at the tail of the context chain. Doing so enables great "intelligent selection" of context elements, allowing for significantly imptoved precision and scalability.

B. The invention of a technique for effectively scaling up any declarative program analysis. This is described in OOPSLA'25 publication "Universal Scalability in Declarative Program Analysis (with Choice-Based Combination Pruning)". [https://dl.acm.org/doi/10.1145/3763129]
The impact of this technique is simply tremendous! It is implemented extremely efficiently, with no overheads for filtering. The result makes any declarative static analysis scale to even the most complex programs in existence, with the completeness loss being entirely tolerable. Speedups of 50x for large analyzed programs are entirely realistic.

C. A static analysis algorithm for reconstruction of memory layout from low-level instructions that access this memory. The technique finds particular application to the storage layout of Ethereum smart contracts, and constitutes a major advance compared to past efforts. Not only does the analysis exhibit unprecedented scalability (e.g. 99.4% completion rate, vs. 50.5% for the previous state of the art) but it also offers near-perfect completeness in recovering data structures, almost flawlessly (at 97.7% precision). The technique is documented in the paper ""Precise Static Identification of Ethereum Storage Variables", accepted for publication at ICSE'26 (to be published in April 2026). A preprint can be found at https://yanniss.github.io/eth-storage-icse26-extended.pdf .