We have made progress in all three directions and mention a few highlights.
As part of the theoretical foundations, we have developed a theory of quantitative safety and liveness based on monitoring. Safety and liveness are elementary concepts of computer science, which lie at the heart of most verification paradigms. The safety-liveness classification of boolean properties characterizes whether a given property can be falsified by observing a finite prefix of an infinite computation trace (always for safety, never for liveness). In quantitative specification and verification, properties assign not truth values, but quantitative values to computation traces (e.g. a cost, or the distance to a boolean property). We introduce quantitative safety and liveness and show that every quantitative property can be written as the pointwise minimum of a quantitative safety property and a quantitative liveness property. Consequently, like boolean properties, also quantitative properties - and their monitors - can be decomposed into safety and liveness parts. Moreover, quantitative properties can be approximated naturally. We prove that every quantitative property that has both safe and co-safe approximations can be monitored arbitrarily precisely by a monitor that uses only a finite amount of memory. [FOSSACS 2023]
As part of the monitoring applications, we have developed algorithms for monitoring the degree of bias in decision making software. We assume that the monitored system is unknown, but has a Markov chain structure. We introduce a specification language that can model common algorithmic fairness properties such as demographic parity, equal opportunity, and social burden. Our algorithms observe a long sequence of events as generated by the monitored system, and produce, after each observation, a quantitative estimate of how fair or biased the system was on that execution trace up to that point in time. The estimate is proven to be correct modulo an error bound and confidence level. Our monitors are of two types that use, respectively, frequentist and Bayesian statistical inference techniques. [CAV 2023]
As part of the envisioned middleware infrastructure, we have built a first prototype, called VAMOS (like the project), for the runtime monitoring of software which is designed specifically to support third-party and best-effort scenarios. In a third-party scenario, the monitoring software can observe only the inputs and outputs of the monitored software. In a best-effort scenario, the monitoring software must drop some observed events in order to keep up with the monitored software. The design goals of the VAMOS middleware are (i) efficiency (low overhead), (ii) flexibility (the connectability of monitors to different event sources and specification languages), and (iii) ease-of-use. [FASE 2023]
