In order to realize our vision of just-in-time verification of autonomous systems, we have rethought standard offline verification techniques in many ways: (1) we have developed anytime verification to adjust to changing environments and timing-constraints, (2) we considered other surrounding intelligent agents that appear and disappear on the fly, (3) and we combined planning and control with verification techniques to repair failed verification attempts online.
Since formal methods consider all eventualities, plans are potentially refused due to a small possibility of failure. However, many long-term plans that are initially unsafe for the entire considered time horizon might become safe after a short time due to the development of the current situation. Thus, we developed a method that performs trajectory planning for two time horizons in parallel: Long-term trajectories are generated using non-formal techniques, such as established planning techniques and/or machine learning. Since the uncertainty of possible behaviors of surrounding intelligent agents grows over time, we apply our verification concept only to the first part of the long-term reference trajectories. The time horizon of this combined trajectory (first part of intended trajectory plus fail-safe trajectory) is short, such that our set-based techniques do not block overly large regions for trajectory planning. The fail-safe trajectory brings the system into a safe state. If the maneuver is safe, the next part of the long-term plan is executed; otherwise, the fail-safe trajectory is initiated.
Our new approach quickly obtains verification results for changing environments by reusing reachable sets of the previous snapshot of surrounding agents and the host system. This can be done since the reachable sets of the previous sensor update have been computed in an over-approximative way, such that they continue to contain all possible behaviors up to the time horizon of the previous verification.
The reused reachable sets of other agents and the host system are refined in an anytime fashion to tighten the over-approximation as long as time remains. We developed methods to compute several abstractions of detected agents on the fly with increasing complexity and individual properties. Each time the reachable set of an abstraction has been obtained on time, the result is aggregated, reducing the over-approximation as long as time permits.
Besides refining previously computed reachable sets, our concept also considers an on-the-fly integration of newly detected agents. Integrating new agents into a single model of all agents is computationally infeasible and also impractical since the interaction between agents, which is required for a common model, is typically unknown (unless they communicate their plans). For this reason, we consider a set of possible interaction mechanisms that are only constrained by impossible joint behaviors, e.g. behaviors resulting in occupying the same space are removed.
During the refinement of reachable sets, it is also checked which planned trajectories violate the formal specifications. To improve robustness of the approach by repairing almost safe plans, we interleave trajectory planning and verification techniques.
One aspect that is often overlooked in formal verification of autonomous systems is whether all possible behaviors of the real system can be generated by uncertain models. In contrast to standard techniques for system identification, we determine a range of system parameters rather than a single optimal value. We developed new techniques using optimization techniques that determine those required sets of possible system parameters.
