Periodic Reporting for period 1 - EVENFLOW (Robust Learning and Reasoning for Complex Event Forecasting)
Reporting period: 2022-10-01 to 2024-03-31
EVENFLOW develops hybrid learning techniques for complex event forecasting, which combine deep learning with logic-based learning and reasoning into neuro-symbolic forecasting models. The envisioned methods combine (i) neural representation learning techniques, capable of constructing event-based features from streams of perception-level data with (ii) powerful symbolic learning and reasoning tools, that utilize such features to synthesize high-level, interpretable patterns of critical situations to be forecast.
Crucial in the EVENFLOW approach is the online nature of the learning methods, which makes them applicable to evolving data flows and allows to utilize rich domain knowledge that is becoming available progressively. To deal with the brittleness of neural predictors and the high volume/velocity of temporal data flows, the EVENFLOW techniques rely on novel, formal verification techniques for machine learning, in addition to a suite of scalability algorithms for federated training and incremental model construction. The learnt forecasters will be interpretable and scalable, allowing for fully explainable insights, delivered in a timely fashion and enabling proactive decision making.
EVENFLOW is evaluated on three challenging use cases related to (1) oncological forecasting in precision medicine, (2) safe and efficient behaviour of autonomous transportation robots in smart factories and (3) reliable life cycle assessment of critical infrastructure.
Crucial in the EVENFLOW approach is the online nature of the learning methods, which makes them applicable to evolving data flows and allows to utilize rich domain knowledge that is becoming available progressively. To deal with the brittleness of neural predictors and the high volume/velocity of temporal data flows, the EVENFLOW techniques rely on novel, formal verification techniques for machine learning, in addition to a suite of scalability algorithms for federated training and incremental model construction. The learnt forecasters will be interpretable and scalable, allowing for fully explainable insights, delivered in a timely fashion and enabling proactive decision making.
EVENFLOW is evaluated on three challenging use cases related to (1) oncological forecasting in precision medicine, (2) safe and efficient behaviour of autonomous transportation robots in smart factories and (3) reliable life cycle assessment of critical infrastructure.
An outline of the scientific/technical work that was carried-out in the project is summarized as follows:
(a) Use cases and data generation/collection activities:
- Requirements elicitation per use case and alignment with the project's objectives.
- For the "Industry 4.0" and the "Infrastructure Life Cycle Assessment" use cases, setting up simulation environments and conducting simulations for data generation, over multiple iterations. For the "Personalized Medicine" use case, harvesting cancer-related data from open *omics data bases and using the data for training a Variational Autoencoder to generate synthetic (virtual) patients; Alignment of gene expression-level information with high-level, activation pathway information, in order to use the latter for interpretable ML; Generating time-evolving trajectories of disease progression based on the virtual patients, in order to train temporal models for predicting disease deterioration.
(b) Neuro-symbolic (NeSy) event forecasting
- Thorough review of the NeSy literature and empirical evaluation of the most promising existing NeSy techniques on surrogate data.
- Development of a novel temporal NeSy learning and reasoning technique, which significantly extents the state of the art.
- Development of a novel event pattern learning techniques, which significantly extents the state of the art.
- Complex event forecasting as an application of the new NeSy learning and reasoning technique.
- First steps towards interpretable forecasting/explainable forecasts.
- Evaluation of the aforementioned techniques on EVENFLOW data, during various phases of the data generation process, in addition to evaluation on surrogate data.
(c) Scalable training of neural models.
- Novel techniques for scalable neural training, including data synopses and distributed and parallel training methods.
- Thorough evaluation on EVENFLOW data and surrogate datasets.
(d) Verification of neural models
- Evaluation of existing neural network verification techniques on temporal data, using transformer-based neural models.
- Novel techniques for verifiable training.
- Novel techniques for verifying neuro-symbolic models.
(e) EVENFLOW platform
- Requirement analysis and architecture design for a streaming platform, enabling the connections between the project innovative components descibed above.
- First deployment of the platform, supporting orchestration, streaming, historical data handling, mutiple users.
(a) Use cases and data generation/collection activities:
- Requirements elicitation per use case and alignment with the project's objectives.
- For the "Industry 4.0" and the "Infrastructure Life Cycle Assessment" use cases, setting up simulation environments and conducting simulations for data generation, over multiple iterations. For the "Personalized Medicine" use case, harvesting cancer-related data from open *omics data bases and using the data for training a Variational Autoencoder to generate synthetic (virtual) patients; Alignment of gene expression-level information with high-level, activation pathway information, in order to use the latter for interpretable ML; Generating time-evolving trajectories of disease progression based on the virtual patients, in order to train temporal models for predicting disease deterioration.
(b) Neuro-symbolic (NeSy) event forecasting
- Thorough review of the NeSy literature and empirical evaluation of the most promising existing NeSy techniques on surrogate data.
- Development of a novel temporal NeSy learning and reasoning technique, which significantly extents the state of the art.
- Development of a novel event pattern learning techniques, which significantly extents the state of the art.
- Complex event forecasting as an application of the new NeSy learning and reasoning technique.
- First steps towards interpretable forecasting/explainable forecasts.
- Evaluation of the aforementioned techniques on EVENFLOW data, during various phases of the data generation process, in addition to evaluation on surrogate data.
(c) Scalable training of neural models.
- Novel techniques for scalable neural training, including data synopses and distributed and parallel training methods.
- Thorough evaluation on EVENFLOW data and surrogate datasets.
(d) Verification of neural models
- Evaluation of existing neural network verification techniques on temporal data, using transformer-based neural models.
- Novel techniques for verifiable training.
- Novel techniques for verifying neuro-symbolic models.
(e) EVENFLOW platform
- Requirement analysis and architecture design for a streaming platform, enabling the connections between the project innovative components descibed above.
- First deployment of the platform, supporting orchestration, streaming, historical data handling, mutiple users.
(a) Novel temporal neuro-symbolic learning and reasoning techniques, that significantly advance the state-of-the-art. Notably, learning from perceptual input acros-time, while reasoning with domain knowledge of temporal nature was not possible using existing neuro-symbolic techniques, prior to the techniques that have been introduced in EVENFLOW.
(b) Neuro-symbolic complex event forecasting as a concrete application of the above. This allows to make forecasts of critical events in a timely fashion, from perceptual data streams, which was not possible prior to the EVENFLOW complex event forecasting techniques.
(c) Explainable forecasts as a natural byproduct of EVENFLOW’s neuro-symbolic complex event forecasting techniques.
(d) Novel techniques for scalable neural training, based on Synopses-based Training Optimization and distributed/parallel training, yielding significant improvements in terms of training times vs predictive accuracy trade-offs.
(e) Novel techniques for formally verifying the robustness of neuro-symbolic systems in an end-to-end fashion, i.e. from input perturbations to neural predictions, to the robustness of logic-based inferences. Notably, verifying the robustness of compositional systems that combine neural-based learning with logic-based reasoning was not possible prior to the techniques introduced in EVENFLOW.
(b) Neuro-symbolic complex event forecasting as a concrete application of the above. This allows to make forecasts of critical events in a timely fashion, from perceptual data streams, which was not possible prior to the EVENFLOW complex event forecasting techniques.
(c) Explainable forecasts as a natural byproduct of EVENFLOW’s neuro-symbolic complex event forecasting techniques.
(d) Novel techniques for scalable neural training, based on Synopses-based Training Optimization and distributed/parallel training, yielding significant improvements in terms of training times vs predictive accuracy trade-offs.
(e) Novel techniques for formally verifying the robustness of neuro-symbolic systems in an end-to-end fashion, i.e. from input perturbations to neural predictions, to the robustness of logic-based inferences. Notably, verifying the robustness of compositional systems that combine neural-based learning with logic-based reasoning was not possible prior to the techniques introduced in EVENFLOW.