Periodic Reporting for period 2 - SYMBIOTIK (Context-aware adaptive visualizations for critical decision making)
Reporting period: 2023-10-01 to 2025-03-31
We live in an era of information overload that impairs objective decision making, especially in time-sensitive contexts. Information Visualization (InfoVis) systems have been used to mitigate information overload, yet they have not yet unlocked their potential in critical decision-making scenarios. From emergency rooms and autonomous cars to operational command centres, a clear understanding and rapid assessment based on the available data can make the difference between life and death. SYMBIOTIK envisions an effortless interaction dialogue between human and InfoVis systems to support decision making processes, inspired by known biological principles and guided by artificial intelligence (AI). Critically, this dialogue requires AI solutions with context awareness, emotion sensing, and expressing capabilities. We propose a novel framework where both the human and the machine cooperate towards a common goal and evolve together. The overall objectives of the project are:
1. Support a natural dialogue between AI agents and humans following symbiosis principles.
2. Develop human-centric perception analysis methods and tools for situation/time-sensitive tasks in InfoVis systems.
3. Facilitate a novice-to-expert transition in InfoVis systems by means of context-aware self-adaptive graphical user interfaces.
Awareness principles will allow us to engineer complex systems, making them more resilient and more human-centric. We will define an integrative approach for awareness engineering and propose a specific open source implementation. Finally, we will demonstrate and validate the role and added-value of such an awareness framework in two scenarios: supporting novice-to-expert transitions and critical decision making. The awareness principles to be developed in this project can support learning, adaptation, and self-development of intelligent systems over long periods of time, not only in the InfoVis domain. Therefore, SYMBIOTIK has potential to achieve the real breakthroughs needed to bring awareness and emotional intelligence for decision-making tasks in computing systems. The results of the project will benefit a range of stakeholders, from human vision and brain researchers, computer scientists, citizens, as well as research funding bodies and policy makers.
1. Support a natural dialogue between AI agents and humans following symbiosis principles.
2. Develop human-centric perception analysis methods and tools for situation/time-sensitive tasks in InfoVis systems.
3. Facilitate a novice-to-expert transition in InfoVis systems by means of context-aware self-adaptive graphical user interfaces.
Awareness principles will allow us to engineer complex systems, making them more resilient and more human-centric. We will define an integrative approach for awareness engineering and propose a specific open source implementation. Finally, we will demonstrate and validate the role and added-value of such an awareness framework in two scenarios: supporting novice-to-expert transitions and critical decision making. The awareness principles to be developed in this project can support learning, adaptation, and self-development of intelligent systems over long periods of time, not only in the InfoVis domain. Therefore, SYMBIOTIK has potential to achieve the real breakthroughs needed to bring awareness and emotional intelligence for decision-making tasks in computing systems. The results of the project will benefit a range of stakeholders, from human vision and brain researchers, computer scientists, citizens, as well as research funding bodies and policy makers.
* Three contacts established with EU projects (KPI2.4): BITSCOPE and BANANA, both from the CHIST-ERA program (BCI call), and SUSTAIN from the EIC “Awareness Inside” Pathfinder challenge.
* Three components/software available for internal use (KPI2.5).
* One more dataset collected (KPI2.6) about three adaptive InfoVis widgets (LUX & TID). Another dataset collection is ongoing (LUX & UCL).
* Thirty blog posts written (KPI3.2) on https://symbiotik-infovis.eu/blog/(opens in new window)
* Publications in eight different ACM conferences (KPI4.2): Multimedia, NordiCHI, EICS, UMAP, ETRA, UIST, IUI, and SIGIR. All top-tier venues in their research domains.
* Thirty papers published in RP2 (KPI4.3) out of which 25 were in top venues (Q1 or Q2 journals or CORE A or B conferences, KPI4.4) and 7 were co-authored by two or more partners (KPI4.5).
* Two more co-supervised academic works (KPI4.6) totalling six so far.
* Two PhDs are now in their second year (KPI5.1).
* Two more post-doctoral scholars involved (KPI5.2) totalling seven so far.
* Two more MSc degrees awarded (KPI5.3) totalling six so far.
* Three components/software available for internal use (KPI2.5).
* One more dataset collected (KPI2.6) about three adaptive InfoVis widgets (LUX & TID). Another dataset collection is ongoing (LUX & UCL).
* Thirty blog posts written (KPI3.2) on https://symbiotik-infovis.eu/blog/(opens in new window)
* Publications in eight different ACM conferences (KPI4.2): Multimedia, NordiCHI, EICS, UMAP, ETRA, UIST, IUI, and SIGIR. All top-tier venues in their research domains.
* Thirty papers published in RP2 (KPI4.3) out of which 25 were in top venues (Q1 or Q2 journals or CORE A or B conferences, KPI4.4) and 7 were co-authored by two or more partners (KPI4.5).
* Two more co-supervised academic works (KPI4.6) totalling six so far.
* Two PhDs are now in their second year (KPI5.1).
* Two more post-doctoral scholars involved (KPI5.2) totalling seven so far.
* Two more MSc degrees awarded (KPI5.3) totalling six so far.
The results of the project will be readily available in four different forms: (R1) Guidelines for other researchers (see task T6.3 and deliverable D6.3) aiming at adapting visual displays in a symbiotic way; (R2) Annotations of visual materials that can be leveraged for further investigation; (R3) Proven algorithms and computational procedures that can be reused for similar problems; and (R4) Proof-of-concept prototypes that demonstrate the potential of the technology and inspire further research and development; (R5) A validated taxonomy to uncover latent relationships between neurophysiological and cognitive states of the user. So far R2-5 have been achieved and we are collectively working on R1, which will be delivered in D6.4 in the next reporting period.
Our project and subsequent work will be promoted via spin-off creation (KPI5.4) software licences, and through a SaaS platform that will pave the way for companies to follow this paradigm, contributing to the market expansion and the goals of Digital Single Market. As we discussed in Section 2.1 of the DoA document, the InfoVis and Business Intelligence markets are the most accessible ones for our technology. Application examples in the InfoVis domain include retail and e-commerce, logistics, and real estate. Application examples in Business Intelligence include digital marketing and sales operations. Large players in these markets are SAP SE, Microsoft, Rackspace, or IBM. They are currently searching for new competitive advantages like AI integration, and our technology could help visualise specific figures categories.
Our project and subsequent work will be promoted via spin-off creation (KPI5.4) software licences, and through a SaaS platform that will pave the way for companies to follow this paradigm, contributing to the market expansion and the goals of Digital Single Market. As we discussed in Section 2.1 of the DoA document, the InfoVis and Business Intelligence markets are the most accessible ones for our technology. Application examples in the InfoVis domain include retail and e-commerce, logistics, and real estate. Application examples in Business Intelligence include digital marketing and sales operations. Large players in these markets are SAP SE, Microsoft, Rackspace, or IBM. They are currently searching for new competitive advantages like AI integration, and our technology could help visualise specific figures categories.