Skip to main content
Go to the home page of the European Commission (opens in new window)
English en
CORDIS - EU research results
CORDIS

Higher-order multimodal Interaction: Presenting novel structure in Real-world NETworked problems

Project description

The existence of higher-order multimodal node relationships

Graphs are a fundamental mathematical tool for representing interconnected subsystems and their relationships. However, many real-world problems, such as gene expression and financial networks, require greater complexity than traditional graphs can provide. Supported by the Marie Skłodowska-Curie Actions programme, the HIPeR-NET project will demonstrate the existence of proposed higher-order multimodal node relationships in physical systems, biological structures, financial networks and more. To achieve this, it will study key insights and mechanisms from machine learning, research on higher-order complex structures and its own recent work, while utilising these findings to address challenges in triadic interaction, single-cell data analysis, generative learning and multimodal perception.

Objective

Graphs represent a mathematical concept used to study interconnected subsystems, by formulating mutual relationships as connected node-pairs. However, traditional graphs are inadequate to capture interactions in many real-world problems (e.g. brain networks, gene expression or financial networks, to name a few), due to limited scope of expression. Recent research draws attention to higher-order complex structures (e.g. hypergraphs, allowing multi-body connection; multilayer networks, encoding multimodal connections; and temporal networks, whose connections change over time). Interaction is also often described as form of network flow (message passing) with respect to graph-like objects. This mechanism is commonly utilised in machine learning, a concept performing a type of relationship inference from large amounts of data to describe the underlying problems. Our recent work provides an extension of network flow to higher-order complex objects, in form of multimodal heterogeneous network flow. This is important as recent research has demonstrated that most real-world systems exhibit higher-order complex network features, which may benefit from learning-based relationship inference over higher-order complex structures. We demonstrate existence of the proposed higher-order multimodal node relationship in physical systems, financial networks, and biological structures, among others. Given limited scope of expression in existing frameworks, our proposed framework offers new, previously unexplored insight potential, transcending graph transformers. To that end, following open challenges will be addressed: triadic interaction and generative learning (expanding scope of expression in generative adversarial models); single-cell data analysis (assessing gene expression programmes for cancer therapy, in line with its expressed need for the new paradigm); and multimodal perception (offering new insights to the neuroscientific basis for haptic-enhanced multimodal interaction).

Fields of science (EuroSciVoc)

Keywords

Programme(s)

Topic(s)

Funding Scheme

HORIZON-TMA-MSCA-PF-EF - HORIZON TMA MSCA Postdoctoral Fellowships - European Fellowships

See all projects funded under this funding scheme

Call for proposal

(opens in new window) HORIZON-MSCA-2025-PF

See all projects funded under this call

Coordinator

TECHNISCHE UNIVERSITAET MUENCHEN
Net EU contribution
€ 217 965,12
Address
Arcisstrasse 21
80333 Muenchen
Germany
Region
Bayern Oberbayern München, Kreisfreie Stadt
Activity type
Higher or Secondary Education Establishments
Links
Total cost
No data