Periodic Reporting for period 3 - SONATA (Semantics-empowered Wireless Connectivity: Theoretical and Algorithmic Foundations)
Reporting period: 2024-04-01 to 2025-09-30
Summary of the context and overall objectives of the project
• What is the problem/issue being addressed?
We seek the theoretical foundations of goal-oriented semantic communication, and pursue the mathematical convergence between event- and signal-aware data generation and processing, and goal-oriented information transmission and media reconstruction. Our aim is to develop a novel communication theory and to design practical and powerful semantics-empowered algorithms. This new communication paradigm promises to fundamentally transform several prevailing design principles and to unearth a hidden cohesion between data generation, transmission, and utilization. SONATA envisions that information-bearing signals are semantically processed, filtered, and controlled from the very beginning of the communication process, i.e. during data generation, acquisition, and transfer. This is the key to meaningfully reducing the problem dimensionality by removing redundancy (irrelevant or useless information through goal-driven sparse data/signal representations) and meeting the application objectives by gathering and prioritizing information according to its anticipated importance and effectiveness in achieving the end user’s goals at the destination.
• Why is it important for society?
SONATA has the potential to create a new unified theory that, if successfully developed, will radically transform the way we generate, transmit, and reconstruct data in time-sensitive and data-intensive communication systems. It could reveal how by jointly optimizing the entire communication lifecycle (data generation, processing, transmission, usage) under the prism of semantics (importance and usefulness) of information and guided by goal-oriented objectives, we can enable terse and timely delivery of valuable and effective information in a variety of envisioned scalable and resource-efficient network architectures. The scientific developments of SONATA will pave the way towards fundamentally new concepts of information handling. The implementation of a semantic design methodology leveraging goal-oriented data importance, filtering, and prioritization could enable a significant reduction in the communication, computing, and energy resources required for data processing and transmission. The proposed methodology will advance the technology of data processing and will render distributed sensing, decision-making, and machine learning, faster, scalable, and more efficient. Finally, SONATA will provide indispensable technology for emerging cyber-physical systems and socially beneficial services, including connected robotics, smart cities, autonomous transportation, healthcare, and industry automation, by unlocking the power of resource-intensive connected devices with sophisticated sensing and computing capabilities.
• What are the overall objectives?
The first objective of SONATA is to explore the fundamentals of goal-oriented semantic information and communication. The key idea is to establish new, insightful, operational, and amenable to analysis semantics-aware metrics and information measures, which incorporate the notions of importance, timing, and effectiveness into the existing information and communication theoretic edifice. These metrics should capture both the objective, quantitative, inherent attributes of information generated, as well as the subjective, qualitative, context-dependent, goal-oriented, and end-user/application perceived information utility.
The second scientific objective is to develop a theory of optimal joint active sampling, communication, and source reconstruction of multidimensional signals by exploiting sparsity, correlation, and information semantics. This passes through the establishment of the theoretical framework for joint sampling and real-time reconstruction under communication constraints and delays, the introduction of new functionalities, such as semantic filtering, semantic control, and semantic reconstruction for certain promising scenarios, as well as the development of a theory that optimally characterize the tradeoffs between information rate, distortion, and semantic perceptual quality.
The third objective is to design algorithmic solutions for semantic-empowered communication. The aim is to develop transmission, multiple access, and resource allocation techniques, which can harvest the potential high gains of goal-oriented semantic communication. This involves the introduction of semantic multiple access, risk-averse decision-making for resource allocation under promises/prospects, and real-time scheduling under multiple choices
Another objective includes exploring semantics-empowered wireless connectivity in large network topologies. The focus is on the investigation of how network topology affects the performance of semantics-aware communication algorithms and how to adapt them to different network topologies.
The last objective involves proof-of-concept and real-world testbed experimentation of goal-oriented semantic communication algorithms. In this context, we wish to conduct extensive experimental validation and feasibility studies of the algorithmic outcomes.
We seek the theoretical foundations of goal-oriented semantic communication, and pursue the mathematical convergence between event- and signal-aware data generation and processing, and goal-oriented information transmission and media reconstruction. Our aim is to develop a novel communication theory and to design practical and powerful semantics-empowered algorithms. This new communication paradigm promises to fundamentally transform several prevailing design principles and to unearth a hidden cohesion between data generation, transmission, and utilization. SONATA envisions that information-bearing signals are semantically processed, filtered, and controlled from the very beginning of the communication process, i.e. during data generation, acquisition, and transfer. This is the key to meaningfully reducing the problem dimensionality by removing redundancy (irrelevant or useless information through goal-driven sparse data/signal representations) and meeting the application objectives by gathering and prioritizing information according to its anticipated importance and effectiveness in achieving the end user’s goals at the destination.
• Why is it important for society?
SONATA has the potential to create a new unified theory that, if successfully developed, will radically transform the way we generate, transmit, and reconstruct data in time-sensitive and data-intensive communication systems. It could reveal how by jointly optimizing the entire communication lifecycle (data generation, processing, transmission, usage) under the prism of semantics (importance and usefulness) of information and guided by goal-oriented objectives, we can enable terse and timely delivery of valuable and effective information in a variety of envisioned scalable and resource-efficient network architectures. The scientific developments of SONATA will pave the way towards fundamentally new concepts of information handling. The implementation of a semantic design methodology leveraging goal-oriented data importance, filtering, and prioritization could enable a significant reduction in the communication, computing, and energy resources required for data processing and transmission. The proposed methodology will advance the technology of data processing and will render distributed sensing, decision-making, and machine learning, faster, scalable, and more efficient. Finally, SONATA will provide indispensable technology for emerging cyber-physical systems and socially beneficial services, including connected robotics, smart cities, autonomous transportation, healthcare, and industry automation, by unlocking the power of resource-intensive connected devices with sophisticated sensing and computing capabilities.
• What are the overall objectives?
The first objective of SONATA is to explore the fundamentals of goal-oriented semantic information and communication. The key idea is to establish new, insightful, operational, and amenable to analysis semantics-aware metrics and information measures, which incorporate the notions of importance, timing, and effectiveness into the existing information and communication theoretic edifice. These metrics should capture both the objective, quantitative, inherent attributes of information generated, as well as the subjective, qualitative, context-dependent, goal-oriented, and end-user/application perceived information utility.
The second scientific objective is to develop a theory of optimal joint active sampling, communication, and source reconstruction of multidimensional signals by exploiting sparsity, correlation, and information semantics. This passes through the establishment of the theoretical framework for joint sampling and real-time reconstruction under communication constraints and delays, the introduction of new functionalities, such as semantic filtering, semantic control, and semantic reconstruction for certain promising scenarios, as well as the development of a theory that optimally characterize the tradeoffs between information rate, distortion, and semantic perceptual quality.
The third objective is to design algorithmic solutions for semantic-empowered communication. The aim is to develop transmission, multiple access, and resource allocation techniques, which can harvest the potential high gains of goal-oriented semantic communication. This involves the introduction of semantic multiple access, risk-averse decision-making for resource allocation under promises/prospects, and real-time scheduling under multiple choices
Another objective includes exploring semantics-empowered wireless connectivity in large network topologies. The focus is on the investigation of how network topology affects the performance of semantics-aware communication algorithms and how to adapt them to different network topologies.
The last objective involves proof-of-concept and real-world testbed experimentation of goal-oriented semantic communication algorithms. In this context, we wish to conduct extensive experimental validation and feasibility studies of the algorithmic outcomes.
Work performed from the beginning of the project to the end of the period covered by the report and main results achieved so far
Some of the work performed since the beginning of the project can be found below.
The objective of WP1 is to investigate the fundamentals of semantics-empowered communication. The main research achievements along the main activities and the planned outcomes are as follows:
• Formal, generic, and operational definition of the concept of “semantics of information”, as a composite, multidimensional function of innate and contextual information attributes, capturing the importance and usefulness of information with respect to the goal of data exchange and the application requirements.
• Theoretical analysis of goal-oriented semantic filtering and timely source coding in point-to-point systems, and multiuser communication systems with both homogeneous and heterogeneous goals. Derivation of the optimal codeword lengths in the sense of maximizing a weighted sum of semantic utility functions for all pairs of communicating entities (sensors and monitors).
• Characterization of fundamental tradeoffs among innate and/or quantitative and qualitative information attributes and understanding of the effect of different attributes in different building blocks of the communication process.
• Optimal query control and sampling policies for maximizing information importance in pull-based systems. First analysis of an effective communication system that meaningfully bridges the push-based with the pull-based model under the prism of the semantics of information. Optimization based on the impact a status update has in the system, measured by a grade of effectiveness metric, which incorporates both freshness and usefulness attributes of the communicated updates.
WP2 is dedicated to developing joint semantics-aware sampling/processing, communication, and reconstruction techniques. The main research achievements along the main activities and the planned outcomes are as follows:
• Introduction of a novel class of joint active sampling and communication policies for real-time tracking and remote source reconstruction.
• Introduction of goal-oriented, timing-sensitive metrics for remote actuation and theoretic analysis of such system error metrics, including the real-time reconstruction error, the importance-aware consecutive error, the cost of memory error, and the cost of actuation error.
• Proposition of a randomized stationary sampling and transmission policy, which is formally shown to outperform other policies in terms of real-time reconstruction error minimization under constrained sampling generation, for rapidly evolving information sources.
• Establishment of a general statistical framework of timing requirements in wireless communication systems, which subsumes both latency and age of information. The framework is made by associating a timing component with the two basic statistical operations, decision and estimation.
• Introduction of a rate-distortion framework, in particular a variant of a robust description source coding framework, as a relevant model for goal-oriented semantic information transmission.
• First information theoretic analysis of indirect rate-distortion for finite alphabets, with two separable distortion constraints, as well as for an f-separable distortion criterion. Characterization and analysis of the cardinal role of context-dependent fidelity criteria in goal-oriented semantic communication and introduction of a general, semantics-aware Blahut-Arimoto algorithm.
• First information-theoretic analysis of goal-oriented lossy joint source-channel coding, introducing the criteria needed for the optimality of goal-oriented single-letter codes.
• Computation of the rate-distortion-perception function for discrete memoryless sources subject to a single-letter average distortion constraint and a perception constraint that belongs to the family of f-divergences. Parametric characterization of the optimal solution and introduction of a relaxed iterative algorithm based on the alternating minimization technique.
• Computation of the rate-distortion-perception function for a multivariate Gaussian source under mean squared error distortion and various perception metrics, namely Kullback-Leibler divergence, geometric Jensen-Shannon divergence, squared Hellinger distance, and squared Wasserstein-2 distance.
The focus of WP3 is on communication techniques and multiple access schemes that will support semantic wireless connectivity. The main research achievements are the following:
• Introduction of a novel random access strategy, which exploits the continuous angular group sparsity feature of wireless channels and solves a reconstruction-free goal-oriented optimization problem. The proposed blind goal-oriented multiple access scheme provides low latency, high reliability, and massive access with limited bandwidth resources in an all-in-one package, and more importantly, its performance gains do not depend on the number of devices.
• Exploration of the framework of robust Bayesian learning, showcasing its merits on several important wireless communication problems in terms of accuracy, calibration, and robustness to outliers and misspecification.
The objective of WP1 is to investigate the fundamentals of semantics-empowered communication. The main research achievements along the main activities and the planned outcomes are as follows:
• Formal, generic, and operational definition of the concept of “semantics of information”, as a composite, multidimensional function of innate and contextual information attributes, capturing the importance and usefulness of information with respect to the goal of data exchange and the application requirements.
• Theoretical analysis of goal-oriented semantic filtering and timely source coding in point-to-point systems, and multiuser communication systems with both homogeneous and heterogeneous goals. Derivation of the optimal codeword lengths in the sense of maximizing a weighted sum of semantic utility functions for all pairs of communicating entities (sensors and monitors).
• Characterization of fundamental tradeoffs among innate and/or quantitative and qualitative information attributes and understanding of the effect of different attributes in different building blocks of the communication process.
• Optimal query control and sampling policies for maximizing information importance in pull-based systems. First analysis of an effective communication system that meaningfully bridges the push-based with the pull-based model under the prism of the semantics of information. Optimization based on the impact a status update has in the system, measured by a grade of effectiveness metric, which incorporates both freshness and usefulness attributes of the communicated updates.
WP2 is dedicated to developing joint semantics-aware sampling/processing, communication, and reconstruction techniques. The main research achievements along the main activities and the planned outcomes are as follows:
• Introduction of a novel class of joint active sampling and communication policies for real-time tracking and remote source reconstruction.
• Introduction of goal-oriented, timing-sensitive metrics for remote actuation and theoretic analysis of such system error metrics, including the real-time reconstruction error, the importance-aware consecutive error, the cost of memory error, and the cost of actuation error.
• Proposition of a randomized stationary sampling and transmission policy, which is formally shown to outperform other policies in terms of real-time reconstruction error minimization under constrained sampling generation, for rapidly evolving information sources.
• Establishment of a general statistical framework of timing requirements in wireless communication systems, which subsumes both latency and age of information. The framework is made by associating a timing component with the two basic statistical operations, decision and estimation.
• Introduction of a rate-distortion framework, in particular a variant of a robust description source coding framework, as a relevant model for goal-oriented semantic information transmission.
• First information theoretic analysis of indirect rate-distortion for finite alphabets, with two separable distortion constraints, as well as for an f-separable distortion criterion. Characterization and analysis of the cardinal role of context-dependent fidelity criteria in goal-oriented semantic communication and introduction of a general, semantics-aware Blahut-Arimoto algorithm.
• First information-theoretic analysis of goal-oriented lossy joint source-channel coding, introducing the criteria needed for the optimality of goal-oriented single-letter codes.
• Computation of the rate-distortion-perception function for discrete memoryless sources subject to a single-letter average distortion constraint and a perception constraint that belongs to the family of f-divergences. Parametric characterization of the optimal solution and introduction of a relaxed iterative algorithm based on the alternating minimization technique.
• Computation of the rate-distortion-perception function for a multivariate Gaussian source under mean squared error distortion and various perception metrics, namely Kullback-Leibler divergence, geometric Jensen-Shannon divergence, squared Hellinger distance, and squared Wasserstein-2 distance.
The focus of WP3 is on communication techniques and multiple access schemes that will support semantic wireless connectivity. The main research achievements are the following:
• Introduction of a novel random access strategy, which exploits the continuous angular group sparsity feature of wireless channels and solves a reconstruction-free goal-oriented optimization problem. The proposed blind goal-oriented multiple access scheme provides low latency, high reliability, and massive access with limited bandwidth resources in an all-in-one package, and more importantly, its performance gains do not depend on the number of devices.
• Exploration of the framework of robust Bayesian learning, showcasing its merits on several important wireless communication problems in terms of accuracy, calibration, and robustness to outliers and misspecification.
Progress beyond the state of the art and expected potential impact (including the socio-economic impact and the wider societal implications of the project so far)
Progress beyond the state of the art has been made in all major research thrusts planned in the Description of the Action. SONATA succeeded in envisioning a new communication paradigm and in offering a comprehensive analysis in the uncharted area, introducing new metrics, models, and techniques for supporting goal-oriented semantic communication. The subject matter of the project has attracted vivid interest and attention from both academia and industry, becoming one of the hottest, most important, and fastest-growing areas in communication research today. The vision of SONATA is now considered a key technology enabler for 6G wireless networks, and our results and publications inspired a number of academic and industrial researchers to build technology for goal-oriented semantic communication systems.
Our mathematical definition of the concept of “semantics of information”, which is instrumental to unearthing the hidden cohesion between data generation, information transmission, and media reconstruction, is a major step beyond the state of the art. We provided a new general, formal, operational definition of the concept, and we showed how it can be used to reap the performance gains from a goal-oriented semantic-aware communication approach. This has opened a novel line of research in the information- and communication-theoretic literature. Leveraging our newly proposed mathematical definition of “semantics of information”, we proposed several metrics and joint semantic sampling, communication, and reconstruction policies, which provide unprecedented gains in real-time tracking and actuation. Our proposed methodology could drastically reduce (even around 99%) the volume of unnecessary traffic (packets) generated and communicated in a network, resulting in a significant decrease in the amount of wasteful or useless resources (e.g. energy, computation) used.
Our research results on the rate-distortion-perception theoretical framework, which has been identified as a promising model for capturing the semantic reconstruction quality and for measuring the degree of meaningfulness of the reconstructed source from the perspective of the observer and its communication goals/objectives, have significantly advanced the field beyond the state of the art. This framework has established the link between generative learning and semantic communication and revealed an interesting interplay between model-based information-theoretic approaches and data-driven optimization.
Another area where SONATA results have gone beyond the state of the art is leveraging Bayesian statistics and inference and bridging it with information theory and timing-sensitive metrics (e.g. age/value of information and beyond). The proposed model(s) capture the essence of goal-oriented, context- and observer-dependent information valuation.
Our mathematical definition of the concept of “semantics of information”, which is instrumental to unearthing the hidden cohesion between data generation, information transmission, and media reconstruction, is a major step beyond the state of the art. We provided a new general, formal, operational definition of the concept, and we showed how it can be used to reap the performance gains from a goal-oriented semantic-aware communication approach. This has opened a novel line of research in the information- and communication-theoretic literature. Leveraging our newly proposed mathematical definition of “semantics of information”, we proposed several metrics and joint semantic sampling, communication, and reconstruction policies, which provide unprecedented gains in real-time tracking and actuation. Our proposed methodology could drastically reduce (even around 99%) the volume of unnecessary traffic (packets) generated and communicated in a network, resulting in a significant decrease in the amount of wasteful or useless resources (e.g. energy, computation) used.
Our research results on the rate-distortion-perception theoretical framework, which has been identified as a promising model for capturing the semantic reconstruction quality and for measuring the degree of meaningfulness of the reconstructed source from the perspective of the observer and its communication goals/objectives, have significantly advanced the field beyond the state of the art. This framework has established the link between generative learning and semantic communication and revealed an interesting interplay between model-based information-theoretic approaches and data-driven optimization.
Another area where SONATA results have gone beyond the state of the art is leveraging Bayesian statistics and inference and bridging it with information theory and timing-sensitive metrics (e.g. age/value of information and beyond). The proposed model(s) capture the essence of goal-oriented, context- and observer-dependent information valuation.