Computing Nonlinear Functions over Communication Networks

Project Information

SENSIBILITÉ

Grant agreement ID: 101077361

DOI

10.3030/101077361

EC signature date 9 February 2023

Start date 1 May 2023

End date 30 April 2029

Funded under

• European Research Council (ERC)

Total cost

€ 1 499 061,25

EU contribution

€ 1 499 061,00

Coordinated by

EURECOM GIE
France

Periodic Reporting for period 1 - SENSIBILITÉ (Computing Nonlinear Functions over Communication Networks)

Reporting period: 2023-05-01 to 2025-10-31

Summary of the context and overall objectives of the project

SENSIBILITÉ describes a novel theory for distributed computing of nonlinear functions over communication networks. Motivated by the long-lasting open challenge to invent technologies that scale with the network size, this intriguing and far-reaching theory elevates distributed encoding and joint decoding of information sources, to the critical network computing problem for a class of network topologies and a class of nonlinear functions of dependent sources. Our theory will elevate distributed communication to the realm of distributed computation of any function over any network.

Overall, this problem requires communicating correlated messages over a network, coding distributed sources for computation of functions, and meeting the desired fidelity given a distortion criterion for the given function. In such a scenario, the classical separation theorem of Claude Shannon, which modularizes the design of source and channel codes to achieve the capacity of communication channels, is in general inapplicable.

SENSIBILITÉ envisions a networked computation framework for nonlinear functions. It will use the structural information of the sources and the decomposition of nonlinear functions for efficient distributed compression algorithms. For scalability, it will design message sets that are oblivious to the protocol information. For parsimonious representations across networks, it will grip the curious trade-off between quantization and compression of functions. SENSIBILITÉ has a contemporary vision of network-driven functional compression via accounting for the description length and time complexities towards alleviating large-scale, real-world networks of the future. The advanced theory will be tested in a real-life setting on applications of grand societal impact, such as over-the-air computing for the internet-of-things, massive data compression for computational imaging, and zero-error computation for real-time holographic communications.

Work performed from the beginning of the project to the end of the period covered by the report and main results achieved so far

Below, we summarize several significant achievements that advance the field of distributed computation.

-Structured Codes for Distributed Matrix Multiplication (arXiv:2501.00371)
We develop encoding schemes for distributed source compression tailored to nonlinear transformations, including inner products, symmetric matrix products, and general square matrix products over finite fields. Our approach combines nonlinear mappings of distributed sources A∈Fq^(m×l) and B∈Fq^(m×l) with the structured linear encoding scheme by Körner and Marton, ensuring distributed computation without requiring full data disclosure. We derive achievable and converse bounds on the sum rate for distributed computation of inner products and general matrix products A^T B. Leveraging the lower bounds on the communication cost set by Han-Kobayashi, and Slepian-Wolf, and our scheme that captures the structure of matrix computation, we demonstrate unbounded compression gains over Slepian-Wolf coding in cases with specific source correlations.

We also introduce structured polynomial codes (StPolyDot codes) designed for distributed matrix multiplication in a master-workers-receiver framework. Our results show that StPolyDot codes retain the optimality of the communication cost of Dutta et al.'s approach within a factor of 2, with only a bounded increase in computation cost, inversely proportional to the memory parameter.

-Multi-Server Multi-Function Distributed Computation (ENTROPY2024)
We study the communication cost for a multi-server multi-task distributed computation framework, for a broad class of functions and data statistics. Considering the framework where a user seeks the computation of multiple complex (conceivably non-linear) tasks from a set of distributed servers, we establish communication cost upper bounds for a variety of data statistics, function classes and data placements across the servers. To do so, we proceed to apply, for the first time here, Körner's characteristic graph approach --- known to capture the structural properties of data and functions --- to the promising framework of multi-server multi-task distributed computing. Going beyond the general expressions, we also consider the well-known scenario of cyclic dataset placement and linearly separable functions over the binary field, in which case our approach exhibits considerable gains over the state of the art. Similar gains are identified for the case of multi-linear functions.

-Non-Linear Function Computation Broadcast (ISIT2025)
This work addresses the K-user computation broadcast problem consisting of a master node, that holds all datasets and users for a general class of function demands, including linear and nonlinear functions, over finite fields. The master node sends a broadcast message to enable each of K distributed users to compute their demanded function in an asymptotically lossless manner with user’s side information. We derive bounds on the optimal K-user computation broadcast rate that allows the users to compute their demanded functions by capturing the structures of the computations and available side information. Our achievability scheme involves the design of a novel graph-based coding model to build a broadcast message drawing on Körner’s characteristic graph framework. The converse uses the structures of the demands and the side information available at K users to yield a tight lower bound on the broadcast rate.

Progress beyond the state of the art and expected potential impact (including the socio-economic impact and the wider societal implications of the project so far)

Our approach will re-engineer distributed function computation over networks to incorporate various multi-user communication constraints and capabilities, particularly in multicast, broadcast, and multi-access environments. By blending tools from coding theory, graph theory, source and channel coding, compressive sensing, and task-aware communication, we aim to achieve:
• A multiplicative factor reduction in communication volume over existing techniques.
• Greater energy efficiency and reduced infrastructure cost, essential for sustainable next-generation wireless deployments.

The recent evolution of parallel and distributed algorithms to process big data sets and large language models has exposed the crucial role of communication-computation co-design. Thus, scalable computation frameworks should simultaneously capture encoding, processing, and coordination strategies for the computation of various tasks abstracted by functions. Our vision is to tackle the growing heterogeneous demand by exploiting
• Decentralized topologies with mixed multicast/unicast patterns.
• Correlated data and side information across devices.
• Sparse computation tasks, and function-oriented evaluations.

This calls for a fundamentally new theoretical and practical framework that can capture the recent trends and emerging architectures to be supported by communication systems. In this space, our work will address:
• Exploitation of data correlation, function structure, and topology to reduce transmission load.
• Straggler mitigation and device heterogeneity, critical in real-world deployment scenarios.
• Novel coding and representation strategies for sparse and function-oriented computing.

Our research aligns with Europe’s and the global community’s priorities for green digital transformation, sovereign AI infrastructure, and equitable access to intelligence. Over the next five years, we will pursue these goals through collaborative projects, open testbed development, and targeted contributions to international standardization efforts.

A high level diagram that explains the confluence of structures of data, function, and network.