Periodic Reporting for period 4 - HARMONIC (Discrete harmonic analysis for computer science)
Reporting period: 2023-09-01 to 2025-02-28
The goal of this project has been to develop the mathematical field of Discrete Harmonic Analysis, and apply to it computer science and to mathematics.
Discrete Harmonic Analysis is a generalization of Boolean Function Analysis, a fundamental tool originating in social choice theory and having many applications in theoretical computer science and combinatorics.
Boolean Function Analysis constitutes the study of functions on the Boolean cube using spectral tools; it is a discrete analog of the Fourier transform.
In contrast, Discrete Harmonic Analysis studies functions on more complicated domains which naturally appear in theoretical computer science and combinatorics, such as the symmetric group.
While the project is theoretical in nature, it has the potential for real-world applicability via PCPs and related cryptographic protocols, which have been applied to cryptocurrencies and could have further applications.
At the outset of the project, the study of Discrete Harmonic Analysis was at its infancy.
The duration of the project has seen significant advances to Discrete Harmonic Analysis, both by the PI and (independently) by several other groups of researchers, demonstrating both its timeliness and the feasibility of many of its objectives.
The various teams together realized many of the objectives suggested in the project, culminating in the recent (2024) construction of PCPs using HDXs by Bafna, Minzer and Vyas.
Discrete Harmonic Analysis is a generalization of Boolean Function Analysis, a fundamental tool originating in social choice theory and having many applications in theoretical computer science and combinatorics.
Boolean Function Analysis constitutes the study of functions on the Boolean cube using spectral tools; it is a discrete analog of the Fourier transform.
In contrast, Discrete Harmonic Analysis studies functions on more complicated domains which naturally appear in theoretical computer science and combinatorics, such as the symmetric group.
While the project is theoretical in nature, it has the potential for real-world applicability via PCPs and related cryptographic protocols, which have been applied to cryptocurrencies and could have further applications.
At the outset of the project, the study of Discrete Harmonic Analysis was at its infancy.
The duration of the project has seen significant advances to Discrete Harmonic Analysis, both by the PI and (independently) by several other groups of researchers, demonstrating both its timeliness and the feasibility of many of its objectives.
The various teams together realized many of the objectives suggested in the project, culminating in the recent (2024) construction of PCPs using HDXs by Bafna, Minzer and Vyas.
The original proposal suggested a systematic development of Discrete Harmonic Analysis (Objective 0), focusing on three domains: the Grassmann scheme (Objective 1), high-dimensional expanders (Objective 2), and the quantum hypercube (Objective 3).
As part of Objective 0, a systematic theory of complexity measures of Boolean functions on various domains has been developed, by the PI, his collaborators, and several students (part of it remains unpublished).
As an application, we were able to determine the structure of maximum size 2-intersecting families of permutations, a classical problem in extremal combinatorics which has so far resisted attacks.
Our systematic theory applies to domains such as the slice, the symmetric group, and the perfect matching scheme, but does not apply to domains such as the Grassmann scheme.
Regarding Objective 1, the period just prior to the initiation of the project saw the advent of a new general tool, global hypercontractivity, by Noam Lifshitz.
This tool serves as a unifying principle for Discrete Harmonic Analysis on many domains, and in particular, gives the "right" way to consider the Grassmann scheme.
Due to this, we directed our efforts to the complexity measures direction, so far unsuccessfully.
As part of Objective 2, we developed Discrete Harmonic Analysis on high-dimensional expanders, by viewing them as "approximately sequentially differential".
As an application, we construct explicit hard instances for Sum-of-Squares. Our work was later improved by a group which went beyond simplicial complexes, but otherwise closely followed our footsteps.
Regarding Objective 3, unfortunately we were unable to find proper collaborators, and due to this, we decided to direct our attention at several other related topics: Discrete Harmonic Analysis on the symmetric group and the slice, and Approximate Polymorphisms.
The symmetric group is perhaps the most natural object of study which goes beyond Boolean Function Analysis.
The PI was part of the initial effort of developing Discrete Harmonic Analysis on the symmetric group via the route of global hypercontractivity.
Following this, we developed a notion of "Fourier expansion" for functions on the symmetric group, as well as on the perfect matching scheme.
Additionally, we proved a tight structure theorem for Boolean function close to degree 1, improving on several earlier works by the PI.
This is an analog of the classical Friedgut-Kalai-Naor theorem for the symmetric group; the proof is much more involved.
The slice is an even simpler domain than the symmetric group, closely related to the biased Boolean cube.
When the slice is relatively balanced, it indeed behaves very similarly to the Boolean cube with the matching bias.
This is no longer the case when the slice is heavily skewed.
We determined the exact point at which this happens, by identifying the "junta threshold" for constant degree functions.
Related to the preceding two topics is our work on "sparse juntas" which describes the structure of Boolean functions close to degree d on the biased Boolean cube (or on the corresponding slice).
Our highly tight structure theorem enabled us to determine several "critical exponents".
The other area which we focused our attention on was Approximate Polymorphisms, an area motivated by Universal Algebra (and its applications to the study of CSPs in computer science), Property Testing, and PCPs.
We were able to show that in the case of functional Boolean predicates, approximate polymorphisms are close to exact polymorphisms; in upcoming work, we generalize this to arbitrary Boolean polymorphisms.
Additionally, we were able to determine the "pseudorandomness threshold" for the functional AND predicate.
As part of Objective 0, a systematic theory of complexity measures of Boolean functions on various domains has been developed, by the PI, his collaborators, and several students (part of it remains unpublished).
As an application, we were able to determine the structure of maximum size 2-intersecting families of permutations, a classical problem in extremal combinatorics which has so far resisted attacks.
Our systematic theory applies to domains such as the slice, the symmetric group, and the perfect matching scheme, but does not apply to domains such as the Grassmann scheme.
Regarding Objective 1, the period just prior to the initiation of the project saw the advent of a new general tool, global hypercontractivity, by Noam Lifshitz.
This tool serves as a unifying principle for Discrete Harmonic Analysis on many domains, and in particular, gives the "right" way to consider the Grassmann scheme.
Due to this, we directed our efforts to the complexity measures direction, so far unsuccessfully.
As part of Objective 2, we developed Discrete Harmonic Analysis on high-dimensional expanders, by viewing them as "approximately sequentially differential".
As an application, we construct explicit hard instances for Sum-of-Squares. Our work was later improved by a group which went beyond simplicial complexes, but otherwise closely followed our footsteps.
Regarding Objective 3, unfortunately we were unable to find proper collaborators, and due to this, we decided to direct our attention at several other related topics: Discrete Harmonic Analysis on the symmetric group and the slice, and Approximate Polymorphisms.
The symmetric group is perhaps the most natural object of study which goes beyond Boolean Function Analysis.
The PI was part of the initial effort of developing Discrete Harmonic Analysis on the symmetric group via the route of global hypercontractivity.
Following this, we developed a notion of "Fourier expansion" for functions on the symmetric group, as well as on the perfect matching scheme.
Additionally, we proved a tight structure theorem for Boolean function close to degree 1, improving on several earlier works by the PI.
This is an analog of the classical Friedgut-Kalai-Naor theorem for the symmetric group; the proof is much more involved.
The slice is an even simpler domain than the symmetric group, closely related to the biased Boolean cube.
When the slice is relatively balanced, it indeed behaves very similarly to the Boolean cube with the matching bias.
This is no longer the case when the slice is heavily skewed.
We determined the exact point at which this happens, by identifying the "junta threshold" for constant degree functions.
Related to the preceding two topics is our work on "sparse juntas" which describes the structure of Boolean functions close to degree d on the biased Boolean cube (or on the corresponding slice).
Our highly tight structure theorem enabled us to determine several "critical exponents".
The other area which we focused our attention on was Approximate Polymorphisms, an area motivated by Universal Algebra (and its applications to the study of CSPs in computer science), Property Testing, and PCPs.
We were able to show that in the case of functional Boolean predicates, approximate polymorphisms are close to exact polymorphisms; in upcoming work, we generalize this to arbitrary Boolean polymorphisms.
Additionally, we were able to determine the "pseudorandomness threshold" for the functional AND predicate.
In this part, let me outline some directions for future research:
- Developing a theory of complexity measures of functions on the Grassmann scheme
The proof of the 2-to-2 conjecture utilized ad hoc arguments on the Grassmann scheme.
These were systematized and dramatically simplified using global hypercontractivity.
Unpublished preliminary results by Dor Minzer's group at MIT indicate that these tools suffice to determine the structure of Boolean functions close to degree 1, at least in some parameter settings.
Yet these approaches fall short of extending the theory of complexity measures to the Grassmann scheme, and more generally to q-analog domains such as finite matrix groups.
It seems that completely new ideas will be required.
So far the most relevant work is a result of Ihringer which completes the classification of Boolean degree 1 functions (using earlier work together with the PI) for all finite fields.
- Determining the structure of Boolean functions close to degree d on the symmetric group
This direction combines our work on the symmetric group (the case d=1) with our work on the biased Boolean cube (for general d).
Initial research has already pointed the way by proving an appropriate "agreement theorem" [unpublished], the main tool underlying the result on the biased Boolean cube.
However, many complications remain.
- Extending the work on approximate polymorphisms to arbitrary finite alphabets
In universal algebra, it is well-known that polymorphisms on non-Boolean alphabets are a lot harder to understand than in the Boolean setting.
The study of approximate polymorphisms is no exception, though the rather different nature of the tools required suggest that the difficulties may not be as severe.
- Adding Boolean Function Analysis and Discrete Harmonic Analysis to Mathlib, the mathematical library of Lean
Recent years have seen the advent of proof assistants, which are programming languages for writing proofs.
As an expert in both topics, the PI is the perfect person to "teach Lean" these topics, enabling their usage in Lean proofs.
- Developing a theory of complexity measures of functions on the Grassmann scheme
The proof of the 2-to-2 conjecture utilized ad hoc arguments on the Grassmann scheme.
These were systematized and dramatically simplified using global hypercontractivity.
Unpublished preliminary results by Dor Minzer's group at MIT indicate that these tools suffice to determine the structure of Boolean functions close to degree 1, at least in some parameter settings.
Yet these approaches fall short of extending the theory of complexity measures to the Grassmann scheme, and more generally to q-analog domains such as finite matrix groups.
It seems that completely new ideas will be required.
So far the most relevant work is a result of Ihringer which completes the classification of Boolean degree 1 functions (using earlier work together with the PI) for all finite fields.
- Determining the structure of Boolean functions close to degree d on the symmetric group
This direction combines our work on the symmetric group (the case d=1) with our work on the biased Boolean cube (for general d).
Initial research has already pointed the way by proving an appropriate "agreement theorem" [unpublished], the main tool underlying the result on the biased Boolean cube.
However, many complications remain.
- Extending the work on approximate polymorphisms to arbitrary finite alphabets
In universal algebra, it is well-known that polymorphisms on non-Boolean alphabets are a lot harder to understand than in the Boolean setting.
The study of approximate polymorphisms is no exception, though the rather different nature of the tools required suggest that the difficulties may not be as severe.
- Adding Boolean Function Analysis and Discrete Harmonic Analysis to Mathlib, the mathematical library of Lean
Recent years have seen the advent of proof assistants, which are programming languages for writing proofs.
As an expert in both topics, the PI is the perfect person to "teach Lean" these topics, enabling their usage in Lean proofs.