Periodic Reporting for period 5 - PMSB (Principles of Musical Structure Building: Theory, Computation, and Cognition)
Reporting period: 2024-03-01 to 2025-04-30
Music is a central human trait across all cultures and historical periods, and it involves a rich variety of parameters, ways of structure building and related cognitive processes. Exploring the potential of syntactic structures in music and their cognition, this research project aimed at advancing our understanding of human cognition and, specifically, the capacity to represent and process complex auditory sequences and syntactic structures. The project is further highly relevant for understanding how different kinds of structures are employed in music, which may inform a large audience of musicologists, composers, practicing musicians as well as interested members of the general society. Advancing from the PI’s previous work, this interdisciplinary research program has been divided into three core strands:
(A) The theoretical strand is devoted to the investigation of formal principles that govern musical syntax and structure building in Western and non-Western music. The central goal of the theoretical part was to develop a novel unified theory of musical syntax that reconciles harmonic structure with dependencies at the level of the single note and voice leading within a single coherent framework and to extend these ways of modeling to other musical parameters.
(B) Addressing the lack of machine-readable corpora in music research, the second part of the project set out to compile a corpus of digitized syntactic analyses. Further, the second central aim of the computational part was to develop grammar models to process and parse musical structures based on the theories developed as well as to build different models for heuristics of hierarchical structure in music.
(C) The formal framework developed in (A) entails specific predictions about mental representations of musical structure and cognitive processes, which have been empirically explored in the experimental strand of the project. Particular focus has been be on aspects of musical processing, nonlocal dependency relations, revision, and perceived musical tension.
The outcomes of the project will make a significant contribution to both the field of music cognition and to the cognitive sciences in general. The project helps to understand the central questions of how and why music works the way it does.
(A) The theoretical strand is devoted to the investigation of formal principles that govern musical syntax and structure building in Western and non-Western music. The central goal of the theoretical part was to develop a novel unified theory of musical syntax that reconciles harmonic structure with dependencies at the level of the single note and voice leading within a single coherent framework and to extend these ways of modeling to other musical parameters.
(B) Addressing the lack of machine-readable corpora in music research, the second part of the project set out to compile a corpus of digitized syntactic analyses. Further, the second central aim of the computational part was to develop grammar models to process and parse musical structures based on the theories developed as well as to build different models for heuristics of hierarchical structure in music.
(C) The formal framework developed in (A) entails specific predictions about mental representations of musical structure and cognitive processes, which have been empirically explored in the experimental strand of the project. Particular focus has been be on aspects of musical processing, nonlocal dependency relations, revision, and perceived musical tension.
The outcomes of the project will make a significant contribution to both the field of music cognition and to the cognitive sciences in general. The project helps to understand the central questions of how and why music works the way it does.
This project linked three research strands in the fields of music theory, computational research and experimental psychology, in all of which a range of results have been achieved:
In the theoretical line of the project major progress has been made in terms of achieving an overarching theoretical framework to characterize musical structures in a formal, mathematical way. This pertains to the musical parameters of harmony (chord sequences), extended tonality, rhythm, melody, polyphony and voice-leading. We have extended our results from Western music to the music of other cultures.
In the computational line of the project two web applications to annotate musical pieces with expert analyses have been developed. We have published several datasets, notably a digital dataset of hierarchical harmonic analyses of common Jazz standards. In addition, several computational models were developed, which process syntactic structure in music and learn grammars and their properties from examples (so called parsing and grammar inference). In order to approximate complex hierarchical structure in music, another model of hierarchical key scapes was developed. In order to model properties of extended tonality (which occurs in Western music from the 19th century on), a statistical tonal diffusion model was developed, which models pitch distributions over the geometric pitch space of the Tonnetz. We have also achieved a computational model of free polyphony.
In the experimental line of the project, several behavioral experiments examining listeners’ perception of syntactic structures in music were carried out. We examined the interaction between harmony (chords) and voice-leading (the melodic lines of single notes). A novel method to study the phenomenon of musical revision was developed and used to study how listeners reanalyze a fragment of music after they hear new continuations that are in conflict with previous hearings. Furthermore, new methods for examining hierarchical structure in music and embedding in music were employed to reveal evidence for this kind of processing in the mind. We also used priming studies and click paradigms to find evidence for syntactic structure. We further explored the implications of musical syntax for perceived musical tension.
In the theoretical line of the project major progress has been made in terms of achieving an overarching theoretical framework to characterize musical structures in a formal, mathematical way. This pertains to the musical parameters of harmony (chord sequences), extended tonality, rhythm, melody, polyphony and voice-leading. We have extended our results from Western music to the music of other cultures.
In the computational line of the project two web applications to annotate musical pieces with expert analyses have been developed. We have published several datasets, notably a digital dataset of hierarchical harmonic analyses of common Jazz standards. In addition, several computational models were developed, which process syntactic structure in music and learn grammars and their properties from examples (so called parsing and grammar inference). In order to approximate complex hierarchical structure in music, another model of hierarchical key scapes was developed. In order to model properties of extended tonality (which occurs in Western music from the 19th century on), a statistical tonal diffusion model was developed, which models pitch distributions over the geometric pitch space of the Tonnetz. We have also achieved a computational model of free polyphony.
In the experimental line of the project, several behavioral experiments examining listeners’ perception of syntactic structures in music were carried out. We examined the interaction between harmony (chords) and voice-leading (the melodic lines of single notes). A novel method to study the phenomenon of musical revision was developed and used to study how listeners reanalyze a fragment of music after they hear new continuations that are in conflict with previous hearings. Furthermore, new methods for examining hierarchical structure in music and embedding in music were employed to reveal evidence for this kind of processing in the mind. We also used priming studies and click paradigms to find evidence for syntactic structure. We further explored the implications of musical syntax for perceived musical tension.
The main research results advance the current state of the art in the theoretical, computational and experimental subfields.
Our central results include the following:
- formal models of musical harmony, rhythm, and polyphony at the note-level
- a formalization of extended harmony and Tonfeld theory
- extending our research work to Nonwestern music, in particular, North-Indian music and Chinese music
- a web application to explore, annotate and research hierarchical structure in music
- a dataset of hierarchical analyses of music
- computational models of syntactic structure in music
- heuristic computational models of syntactic structure in music based on the method of hierarchical scapes
- a range of psychological experimental results on listeners’ processing and mental representation of syntactic structures in music
Our central results include the following:
- formal models of musical harmony, rhythm, and polyphony at the note-level
- a formalization of extended harmony and Tonfeld theory
- extending our research work to Nonwestern music, in particular, North-Indian music and Chinese music
- a web application to explore, annotate and research hierarchical structure in music
- a dataset of hierarchical analyses of music
- computational models of syntactic structure in music
- heuristic computational models of syntactic structure in music based on the method of hierarchical scapes
- a range of psychological experimental results on listeners’ processing and mental representation of syntactic structures in music