Periodic Reporting for period 3 - PMSB (Principles of Musical Structure Building: Theory, Computation, and Cognition)
Periodo di rendicontazione: 2021-03-01 al 2022-08-31
Music is a central human trait across all cultures and historical periods, and it involves a rich variety of parameters, ways of structure building and related cognitive processes. Exploring the potential of syntactic structures in music and their cognition, this research project aims at advancing our understanding of human cognition and, specifically, the capacity to represent and process complex auditory sequences and syntactic structures. The project is further highly relevant for understanding how different kinds of structures are employed in music, which may inform a large audience of musicologists, composers, practicing musicians as well as interested members of the general society. Advancing from the PI’s previous work, this interdisciplinary research program is divided into three core strands:
(A) The theoretical strand is devoted to the investigation of formal principles that govern musical syntax and structure building in Western and non-Western music. Many current approaches share the expressive power of (tree-based) context-free grammars and are mostly confined to harmonic syntax only. Therefore, they face difficulty in expressing certain essential syntactic phenomena in music. To address this lacuna, the theoretical strand sets out to develop a novel unified theory of musical syntax that reconciles harmonic structure with dependencies at the level of the single note and voice leading within a single coherent framework.
(B) Addressing the lack of machine-readable corpora in music research, a large corpus of digitized syntactic analyses are to be compiled and published in the context of the project’s computational strand. Further, the unified syntax theory will be implemented based on a generalized parsing approaches and graph grammars, and trained and evaluated using the analytical corpus.
(C) The formal framework developed in (A) entails specific predictions about mental representations of musical structure and cognitive processes, which will be empirically tested in the experimental strand of the project. Particular focus will be on aspects of nonlocal dependency relations, learnability, revision, and tension. The outcomes of the project will make a significant contribution to both the field of music cognition and to the cognitive sciences in general.
(A) The theoretical strand is devoted to the investigation of formal principles that govern musical syntax and structure building in Western and non-Western music. Many current approaches share the expressive power of (tree-based) context-free grammars and are mostly confined to harmonic syntax only. Therefore, they face difficulty in expressing certain essential syntactic phenomena in music. To address this lacuna, the theoretical strand sets out to develop a novel unified theory of musical syntax that reconciles harmonic structure with dependencies at the level of the single note and voice leading within a single coherent framework.
(B) Addressing the lack of machine-readable corpora in music research, a large corpus of digitized syntactic analyses are to be compiled and published in the context of the project’s computational strand. Further, the unified syntax theory will be implemented based on a generalized parsing approaches and graph grammars, and trained and evaluated using the analytical corpus.
(C) The formal framework developed in (A) entails specific predictions about mental representations of musical structure and cognitive processes, which will be empirically tested in the experimental strand of the project. Particular focus will be on aspects of nonlocal dependency relations, learnability, revision, and tension. The outcomes of the project will make a significant contribution to both the field of music cognition and to the cognitive sciences in general.
Within the three linked research strands of the project (theoretical, computational, experimental), numerous research results have been achieved:
In the theoretical line of the project several advancements have been made with regard to different musical parameters, in particular, harmony (chord sequences), extended tonality, rhythm, melody, polyphony and voice-leading. So far, these resulted in articles on the syntax of Jazz harmony, rhythm, and North-Indian melody.
In the computational line of the project two applications to annotate musical pieces with expert analyses are developed. This resulted in a published digital database of hierarchical analyses of Jazz chord sequences. In addition, several computational models were developed, which process syntactic structure in music and learn grammars and their properties from examples (so called parsing and grammar inference). In order to approximate complex hierarchical structure in music, another model of hierarchical key scapes was developed. In order to model properties of extended tonality (which occurs in Western music from the 19th century on), a statistical tonal diffusion model was developed, which models pitch distributions over the geometric pitch space of the Tonnetz.
In the experimental line of the project, several behavioral experiments examining listeners’ perception of syntactic structures in music were carried out. One published study examined the interaction between harmony (chords) and voice-leading (the melodic lines of single notes). A novel method to study the phenomenon of musical revision was developed and used to study how listeners reanalyze a fragment of music after they hear new continuations that are in conflict with previous hearings. Furthermore, new methods for examining hierarchicality and embedding in music were designed, which are currently being explored.
In the theoretical line of the project several advancements have been made with regard to different musical parameters, in particular, harmony (chord sequences), extended tonality, rhythm, melody, polyphony and voice-leading. So far, these resulted in articles on the syntax of Jazz harmony, rhythm, and North-Indian melody.
In the computational line of the project two applications to annotate musical pieces with expert analyses are developed. This resulted in a published digital database of hierarchical analyses of Jazz chord sequences. In addition, several computational models were developed, which process syntactic structure in music and learn grammars and their properties from examples (so called parsing and grammar inference). In order to approximate complex hierarchical structure in music, another model of hierarchical key scapes was developed. In order to model properties of extended tonality (which occurs in Western music from the 19th century on), a statistical tonal diffusion model was developed, which models pitch distributions over the geometric pitch space of the Tonnetz.
In the experimental line of the project, several behavioral experiments examining listeners’ perception of syntactic structures in music were carried out. One published study examined the interaction between harmony (chords) and voice-leading (the melodic lines of single notes). A novel method to study the phenomenon of musical revision was developed and used to study how listeners reanalyze a fragment of music after they hear new continuations that are in conflict with previous hearings. Furthermore, new methods for examining hierarchicality and embedding in music were designed, which are currently being explored.
The main research results outlined above constitute research results that advance the current state of the art in the theoretical, computational and experimental subfields. Until the end of the project, some of the following outcomes are expected:
- a formalization of extended harmony and Tonfeld theory
- a model of polyphonic musical syntax based on graph grammars
- a general model of hierarchical scapes
- a formalization of aspects of musical syntax that generalize to Non-Western music, exemplified for Classical North Indian music
- Further experimental results on listeners’ processing and mental representation of syntactic structures in music.
- a formalization of extended harmony and Tonfeld theory
- a model of polyphonic musical syntax based on graph grammars
- a general model of hierarchical scapes
- a formalization of aspects of musical syntax that generalize to Non-Western music, exemplified for Classical North Indian music
- Further experimental results on listeners’ processing and mental representation of syntactic structures in music.