The fellowship period consisted of Design, Music Creation, Interface Creation, Experimentation, and Dissemination phases.
Design
First, the Fellow completed preliminary tasks required prior to undertaking human subjects research. The tasks included acquiring mobile sensors for measuring the electrical activities of the brain and heart, designing experiments, and obtaining ethics approvals for the experiments.
Music Creation
The Fellow composed two original works. The first, SALO for solo piano, was based on a novel by Kurt Vonnegut. The pitches were largely determined algorithmically from the text in two different styles – perpetual motion and improvisation. This work was performed and recorded on a Disklavier. The recording was subsequently used in experimentation.
An additional work called Follow Me was composed by the Fellow for two percussionists. Follow Me exemplifies a composition using embodied principles – it is performed by overt and synchronized physical actions that go beyond what is necessary to play the sounds indicated in the score. Those actions engage the audience and drive movement- and timing-based predictions.
Both SALO and Follow Me were performed live in public concerts in Italy.
Interface Creation
A tappable MIDI interface was created for embodied, interactive control of CCM. For every tap, the playback of the music adjusted to the tap rate in real time. The tapper was therefore like a conductor controlling the nuances of music as it is being played.
The tapping interface was designed to work with recordings in MIDI format, such as those produced by a Disklavier. The interface controlled music playback, recorded inter-tap interval for offline analysis of tapping patterns and music-tap relationships, and streamed data to a global clock for synchronization.
Experimentation
Experiments were carried out at multiple points throughout the creation-research phases described above. The topics under study were [1] appreciation of CCM, [2] physiological responses to CCM, [3] the impact of tapping on temporal processing, and [4] music appreciation after interactive or non-interactive tapping.
Results
[1] The data showed that CCM was found to be significantly more disliked than non-CCM by non-specialized listeners. Even CCM-derived clips 100ms long were reportedly less liked compared to equivalent non-CCM-derived clips.
[2] Brain and heart data from CCM experts (composers and performers) successfully modelled CCM and non-CCM. But non-experts had inconsistent responses to CCM and poor modelling results. Only non-CCM could be modelled from the brain and heart data from non-experts.
[3] Tapping was found to neither improve nor impair temporal processing for CCM-like sound examples. Tapping to tonal sound examples did however impair temporal processing; tapping seemed to facilitate the formation of a groove which diminished temporal error detection.
[4] Finally, interactive tapping with real-time control tended to improve the tappers liking ratings given to CCM compared to non-interactive tapping. Nevertheless, non-CCM remained preferred overall.
Dissemination
The composed music, interface, and experimental results described above were disseminated in a number of formats and venues. The Fellow organized one public concert, performed in 5 additional public concerts, gave 1 public research demo, 2 internal research presentations, 3 external research presentations, 6 international research presentations or demos, ran 1 workshop, and gave an invited talk about DEI in music technology.
