Final Report Summary - BLACKBOX (A collaborative platform to document performance composition: from conceptual structures in the backstage to customizable visualizations in the front-end)
The BlackBox project has been above all about trying to make visible what is not visible at first sight: The metaphor behind its title appeals to the mysterious human brain, the unknown, and to the rehearsal studios of contemporary choreographers and performers. Its general framework being the complexity of the human mind while engaged in the cultural production of intangible cultural heritage, my initial motivation was to unveil the tacit embodied knowledge contained in performing arts practices in order to analyze and document some of the most complex traces of human creativity and decision-making happening behind the curtains of staged performances.
The team was thoroughly interdisciplinary, involving invited artists, neuroscientists, cognitive linguists, computer scientists, designers and video experts. We have proposed innovative advances in theory (connecting debates on the relations between non-verbal language, the neurosciences and performance studies), methodologies (i. a. Performance as Research) and applications (using technologies from computer vision and data visualization to document choreographic processes).
While the ultimate scientific aim of BlackBox was to contribute to the understanding of the underlying principles behind performance composition, it implied, in very concrete terms, a bright response to the following challenge: How can we design cutting-edge models for the documentation of performance composition methods, which help “translate” the artists’ more abstract conceptual structures into concrete front-end visualizations to be displayed in our web-based platform?
Several rehearsal sessions of the three invited choreographers, composed of hundreds of hours of filming, have been taken as case-studies to be documented, annotated and published online at: http://blackbox.fcsh.unl.pt/home.html
In terms of specific research methodologies, we are proud to be one of the still very few labs in Europe where the recent area “Performance as Research” has been openly adopted and set into practice: we have not only mixed the complementary methods of the arts with those of the so-called objective sciences, but we have also undergone uncountable hours of field work using close to ethnographical methodologies, but rather based on the embodied experience of the artists and dancers, with researchers working closely with them.
We have published 37 papers and book chapters in peer-reviewed and indexed publications, and we seriously believe in the high potential of exploring hybrid new ways of studying staged performances as a catalyzer for both the investigation and documentation purposes of the most intangible art forms such as contemporary dance and performance.
Our novel digital tools for performance documentation, analysis and transmission of research outcomes were mainly conceived for the public interface of the BlackBox collaborative platform, both for academics and for the outreach events. From there, four short videos with animated infographic visualizations can be accessed, as well as three long film documentaries on Fiadeiro’s, Graça´s and Rijmer’s work; a 360º multiple viewpoint video for the National Ballet of Portugal can be interactively explored, either on computers or tablets, preferably; a 3D immersive Virtual-Reality installation-gallery (to be experienced with the Vive headset) for the work of Sylvia Rijmer is available since September 2019, and the collaborative knowledge-base (https://tkb.fcsh.unl.pt/content/knowledge-base-performing-arts) is also accessible from the BlackBox’s platform, together with the new version of the video annotation software, with a web-based Beta version (MotionNotes) now available upon demand.
Supporting the cognitive linguistics view that language (both verbal and non-verbal) is inseparable from the artifacts, institutions, cultural practices and behaviour used by humans who undertake complex tasks, we have adapted annotation techniques from multimodal communication for the analysis of the simultaneity between gesture qualities, full body movements and speech in order to arrive at the quali-quantitative results reported in the papers we have published.
In general, we have placed the focus of this project on the subtle power of the human bodies as sustainers of social coordination, believing that our creative realities can be extended by the artistic resources of a preferably shared collective world.
The team was thoroughly interdisciplinary, involving invited artists, neuroscientists, cognitive linguists, computer scientists, designers and video experts. We have proposed innovative advances in theory (connecting debates on the relations between non-verbal language, the neurosciences and performance studies), methodologies (i. a. Performance as Research) and applications (using technologies from computer vision and data visualization to document choreographic processes).
While the ultimate scientific aim of BlackBox was to contribute to the understanding of the underlying principles behind performance composition, it implied, in very concrete terms, a bright response to the following challenge: How can we design cutting-edge models for the documentation of performance composition methods, which help “translate” the artists’ more abstract conceptual structures into concrete front-end visualizations to be displayed in our web-based platform?
Several rehearsal sessions of the three invited choreographers, composed of hundreds of hours of filming, have been taken as case-studies to be documented, annotated and published online at: http://blackbox.fcsh.unl.pt/home.html
In terms of specific research methodologies, we are proud to be one of the still very few labs in Europe where the recent area “Performance as Research” has been openly adopted and set into practice: we have not only mixed the complementary methods of the arts with those of the so-called objective sciences, but we have also undergone uncountable hours of field work using close to ethnographical methodologies, but rather based on the embodied experience of the artists and dancers, with researchers working closely with them.
We have published 37 papers and book chapters in peer-reviewed and indexed publications, and we seriously believe in the high potential of exploring hybrid new ways of studying staged performances as a catalyzer for both the investigation and documentation purposes of the most intangible art forms such as contemporary dance and performance.
Our novel digital tools for performance documentation, analysis and transmission of research outcomes were mainly conceived for the public interface of the BlackBox collaborative platform, both for academics and for the outreach events. From there, four short videos with animated infographic visualizations can be accessed, as well as three long film documentaries on Fiadeiro’s, Graça´s and Rijmer’s work; a 360º multiple viewpoint video for the National Ballet of Portugal can be interactively explored, either on computers or tablets, preferably; a 3D immersive Virtual-Reality installation-gallery (to be experienced with the Vive headset) for the work of Sylvia Rijmer is available since September 2019, and the collaborative knowledge-base (https://tkb.fcsh.unl.pt/content/knowledge-base-performing-arts) is also accessible from the BlackBox’s platform, together with the new version of the video annotation software, with a web-based Beta version (MotionNotes) now available upon demand.
Supporting the cognitive linguistics view that language (both verbal and non-verbal) is inseparable from the artifacts, institutions, cultural practices and behaviour used by humans who undertake complex tasks, we have adapted annotation techniques from multimodal communication for the analysis of the simultaneity between gesture qualities, full body movements and speech in order to arrive at the quali-quantitative results reported in the papers we have published.
In general, we have placed the focus of this project on the subtle power of the human bodies as sustainers of social coordination, believing that our creative realities can be extended by the artistic resources of a preferably shared collective world.