Periodic Reporting for period 1 - DATASOUND (DATASOUND: Understanding data with sound)
Okres sprawozdawczy: 2017-06-01 do 2019-05-31
"The DATASOUND action started with the question ""Can sound be used for Data Science?"" and aimed at throwing light into how sound could be used in the context of modern data science, whether alone or as a complement to the ubiquitous visualisations. Despite humans having five senses available, we have limited ourselves to only use visualisation for exploration and communication of data in scientific contexts.
In this particular context of Data Science, our departing hypothesis was that an enhanced data understanding process (by means of employing additional senses) could lead to faster and more accurate findings and a more streamlined and natural communication of insights. As our society relies more and more on data, this becomes a crucial aspect prone to be optimized.
The application domain chosen for the project (one domain in which sound can help to better understand the underlying data) was that of continuous streams of data. And in particular, in monitoring applications, such as health or energy consumption. In those domains, attention is continuously required and a summary of the observations after a period is not very relevant. Because of this, we put forward to work with continuous time-series data, and particularly with energy consumption data.
The action has therefore seen work in three main lines of research:
a) Studying the use of sound to communicate information, including the cognitive aspects involved. In other words, can sound encode information that everyone can decode?
b) Developing tools to support alternative ways of presenting knowledge and data (including audiovisual content, but also large-scale visualisation).
c) Propose new ways of computationally represent energy consumption data, with the aim of better understanding and spotting the underlying patterns.
High-impact results have been generated for these three lines; not only by the scientific publications, but also by the participation in several outreach events with the larger society, and the development of an open-source software tool.
The action has also been instrumental for the fellow to extend his network of research collaborations, to lead a research group, and to ultimately secure another position in academia.
"
In this particular context of Data Science, our departing hypothesis was that an enhanced data understanding process (by means of employing additional senses) could lead to faster and more accurate findings and a more streamlined and natural communication of insights. As our society relies more and more on data, this becomes a crucial aspect prone to be optimized.
The application domain chosen for the project (one domain in which sound can help to better understand the underlying data) was that of continuous streams of data. And in particular, in monitoring applications, such as health or energy consumption. In those domains, attention is continuously required and a summary of the observations after a period is not very relevant. Because of this, we put forward to work with continuous time-series data, and particularly with energy consumption data.
The action has therefore seen work in three main lines of research:
a) Studying the use of sound to communicate information, including the cognitive aspects involved. In other words, can sound encode information that everyone can decode?
b) Developing tools to support alternative ways of presenting knowledge and data (including audiovisual content, but also large-scale visualisation).
c) Propose new ways of computationally represent energy consumption data, with the aim of better understanding and spotting the underlying patterns.
High-impact results have been generated for these three lines; not only by the scientific publications, but also by the participation in several outreach events with the larger society, and the development of an open-source software tool.
The action has also been instrumental for the fellow to extend his network of research collaborations, to lead a research group, and to ultimately secure another position in academia.
"
The research work aimed to cover 3 objectives:
- Use sounds/music to communicate scientific knowledge.
- Develop tools to effectively explore and communicate data and knowledge, going further than the traditional visualisation.
- Encode energy consumption data in relevant ways to achieve the previous two objectives.
We found early in the project that there is no existing ground truth on how sound can convey meaning, and so we decided to work at two levels: 1) understanding the cognitive aspects that exist in music communication between individuals, and 2) how data can be displayed in alternative means to traditional desktop screens. The aim was for those two strands to convey eventually.
One of the outcomes of this action has been the creation of a Human-Data interaction group at the Data Science Institute, Imperial College London, led by the fellow. This group has mainly worked in the development of a software tool to enable the flexible visualisation of data in large-visualisation displays, advancing the state-of-the-art in this regard, and offering an open-source tool to any interested party.
During the action, 4 papers have been published in relevant journals, and 4 more are currently under review. The fellow has presented his work at 3 international scientific conferences, and he has participated in up to 5 different outreach activities with very diverse audiences.
- Use sounds/music to communicate scientific knowledge.
- Develop tools to effectively explore and communicate data and knowledge, going further than the traditional visualisation.
- Encode energy consumption data in relevant ways to achieve the previous two objectives.
We found early in the project that there is no existing ground truth on how sound can convey meaning, and so we decided to work at two levels: 1) understanding the cognitive aspects that exist in music communication between individuals, and 2) how data can be displayed in alternative means to traditional desktop screens. The aim was for those two strands to convey eventually.
One of the outcomes of this action has been the creation of a Human-Data interaction group at the Data Science Institute, Imperial College London, led by the fellow. This group has mainly worked in the development of a software tool to enable the flexible visualisation of data in large-visualisation displays, advancing the state-of-the-art in this regard, and offering an open-source tool to any interested party.
During the action, 4 papers have been published in relevant journals, and 4 more are currently under review. The fellow has presented his work at 3 international scientific conferences, and he has participated in up to 5 different outreach activities with very diverse audiences.
The work carried out during the action has directly produced 4 research publications on data understanding and communication at large-scale displaying facilities. The fellow has also presented his work in several international conferences and organized a Sonification and Conversational AI workshop at the 2018 edition of the KDD conference.
As part of the project, we also performed a pilot study on how individuals perceive live music, and whether or not a connection occurs between musicians and member of the audience.
It was also under the supervision of the fellow that members of his research group started the development of OVE, Imperial's open-source large-scale visualisation framework for distributed facilities.
The fellow has participated in several outreach activities explaining how sound could be used in the context of data science. Among the most relevant are the 2018 MSCA-FallingWalls event in Brussels (coinciding with the Researchers' Night), and the Chasmata concert at the Guggenheim Bilbao Anniversary.
The action has also strengthened the fellow's network of collaborators, setting the grounds for future work collaborations which will take the form of multidisciplinary collaborative projects.
As part of the project, we also performed a pilot study on how individuals perceive live music, and whether or not a connection occurs between musicians and member of the audience.
It was also under the supervision of the fellow that members of his research group started the development of OVE, Imperial's open-source large-scale visualisation framework for distributed facilities.
The fellow has participated in several outreach activities explaining how sound could be used in the context of data science. Among the most relevant are the 2018 MSCA-FallingWalls event in Brussels (coinciding with the Researchers' Night), and the Chasmata concert at the Guggenheim Bilbao Anniversary.
The action has also strengthened the fellow's network of collaborators, setting the grounds for future work collaborations which will take the form of multidisciplinary collaborative projects.