Periodic Reporting for period 1 - MENHIR (Mental health monitoring through interactive conversations)
Periodo di rendicontazione: 2019-02-01 al 2021-01-31
MENHIR aims at engaging people in the management of self-care and providing a more flexible, immediate and sustained complement to under pressure mental health services. MENHIR chatbot technology provides symptom and mood management, personalised support and motivation, coping strategies, signposting to online resources and complementing the support received by local services. The technology developed will be able to process emotional and mood patterns from different aspects of the voice recorded. This information facilitates rapid intervention and appropriate feedback such as prompts and supportive messages. It will also generate personalized conversations with the users to motivate and engage them at any time and especially in between visits to their local services.
The limitations of previous technology are the high dropout rates due in part to a lack of interactive features and appropriate prompts, difficulties for people with low levels of technological literacy and failure to provide human support. The MENHIR chatbot technology aims to address these difficulties being more usable and intuitive since it simulates everyday human to human conversation. The project will also generate data on how people with anxiety and depression interact with and respond to this technology, and about the benefits it brings to them.
The objectives of MENHIR are to:
- Develop new capabilities and skills for MENHIR participants. As a RISE project, our aim is the sharing of knowledge and ideas across countries, sectors and disciplines. MENHIR will create new opportunities for ICT experts to root technology in a deeper understanding of their prospective users, while psychologists and mental health experts will gain knowledge of the technological tools that can help them improve their clients’ wellbeing.
- Establish the strengths, limitations and requirements of mental health chatbots.
- Develop the MENHIR chatbot technology collaborating with people who suffer anxiety and/or mild depression.
- Understand how users engage with chatbot technology and how it may affect their wellbeing over time.
The limitations of previous technology are the high dropout rates due in part to a lack of interactive features and appropriate prompts, difficulties for people with low levels of technological literacy and failure to provide human support. The MENHIR chatbot technology aims to address these difficulties being more usable and intuitive since it simulates everyday human to human conversation. The project will also generate data on how people with anxiety and depression interact with and respond to this technology, and about the benefits it brings to them.
The objectives of MENHIR are to:
- Develop new capabilities and skills for MENHIR participants. As a RISE project, our aim is the sharing of knowledge and ideas across countries, sectors and disciplines. MENHIR will create new opportunities for ICT experts to root technology in a deeper understanding of their prospective users, while psychologists and mental health experts will gain knowledge of the technological tools that can help them improve their clients’ wellbeing.
- Establish the strengths, limitations and requirements of mental health chatbots.
- Develop the MENHIR chatbot technology collaborating with people who suffer anxiety and/or mild depression.
- Understand how users engage with chatbot technology and how it may affect their wellbeing over time.
In the first reporting period the work plan has advanced as scheduled with all the planned tasks, deliverables and the first milestone completed. We have identified the different procedures and activities to organise the consortium, and generated several internal documents to coordinate our efforts, including a project management plan, data management plan, risk management plan, data recording protocol, data annotation procedure, and ethical procedures manual. We have also defined a plan to disseminate and communicate the project to different key audiences. Our website (https://menhir-project.eu) was published early in the project, and is updated regularly to share our mission, information about the partners, testimonials of secondments, scientific publications, videos, training materials, interesting links and reads, news and events. We are also active in twitter: @MenhirH2020.
MENHIR participants have performed secondments that have helped to know each other better and exchange expertise. We have paid special attention to staff development through secondments with a clear protocol with steps before, during and after each secondment. A co-creation methodology has been adopted so that prospective users participate in the design and development of technology. They are represented in MENHIR by clients of Action Mental Health (AMH). A co-creation workshop with people suffering from anxiety and depression was performed on June 19. The results of the co-creation together with a detailed analysis of the scientific literature about mental health monitoring systems and multimodal emotion and mood recognition, allowed to define the usage scenarios and chatbot strategy for mental health monitoring.
During this reporting period, we have devoted great effort to data collection. Human-to-human conversations with AMH clients and a control group were recorded from October to December 2019. We have worked on a pilot infrastructure for storage, analysis and annotation of data, where the anonymized recordings have been shared between partners. In addition, we have conducted the annotation of the recordings at several levels including time alignment, turn taking, automatic and manual transcription and filled and empty pauses. All these pieces of information are currently being used to perform cross-modal analysis of the recordings and learn the most effective dialogue management behaviours for the scenarios envisioned. We have created a structured data repository for mental health coping strategies and user models based on action plans to be the basis for personalized interaction with the system.
Since the start of the project we have paid special attention to the dissemination and communication of our results to different audiences. We have made scientific publications in green and gold open access, organized conferences and workshops, participated in scientific events, communicated MENHIR through media (press and radio), and taken part in general public scientific communication events (e.g. researchers night and science fairs). We have also established synergies with other H2020 projects with common interests and organized shared events.
In October 2020 we held the MENHIR International Doctoral Summer School, with lecturers of different backgrounds including psychology, psychiatry, computer science, telecommunications, and social work, coming from academic, industry and non-profit sectors. We had 21 participants from 15 affiliations with very positive results. The materials (slides and lectures) are shared in our website along with other videos and resources.
MENHIR participants have performed secondments that have helped to know each other better and exchange expertise. We have paid special attention to staff development through secondments with a clear protocol with steps before, during and after each secondment. A co-creation methodology has been adopted so that prospective users participate in the design and development of technology. They are represented in MENHIR by clients of Action Mental Health (AMH). A co-creation workshop with people suffering from anxiety and depression was performed on June 19. The results of the co-creation together with a detailed analysis of the scientific literature about mental health monitoring systems and multimodal emotion and mood recognition, allowed to define the usage scenarios and chatbot strategy for mental health monitoring.
During this reporting period, we have devoted great effort to data collection. Human-to-human conversations with AMH clients and a control group were recorded from October to December 2019. We have worked on a pilot infrastructure for storage, analysis and annotation of data, where the anonymized recordings have been shared between partners. In addition, we have conducted the annotation of the recordings at several levels including time alignment, turn taking, automatic and manual transcription and filled and empty pauses. All these pieces of information are currently being used to perform cross-modal analysis of the recordings and learn the most effective dialogue management behaviours for the scenarios envisioned. We have created a structured data repository for mental health coping strategies and user models based on action plans to be the basis for personalized interaction with the system.
Since the start of the project we have paid special attention to the dissemination and communication of our results to different audiences. We have made scientific publications in green and gold open access, organized conferences and workshops, participated in scientific events, communicated MENHIR through media (press and radio), and taken part in general public scientific communication events (e.g. researchers night and science fairs). We have also established synergies with other H2020 projects with common interests and organized shared events.
In October 2020 we held the MENHIR International Doctoral Summer School, with lecturers of different backgrounds including psychology, psychiatry, computer science, telecommunications, and social work, coming from academic, industry and non-profit sectors. We had 21 participants from 15 affiliations with very positive results. The materials (slides and lectures) are shared in our website along with other videos and resources.
MENHIR will progress beyond the state of the art by:
(1) Determining the most practical scales for monitoring mental wellbeing.
(2) Compiling mental health coping strategies, prompts, relevant digital content and resources.
(3) Providing new evidence on how to perform a cross-modal analysis of the interactional exchanges to recognize the users’ emotional state and anxiety level.
(4) Defining how to track mood and anxiety from user-system interactions and how to represent user progress in a computational model.
(5) Providing novel approaches to generate computerized models of the users to adapt decision making, conversational models and communication styles.
(6) Finding novel approaches to make chatbot interaction personalized to their users, considering they are heterogeneous in terms of age, gender, mental health condition, progression and responses to system’s strategies.
(7) Identifying adequate dialogue management strategies based on user progress and the history of the user-system interaction.
(8) Establishing adequate proactivity approaches and disparate conversation structures to manage conversation and favour user self-disclosure.
(9) Establishing new methods to favour user-system trust through active listening.
(10) Generating good practices for: a) data storage, annotation and sharing, b) participatory research based on co-creation.
MENHIR will help to achieve the vision that should be accomplished by 2025 according to the Strategic Roadmap for the area of multimodal conversational interaction technologies (developed by CITIA in 2015). In terms of the impact on MENHIR participants, the project will provide researchers with a unique experience that will improve their skills, enhancing their career prospects.
(1) Determining the most practical scales for monitoring mental wellbeing.
(2) Compiling mental health coping strategies, prompts, relevant digital content and resources.
(3) Providing new evidence on how to perform a cross-modal analysis of the interactional exchanges to recognize the users’ emotional state and anxiety level.
(4) Defining how to track mood and anxiety from user-system interactions and how to represent user progress in a computational model.
(5) Providing novel approaches to generate computerized models of the users to adapt decision making, conversational models and communication styles.
(6) Finding novel approaches to make chatbot interaction personalized to their users, considering they are heterogeneous in terms of age, gender, mental health condition, progression and responses to system’s strategies.
(7) Identifying adequate dialogue management strategies based on user progress and the history of the user-system interaction.
(8) Establishing adequate proactivity approaches and disparate conversation structures to manage conversation and favour user self-disclosure.
(9) Establishing new methods to favour user-system trust through active listening.
(10) Generating good practices for: a) data storage, annotation and sharing, b) participatory research based on co-creation.
MENHIR will help to achieve the vision that should be accomplished by 2025 according to the Strategic Roadmap for the area of multimodal conversational interaction technologies (developed by CITIA in 2015). In terms of the impact on MENHIR participants, the project will provide researchers with a unique experience that will improve their skills, enhancing their career prospects.