Periodic Reporting for period 2 - MENHIR (Mental health monitoring through interactive conversations)
Periodo di rendicontazione: 2021-02-01 al 2024-01-31
MENHIR aims at engaging people in the management of self-care and providing a more flexible, immediate and sustained complement to under pressure mental health services. MENHIR chatbot technology provides symptom and mood management, personalised support and motivation, coping strategies, signposting to online resources and complementing the support received by local services. The technology developed will be able to process emotional and mood patterns from different aspects of the voice recorded. This information facilitates rapid intervention and appropriate feedback such as prompts and supportive messages. It will also generate personalised conversations with the users to motivate and engage them at any time and especially in between visits to their local services.
The limitations of previous technology are the high dropout rates due in part to a lack of interactive features and appropriate prompts, difficulties for people with low levels of technological literacy and failure to provide human support. The MENHIR chatbot technology aims to address these difficulties being more usable and intuitive since it simulates everyday human to human conversation. The project also generates data on how people with anxiety and depression interact with and respond to this technology, and about the benefits it brings to them.
The objectives of MENHIR are to:
- Develop new capabilities and skills for MENHIR participants. As a RISE project, our aim is the sharing of knowledge and ideas across countries, sectors and disciplines. MENHIR is creating new opportunities for ICT experts to root technology in a deeper understanding of their prospective users, while psychologists and mental health experts will gain knowledge of the technological tools that can help them improve their clients’ wellbeing.
- Establish the strengths, limitations and requirements of mental health chatbots.
- Develop the MENHIR chatbot technology collaborating with people who suffer anxiety and/or mild depression.
- Understand how users engage with chatbot technology and how it may affect their wellbeing over time.
The limitations of previous technology are the high dropout rates due in part to a lack of interactive features and appropriate prompts, difficulties for people with low levels of technological literacy and failure to provide human support. The MENHIR chatbot technology aims to address these difficulties being more usable and intuitive since it simulates everyday human to human conversation. The project also generates data on how people with anxiety and depression interact with and respond to this technology, and about the benefits it brings to them.
The objectives of MENHIR are to:
- Develop new capabilities and skills for MENHIR participants. As a RISE project, our aim is the sharing of knowledge and ideas across countries, sectors and disciplines. MENHIR is creating new opportunities for ICT experts to root technology in a deeper understanding of their prospective users, while psychologists and mental health experts will gain knowledge of the technological tools that can help them improve their clients’ wellbeing.
- Establish the strengths, limitations and requirements of mental health chatbots.
- Develop the MENHIR chatbot technology collaborating with people who suffer anxiety and/or mild depression.
- Understand how users engage with chatbot technology and how it may affect their wellbeing over time.
The work plan has advanced as scheduled with all the planned tasks, deliverables and two milestones completed. We have identified the different procedures and activities to organise the consortium, and generated several internal documents to coordinate our efforts, including a project management plan, data management plan, risk management plan, data recording protocol, data annotation procedure, and ethical procedures manual. We have also defined a plan to disseminate and communicate the project to different key audiences. Our website (https://menhir-project.eu(si apre in una nuova finestra)) was published early in the project, and is updated regularly to share our mission, information about the partners, testimonials of secondments, scientific publications, videos, training materials, interesting links and reads, news and events.
MENHIR participants have established a solid network of collaboration. We have paid special attention to staff development through secondments with a clear protocol with steps before, during and after each secondment. ESR development has been key, with 3 doctoral theses completed and several more to be finalised before the end of the project. A co-creation methodology has been adopted so that prospective users participate in the design and development of technology. They are represented in MENHIR by clients of Action Mental Health (AMH). A co-creation early in the project. The results of the co-creation together with a detailed analysis of the scientific literature about mental health monitoring systems and multimodal emotion and mood recognition, allowed to define the usage scenarios and chatbot strategy for mental health monitoring.
During this period, we have devoted great effort to data collection. Human-to-human conversations with AMH clients and a control group were recorded, we developed a pilot infrastructure for storage, analysis and annotation of data, where the anonymized recordings have been shared between partners. In addition, we have conducted the annotation of the recordings at several levels including time alignment, turn taking, automatic and manual transcription and filled and empty pauses. All these pieces of information are currently being used to perform cross-modal analysis of the recordings and learn the most effective dialogue management behaviours for the scenarios envisioned. We have created a structured data repository for mental health coping strategies and user models based on action plans to be the basis for personalised interaction with the system. We have also advanced in the development of conversational technology in the different use cases envisioned, specially for the diary and smart reminder scenarios.
Since the start of the project we have fostered the dissemination and communication of our results to different audiences. We have made more than 100 scientific publications in green and gold open access, organised conferences and workshops, participated in scientific events, communicated MENHIR through media (press and radio), and taken part in general public scientific communication events (e.g. researchers’ night and science fairs). We have also established synergies with other H2020 projects with common interests and organised shared events.
In 2020 and 2022 we held two MENHIR Int. Doctoral Summer Schools, involving academic, industry and non-profit sectors. The materials are shared on our website along with other videos and resources. In 2023, we organized the 1st Int. Digital Mental Health and Wellbeing Conference.
MENHIR participants have established a solid network of collaboration. We have paid special attention to staff development through secondments with a clear protocol with steps before, during and after each secondment. ESR development has been key, with 3 doctoral theses completed and several more to be finalised before the end of the project. A co-creation methodology has been adopted so that prospective users participate in the design and development of technology. They are represented in MENHIR by clients of Action Mental Health (AMH). A co-creation early in the project. The results of the co-creation together with a detailed analysis of the scientific literature about mental health monitoring systems and multimodal emotion and mood recognition, allowed to define the usage scenarios and chatbot strategy for mental health monitoring.
During this period, we have devoted great effort to data collection. Human-to-human conversations with AMH clients and a control group were recorded, we developed a pilot infrastructure for storage, analysis and annotation of data, where the anonymized recordings have been shared between partners. In addition, we have conducted the annotation of the recordings at several levels including time alignment, turn taking, automatic and manual transcription and filled and empty pauses. All these pieces of information are currently being used to perform cross-modal analysis of the recordings and learn the most effective dialogue management behaviours for the scenarios envisioned. We have created a structured data repository for mental health coping strategies and user models based on action plans to be the basis for personalised interaction with the system. We have also advanced in the development of conversational technology in the different use cases envisioned, specially for the diary and smart reminder scenarios.
Since the start of the project we have fostered the dissemination and communication of our results to different audiences. We have made more than 100 scientific publications in green and gold open access, organised conferences and workshops, participated in scientific events, communicated MENHIR through media (press and radio), and taken part in general public scientific communication events (e.g. researchers’ night and science fairs). We have also established synergies with other H2020 projects with common interests and organised shared events.
In 2020 and 2022 we held two MENHIR Int. Doctoral Summer Schools, involving academic, industry and non-profit sectors. The materials are shared on our website along with other videos and resources. In 2023, we organized the 1st Int. Digital Mental Health and Wellbeing Conference.
MENHIR will progress beyond the state of the art by:
(1) Determining the most practical scales for monitoring mental wellbeing.
(2) Compiling mental health coping strategies, prompts, relevant digital content and resources.
(3) Providing new evidence on how to perform a cross-modal analysis of the interactional exchanges to recognize the users’ emotional state and anxiety level.
(4) Defining how to track mood and anxiety from user-system interactions and how to represent user progress in a computational model.
(5) Providing novel approaches to generate computerised models of the users to adapt decision making, conversational models and communication styles.
(6) Finding novel approaches to make chatbot interaction personalised to their users, considering they are heterogeneous in terms of age, gender, mental health condition, progression and responses to system’s strategies.
(7) Identifying adequate dialogue management strategies based on user progress and the history of the user-system interaction.
(8) Develop new models to generate an adequate language for the system when interacting with the user.
(9) Establishing adequate proactivity approaches and disparate conversation structures to manage conversation and favour user self-disclosure.
(10) Establishing new methods to favour user-system trust through active listening.
(11) Generating good practices for: a) data storage, annotation and sharing, b) participatory research based on co-creation.
MENHIR will help to achieve the vision that should be accomplished by 2025 according to the Strategic Roadmap for the area of multimodal conversational interaction technologies. In terms of the impact on MENHIR participants, the project will provide researchers with a unique experience that will improve their skills, enhancing their career prospects.
(1) Determining the most practical scales for monitoring mental wellbeing.
(2) Compiling mental health coping strategies, prompts, relevant digital content and resources.
(3) Providing new evidence on how to perform a cross-modal analysis of the interactional exchanges to recognize the users’ emotional state and anxiety level.
(4) Defining how to track mood and anxiety from user-system interactions and how to represent user progress in a computational model.
(5) Providing novel approaches to generate computerised models of the users to adapt decision making, conversational models and communication styles.
(6) Finding novel approaches to make chatbot interaction personalised to their users, considering they are heterogeneous in terms of age, gender, mental health condition, progression and responses to system’s strategies.
(7) Identifying adequate dialogue management strategies based on user progress and the history of the user-system interaction.
(8) Develop new models to generate an adequate language for the system when interacting with the user.
(9) Establishing adequate proactivity approaches and disparate conversation structures to manage conversation and favour user self-disclosure.
(10) Establishing new methods to favour user-system trust through active listening.
(11) Generating good practices for: a) data storage, annotation and sharing, b) participatory research based on co-creation.
MENHIR will help to achieve the vision that should be accomplished by 2025 according to the Strategic Roadmap for the area of multimodal conversational interaction technologies. In terms of the impact on MENHIR participants, the project will provide researchers with a unique experience that will improve their skills, enhancing their career prospects.