Periodic Reporting for period 2 - ESSENCE (ESSENCE: Empathic platform to personally monitor, Stimulate, enrich, and aSsist Elders aNd Children in their Environment)
Okres sprawozdawczy: 2021-11-01 do 2023-04-30
ESSENCE aims at transforming the lessons learnt from the COVID-19 in a huge opportunity that exploits technology toward a deep evolution of the services targeting vulnerable populations: non- or pre-frail seniors, and children of the first years of primary school. To contribute to the public health response in the context of the COVID-19 epidemic, ESSENCE aims at boosting the creation of a new model of home-based care that relies on stimulation, remote monitoring, tele-assistance, and connection between users, families, and professionals. ESSENCE adopts an iterative optimization process between two pillars 1) technology transfer of a subset of successful modules and 2) feedback received from testing on target users – with the final aim of accelerating the route towards CE mark.
To this end, ESSENCE is a modular platform including:
- The Community-Based Activity Center. It allows access to virtual rooms for communication with other users, single and multi-player games, as well as the possibility to contact professionals.
- The Monitoring Module aimed at supervising the users’ condition in the physical, cognitive and emotional domains. This module collects data coming from activities performed in the CBAC, handwriting performances executed through a smart ink pen and, for seniors only, voice analysis during phone calls. Such information is exploited by algorithms based on artificial intelligence, which are in charge of alerting the professional if unexpected behaviours are detected, so to provide a prompt intervention plan.
- The ESSENCE manager, which represents the control unit of the platform. It is in charge of authentication, user profiling, system configuration, notifications and alerts delivery and management of malfunctioning.
- The Light Health Monitoring Module (LHM), the main interface for professional, used to have an overview of all users, plan specific activities and monitor the adherence. At the point of need it may also schedule a teleconsultation session with integrated clinical tests that can be performed and evaluated.
ESSENCE was field tested for 1 year in the 2 ecosystems (66 children in Italy and 68 seniors in Spain) with the main common objective to evaluate the usability and satisfaction after a prolonged use. The testing allowed us to evaluate the appreciation of the system, to collect data for training the AI monitoring algorithms and to evaluate the support that ESSENCE provides in the early diagnosis of decline in seniors and in the early detection of learning difficulties in children.
Given the promising results of both field testing, the consortium tried to accelerate the route to market. A regulatory strategy was identified, risk analysis and verification testing were carried out. Thanks to these activities, the system (especially the smart ink pen) is close to achieving the CE mark.
To this end, ESSENCE is a modular platform including:
- The Community-Based Activity Center. It allows access to virtual rooms for communication with other users, single and multi-player games, as well as the possibility to contact professionals.
- The Monitoring Module aimed at supervising the users’ condition in the physical, cognitive and emotional domains. This module collects data coming from activities performed in the CBAC, handwriting performances executed through a smart ink pen and, for seniors only, voice analysis during phone calls. Such information is exploited by algorithms based on artificial intelligence, which are in charge of alerting the professional if unexpected behaviours are detected, so to provide a prompt intervention plan.
- The ESSENCE manager, which represents the control unit of the platform. It is in charge of authentication, user profiling, system configuration, notifications and alerts delivery and management of malfunctioning.
- The Light Health Monitoring Module (LHM), the main interface for professional, used to have an overview of all users, plan specific activities and monitor the adherence. At the point of need it may also schedule a teleconsultation session with integrated clinical tests that can be performed and evaluated.
ESSENCE was field tested for 1 year in the 2 ecosystems (66 children in Italy and 68 seniors in Spain) with the main common objective to evaluate the usability and satisfaction after a prolonged use. The testing allowed us to evaluate the appreciation of the system, to collect data for training the AI monitoring algorithms and to evaluate the support that ESSENCE provides in the early diagnosis of decline in seniors and in the early detection of learning difficulties in children.
Given the promising results of both field testing, the consortium tried to accelerate the route to market. A regulatory strategy was identified, risk analysis and verification testing were carried out. Thanks to these activities, the system (especially the smart ink pen) is close to achieving the CE mark.
The activities of the projects were completely aligned with the Description of Action and all the objectives were reached. Indeed, ESSENCE was fully developed as a modular system specifically co-designed and refined with relevant stakeholders in the two target ecosystems. On seniors the aim was to early detect signs of decline and on children was to support the process of early screening of learning disabilities at school.
The whole platform was deployed and longitudinally tested separately on seniors in Spain and children in Italy.
In the children ecosystem, 66 children attending the first and second grade of the primary school were recruited together with their teachers. Different activities were provided, such as, serious games for monitoring and training of learning problems, social inclusion, and to achieve a remote clinical evaluation at the point of need. Usability was high (median system usability score of 82), as well as satisfaction (92% of users were satisfied), and compliance was in line with expectations. It has been highlighted how the effect of personalization and gamification has influenced the compliance of children, which has significantly increased, together with the accuracy of their performance. AI algorithms trained on serious games for handwriting screening were able to detect problems with an area under the precision-recall curve of 0.84.
The smart ink pen was adopted to collect data from more than 500 children of the primary school in order to have normative data on graphomotor fluency skills, associated with the parameters measured using the sensorized pen, which will be able to provide significant added value to research and clinical practice.
In the senior ecosystem, the usage of the system was not only consistent and varied, but also fairly intense and way above the expectations envisaged in the DoA. In fact, the desired weekly usage per person reached more than 2 hours during the whole field testing (target was 30 minutes per week). For seniors, usability was at the threshold of acceptability but the 91% of them will continue using the system after the end of the project. Data collected from the smart ink pen and a voice app were leveraged to build AI algorithms, that reached good accuracy in predicting cognitive decline of 0.93 and 0.82 respectively.
The appreciation for the system was evident from the desire that target users, in both ecosystems, expressed to keep using ESSENCE after the end of the project. This overall appreciation is the most important result towards the exploitation. Indeed, the consortium identified 1 exploitable result targeted on children that is the whole platform to be deployed at school to better screen learning disorders and connect schools, families and child neuropsychiatrists at the point of need. This idea is completely aligned with the Italian law and the new international guidelines in the field.
In addition, 3 exploitable results were identified on seniors independently living at home: the monitoring of handwriting, the monitoring of voice analysis, the stimulating virtual rooms of the CBAC integrated with a telehealth platform to plan and monitor intervention and eventually schedule and carry out teleconsultation at the point of need.
For each of the exploitable results, the economic feasibility and the social impact were computed showing encouraging results for the future exploitation.
To conclude ESSENCE accelerated the route to CE mark. All the tests carried out had positive results indicating that the engineered prototype is quite ready for the CE mark procedure.
The whole platform was deployed and longitudinally tested separately on seniors in Spain and children in Italy.
In the children ecosystem, 66 children attending the first and second grade of the primary school were recruited together with their teachers. Different activities were provided, such as, serious games for monitoring and training of learning problems, social inclusion, and to achieve a remote clinical evaluation at the point of need. Usability was high (median system usability score of 82), as well as satisfaction (92% of users were satisfied), and compliance was in line with expectations. It has been highlighted how the effect of personalization and gamification has influenced the compliance of children, which has significantly increased, together with the accuracy of their performance. AI algorithms trained on serious games for handwriting screening were able to detect problems with an area under the precision-recall curve of 0.84.
The smart ink pen was adopted to collect data from more than 500 children of the primary school in order to have normative data on graphomotor fluency skills, associated with the parameters measured using the sensorized pen, which will be able to provide significant added value to research and clinical practice.
In the senior ecosystem, the usage of the system was not only consistent and varied, but also fairly intense and way above the expectations envisaged in the DoA. In fact, the desired weekly usage per person reached more than 2 hours during the whole field testing (target was 30 minutes per week). For seniors, usability was at the threshold of acceptability but the 91% of them will continue using the system after the end of the project. Data collected from the smart ink pen and a voice app were leveraged to build AI algorithms, that reached good accuracy in predicting cognitive decline of 0.93 and 0.82 respectively.
The appreciation for the system was evident from the desire that target users, in both ecosystems, expressed to keep using ESSENCE after the end of the project. This overall appreciation is the most important result towards the exploitation. Indeed, the consortium identified 1 exploitable result targeted on children that is the whole platform to be deployed at school to better screen learning disorders and connect schools, families and child neuropsychiatrists at the point of need. This idea is completely aligned with the Italian law and the new international guidelines in the field.
In addition, 3 exploitable results were identified on seniors independently living at home: the monitoring of handwriting, the monitoring of voice analysis, the stimulating virtual rooms of the CBAC integrated with a telehealth platform to plan and monitor intervention and eventually schedule and carry out teleconsultation at the point of need.
For each of the exploitable results, the economic feasibility and the social impact were computed showing encouraging results for the future exploitation.
To conclude ESSENCE accelerated the route to CE mark. All the tests carried out had positive results indicating that the engineered prototype is quite ready for the CE mark procedure.
ESSENCE is a modular platform to support two vulnerable populations.
The obtained impacts are:
• To contribute to the public health response to COVID-19
• To generate a new model of care to facilitate the contact with professionals and to monitor users physical, cognitive and emotional conditions.
• To promote stimulation, through physical and cognitive activities and games.
• To favour social inclusion, allowing users to interact with each other in virtual rooms.
• To accelerate the market uptake of mature e-health technologies.
Such impacts are not restricted to the pandemic, indeed ESSENCE promotes prevention and early detection of deviations from physiological behaviours (seniors) or normal learning process (children), to favour the anticipation of intervention.
To further contribute to progress the knowledge on the field, we are making data publicly available as Open Data: https://zenodo.org/communities/essence2020/
The obtained impacts are:
• To contribute to the public health response to COVID-19
• To generate a new model of care to facilitate the contact with professionals and to monitor users physical, cognitive and emotional conditions.
• To promote stimulation, through physical and cognitive activities and games.
• To favour social inclusion, allowing users to interact with each other in virtual rooms.
• To accelerate the market uptake of mature e-health technologies.
Such impacts are not restricted to the pandemic, indeed ESSENCE promotes prevention and early detection of deviations from physiological behaviours (seniors) or normal learning process (children), to favour the anticipation of intervention.
To further contribute to progress the knowledge on the field, we are making data publicly available as Open Data: https://zenodo.org/communities/essence2020/