Remote monitoring system can flag up developmental and degenerative problems
“The aim was to leverage technology for a new model of care that relies on tele-assistance, remote monitoring and social inclusion,” explains project coordinator Simona Ferrante, professor at the Polytechnic University of Milan in Italy. The ESSENCE platform targets two groups – seniors, and children 6-8 years old – and integrates artificial intelligence (AI) modules. “These are able to detect that something is gradually degenerating in seniors. In children they can indicate that they are not following the typical learning curve,” she says. The project responded to an EU emergency COVID-19 call in 2020, and included consortium members that previously worked on the EU-funded MoveCare project monitoring and assisting the elderly to counter decline and social exclusion. The ESSENCE platform expands on this to cover what she calls “two ecosystems”, using the same technology but different modules, such as the interface for professionals, serious games for learning or cognitive stimulation or virtual rooms for socialisation while playing collaborative or competitive games. One ecosystem was tested with 70 seniors in the Extremadura region of Spain and the other on 66 primary schoolchildren in Lombardy, Italy.
Children’s classroom activities
The platform supported teachers in planning class activities and observing children’s difficulties in reading, calculation and writing. Through the platform’s dashboard for professionals, teachers were able to monitor children playing specially devised games on a tablet, that were structured around child development indicators. These included a handwriting game using a novel smart ink pen patented by the participating institutions. “Before the project, we had already acquired data on about 250 children playing the game for comparison. We also collected handwriting data for about 500 children using the smart ink pen, to model learning curves and to detect deviations that need attention,” Ferrante remarks. “Based on the screening, teachers were supported in selecting what training to provide in writing, calculation and reading through serious games or specific exercises, but also through virtual rooms for social games.”
AI function produces alerts
“Once teachers identified any weakness, ESSENCE provided a first contact with clinicians through a tele-consultation app that integrates some digital tests,” Ferrante adds. “We were able to help teachers in standardising the way to detect difficulties and help with the first contact with clinicians, so families felt more protected in this environment.” Children in primary grades 1 and 2 and their teachers were involved in testing the platform over a whole school year. Usability and user satisfaction were high, Ferrante notes.
Detecting early degeneration in the elderly
For the senior ecosystem, handwriting data using the smart ink pen and voice analysis data gathered during phone calls via a voice app were analysed for specific indicators of cognitive or emotional status. These were used to build AI algorithms, leveraging data from the previous MoveCare project, that had achieved good accuracy in predicting cognitive decline, Ferrante says. “The system could understand how these indicators were degenerating over time.” “Handwriting involves both cognitive and physical aspects and is a prime marker of degeneration,” she notes. Stimulation was achieved through games played in virtual rooms with peers. “They got to know each other and it was very positive because it was a way to meet other people,” Ferrante explains. “I was not expecting such good compliance,” she adds, noting most participants wanted to continue using the system beyond the end of the project. Ferrante said work would continue with further projects in both ecosystems.
Keywords
ESSENCE, COVID-19, child development, elderly, seniors, cognitive degeneration, learning difficulties, remote monitoring, MoveCare, handwriting, virtual rooms, artificial intelligence
