Personalized intelligent platform enabling interaction with digital services to individuals with profound and multiple learning disabilities

Projektinformationen

INSENSION

ID Finanzhilfevereinbarung: 780819

DOI

10.3030/780819

Projekt abgeschlossen

EK-Unterschriftsdatum 7 November 2017

Startdatum 1 Januar 2018

Enddatum 31 Oktober 2021

Finanziert unter

• INDUSTRIAL LEADERSHIP - Leadership in enabling and industrial technologies - Information and Communication Technologies (ICT)

Gesamtkosten

€ 2 255 875,00

EU-Beitrag

€ 2 255 875,00

Koordiniert durch

INSTYTUT CHEMII BIOORGANICZNEJ POLSKIEJ AKADEMII NAUK
Poland

Periodic Reporting for period 2 - INSENSION (Personalized intelligent platform enabling interaction with digital services to individuals with profound and multiple learning disabilities)

Berichtszeitraum: 2019-07-01 bis 2021-10-31

Zusammenfassung vom Kontext und den Gesamtzielen des Projekts

The INSENSION project aims at designing and developing an ICT platform that is capable of recognizing non-symbolic behaviors of people with profound intellectual and multiple disabilities (PIMD, also referred to as profound and multiple learning disabilities – PMLD), and interpreting them in the situational context as intentions or decisions of the given individual with PIMD. Such an intention can then be consumed by specialized digital assistive applications. This way the actions undertaken by those applications are triggered by the individual with PIMD and reflect her/his current needs. Thanks to this people with this type of disability receive an opportunity for increasing their self-determination. As a consequence the life shall be enriched with a possibility of experiencing a wider range of opportunities and the quality of their life shall increase.

Today, people with PIMD who often communicate on the pre-verbal level can interact with a very limited group of caregivers familiar with the highly individual characteristics of a given person with disability. Transition from being under the care of one person to another is therefore very difficult. By using the INSENSION system the person with PIMD gains a sort of prosthesis of communication enabling them to potentially engage into interacting with other people. Moreover, the system which is capable of correctly interpreting the need of its user with disability allows her/him to undertake certain action such as switching on the heating device when they feel cold completely on their own, without the need of other people’s intervention.

To construct such a platform the INSENSION project employs a range of state-of-the-art ICT technologies from the area of artificial intelligence, computer vision and distributed systems. The objective of the project is therefore advancing relevant technologies to enable them for the integration within the targeted solution.

Arbeit, die ab Beginn des Projekts bis zum Ende des durch den Bericht erfassten Berichtszeitraums geleistet wurde, und die wichtigsten bis dahin erzielten Ergebnisse

In the first reporting period the project gathered a deep understanding of the potential communication schemes of people with PIMD and practically verified the possibility of constructing a system capable of recognizing relevant non-symbolic behaviors of people with PIMD. To this end a comprehensive assessment tool built with the use of questionnaires assessing relevant aspects of interaction of people with profound intellectual disabilities with others has been created and validated on more than 20 persons with PIMD from Poland and Germany. The information collected with the use of this tool as well as video and audio recordings of non-symbolic behaviors of people with PIMD allowed the project to verify a number of artificial intelligence methods for the recognition and interpretation of facial expressions, gestures and vocalizations. These have been integrated into a first prototype of the system. This prototype has been constructed based on the detailed definition of the technical design of the system, enabling to demonstrate how the developed solution could work on the technical level.

The main results achieved in the first reporting period are:
- definition of the INSENSION assessment tool for the comprehensive assessment of interaction of people with PIMD;
- development of proof-of-concept software components capable of recognizing and interpreting the non-symbolic behaviors of people with PIMD;
- integration of a demonstration prototype acting as a mockup of the technical concept of the developed system.

Fortschritte, die über den aktuellen Stand der Technik hinausgehen und voraussichtliche potenzielle Auswirkungen (einschließlich der bis dato erzielten sozioökonomischen Auswirkungen und weiter gefassten gesellschaftlichen Auswirkungen des Projekts)

In the first reporting period the project has significantly progresses beyond the state of the art in the field of understanding the interaction of people with PIMD with others. It has defined a comprehensive assessment tool enabling to collect relevant information. It created the so called Global PIMD Atlas which is an electronic repository system aimed at gathering information on the PIMD population thus facilitating further studies of this disability type. As far as the technological state of the art is concerned, the project has used several advanced ICT technologies in the context of direct assisting of people with PIMD. To the knowledge of the project consortium this has most likely been the first such attempt in the world that allowed to successfully confirm that employing a range of technologies from the AI and machine learning domain has a vast potential in opening new opportunities for people with PIMD, including in their everyday life and education. It is expected that until its end the project constructs a fully working, integrated technical solution for the recognition and interpretation of the non-symbolic behavioral signals of people with PIMD, and enables it for use within application scenarios ranging from communication of people with PIMD with others, through entertainment to smart home. The project is foreseen to validate the developed solution in a comprehensive study with the participation of a number of actual individuals with PIMD, thus providing evidence of the influence of advanced ICT on the life of people with PIMD and persons directly supporting them.

It is expected that the technological solution designed and developed within the project will contribute to the implementation of disability policies and other EU and UN level strategies concerned with the inclusion of people with disabilities in the society, to the enhancement of the quality of life of people with PIMD and their caregivers, and to the increase of assistive technologies market.

Project logo

INSENSION Platform concept

Visualization of results of computer vision techniques usage in Insension