Community Research and Development Information Service - CORDIS

  • European Commission
  • Projects and Results
  • Periodic Reporting for period 1 - eNHANCE (eNHANCE - intention based enhancement of reaching and grasping in physically disabled people - personalized to maximize user performance)

eNHANCE Report Summary

Project ID: 644000
Funded under: H2020-EU.

Periodic Reporting for period 1 - eNHANCE (eNHANCE - intention based enhancement of reaching and grasping in physically disabled people - personalized to maximize user performance)

Reporting period: 2015-02-01 to 2016-07-31

Summary of the context and overall objectives of the project

The overall objective of the eNHANCE project is to develop, demonstrate and initiate exploitation of new concepts to enhance and train upper extremity motor function during daily-life in people with physical disabilities. We focus on two different patient groups: Stroke patients and patients with Duchenne muscular dystrophy. With the products that are finally developed from these concepts we expect that patient will enhance their self-support and, by being less dependent, improve their quality of life.

More specifically, the objectives are:

1) To develop an intelligent bidirectional multimodal adaptive interface for supporting upper extremity motor function, including
- a user input interface based on eye tracking and supplementary multimodal on-body sensing
- a controllable mechatronic arm and hand support interface
- an environmental observation interface.
2) To supplement this multimodal interface with intelligence to:
- assess user intention, behaviour and performance from the bidirectional interaction with the user through the multimodal interface with the user in interaction with the environment
- intelligently control the amount of support to the user through the bidirectional multimodal interface, taking into account user intention, behaviour and performance
- motivate the user to optimize level and quality of motor activity through this intelligent interface
- be physically challenging
- be aware and take into account context and social interaction
3) To demonstrate and evaluate the motor performance of the user in interaction with this intelligent interface. The demonstration and evaluation will involve two specific target groups of disabled people with upper-extremity motor impairments:
- Duchenne Muscular Dystrophies, as example of motor impairment of muscular origin
- Stroke, as example of motor impairment of central neurological origin
4) To prepare exploitation of the eNHANCE concepts in a Public Private Partnership (PPP) setting, involving key companies, end user communities, care institutions and knowledge institutions
5) To create strong and continuous user involvement through their involvement in user requirement identification, evaluation and dissemination of the eNHANCE concepts to stake holders and the general public

Work performed from the beginning of the project to the end of the period covered by the report and main results achieved so far

A controllable mechatronic arm and a hand support interface were developed. Although functional as support systems by themselves, these mechatronic interfaces will be used as test platforms for the intelligent multimodal adaptive control interface. Subsequently, an environmental observation interface was developed. Information from this environmental observation interface will be used in a later stage as input for the multimodal adaptive control interface. To control the mechatronic arm and hand support, a method to detect user intention has been developed. A head-mounted eye tracking system has been used to detect the 3D gaze point of the subject and a wink control has been implemented to differentiate “intentional” eye commands, meant to drive the arm support to a specific point in space, from “unintentional” eye movements, meant to simply explore the scene.

Different modalities to retrieve environmental information and user status have been implemented and combined together. Such modalities include eye tracking to study the gaze behaviour of the user, mechanomyography (MMG) and electromyohraphy (EMG) to record his muscular activity, full body tracking to study the movement of the whole body, head-mounted cameras to record the scene in first perspective. In a next step, this information will be used to develop algorithms that define the user intention according to his current status and his environment.

Literature is being reviewed for the possible motivation measures which can be implemented with the arm support system to measure the current motivation level. To check the quality of motor activity through the intelligent interface, performance measures are defined and tested for sensitivity towards different performance levels. The defined performance measures are found to be sensitive towards different performance levels. As core of the eNHANCE intelligence, a behavioural model is currently under development with the main goal of inferring the most probable actions the user is about to perform, and provide him with the adequate support. This model takes into account the environmental information (e.g. room in which the user is, objects around the user) and the intention of the user extracted from the multimodal sensor interface described above.

To demonstrate and evaluate the motor performance of the user in interaction with this intelligent interface, several series of user testing will be performed. As a first step, evaluations with healthy persons were performed. Evaluations in people with stroke and Duchenne Muscular Dystrophy are being planned.

To create user involvement and interest, an advisory board was established, with representatives of the industrial community, associations of stroke and Duchenne patients, medical professionals in clinical rehabilitation and home care, and health insurance agencies. Members of this advisory board were involved in the user requirements identification and evaluation. Additionally, two focus groups were organised in which people with stroke and Duchenne participated.

Progress beyond the state of the art and expected potential impact (including the socio-economic impact and the wider societal implications of the project so far)

The eNHANCE support enables the users to achieve their desired movement actions, while motivating the users to maximise their contribution. This will maximise the user performance relative to his or her personal capacity, and thus maximise the therapeutic effect. Maximising user performance requires a personalised and motivating support taking into account personal behaviour in response to arm and hand support characteristics and motivational inputs provided by the system, taking into account environment, context and social interaction. Personal behaviour will be represented by a personalised behaviour model that is adaptively identified based on the discrepancy between the performance that is actually observed and the one predicted by the model.

The eNHANCE concept for enhancing and training upper extremity motor function in people with physical disabilities includes:

1) a multimodal interface, consisting of:
- a user control input interface, with eye tracking being the key modality, supplemented with on-body sensing modalities including EMG, inertial body movement sensing and interaction forces with environment and support system;
- a mechatronic arm and hand support interface with supplementary communication; and
- an environment observation interface to create context-awareness.
2) A system to assess and characterise the user real-time, using the user control input and the data of the environmental observations, with the following components:
- user intention detection;
- user performance assessment.
- user behaviour modelling, predicting the user’s performance in response to the physical arm and hand support provided by the system and supplementary motivational communication, taking into account the observed environment, context and social interactions,
3) a personalised motor support intelligence that determines real-time the functional characteristics of the eNHANCE support, driving the multimodal adaptive interface:
- supporting user intention,
- optimizing this support and supplementary motivational communication based on the performance prediction by the user behavioural model.
- motivating the user to optimise level and quality of motor activity in interaction with the environment, taking into account potential cognitive limitations to provide physically and cognitively challenging interaction (not too difficult, not too easy)
- physically supporting the user to enable him to perform the intended motor tasks
- stimulate and enable social interaction, supporting joint activity with other people.
- supplementary persistent random excitation of user support and motivational communication to improve adaptive identification of the user behaviour model.

We foresee the following impact:

Scientific: We foresee that the eNHANCE project will contribute to new intelligent and adaptive multimodal interfaces to support people with motor impairments. These interfaces are expected to enable these people to improve their participation in society and are expected to be the basis for improving the innovation capacity of European companies in this domain, but also in much wider application domains. It is a necessary step towards context-driven support systems and towards a smart environment with a larger interaction between support systems and environment. The development of the ‘internet of things’ could be viewed in this context.

General public: eNHANCE will advance the capacity of human-machine interaction technologies to enable people with motor impairments to improve their participation in society by enabling and stimulating them to be physically active in their daily-life environment and in social interaction with other people. The potential of the technologies was explained to the general public different interviews at BBC One, and it was presented at conferences of various patient organizations.

Economy: The eNHANCE consortium includes well-chosen dynamic SME companies that have complementary exploitation plans for the eNHANCE concepts. Due to the bankruptcy of the company EyeBrain, the envisioned prototype of a low cost high performance eye tracking with improved intention detection system will not be made. Instead of this prototype we, however, envision that the company Focal Meditech will develop a prototype of a robotic arm (one of their existing products) that is controlled by an Eyetracking interface. A first derived prototype in which eNHANCE technology was implemented was developed and presented by Focal at the Rehacare exhibition in Dusseldorf (September 28th 2016 until October 1st 2016).

Related information

Follow us on: RSS Facebook Twitter YouTube Managed by the EU Publications Office Top