Coach Assistant via Projected and Tangible Interface

Project Information

CAPTAIN

Grant agreement ID: 769830

DOI

10.3030/769830

Project closed

EC signature date 1 August 2017

Start date 1 December 2017

End date 30 June 2021

Funded under

• SOCIETAL CHALLENGES - Health, demographic change and well-being

Total cost

€ 3 990 200,00

EU contribution

€ 3 990 200,00

Coordinated by

ARISTOTELIO PANEPISTIMIO THESSALONIKIS
Greece

Periodic Reporting for period 2 - CAPTAIN (Coach Assistant via Projected and Tangible Interface)

Reporting period: 2019-06-01 to 2021-06-30

Summary of the context and overall objectives of the project

The Captain project was funded by the European Commission Horizon2020 programme under the topic on “Personalised coaching for well-being and care of people as they age” in the period 2017-2021. It aimed to develop and validate useful and easy to use ICT tools for empowering and motivating people in need of guidance and care. It proposed a technology designed to turn the home of an older adult into a ubiquitous assistant introducing home augmented reality using interactive and tangible interfaces. To achieve this, CAPTAIN fosters a user-centered co-design methodology which is based on constant involvement of older adults (the Captain Stakeholders’ Community) in the design, development and testing stages.
Older adults can benefit from the creation of effective contextualised assistance in their home environment forming a virtual coach who is stimulating and enhancing elderly capabilities. The consortium has analysed scope of coaching activities for CAPTAIN and a personalized behaviour change digital intervention has been developed and tested with real users addressing 4 different lifestyle issues: healthy nutrition guidance, physical activity support, cognitive training, and socialization support. This coach acts as a vigilant and pleasant companion, ubiquitously present, that provides contextualised advice to help preserving elderly mental and physical, cognitive and social well-being. The participatory design process adopted in Captain harnesses the expertise of all levels of the stakeholder ecosystem in order to deliver an effective tool for people in need of guidance and care due to age related conditions. The releases of the CAPTAIN system were tested in 6 living labs/partners of the consortium.
A multidimensional evaluation was planned, including user acceptability, ease of use, user satisfaction and perceived usefulness of the technology. Real-home deployments were possible in some of the countries involved through a revised (due to the pandemic) protocol where data was gathered from primary CAPTAIN end users over a short period of time. An overall assessment of the CAPTAIN project was performed to evaluate and validate the quality of the project methodology and produced technology, which demonstrate the potential of the CAPTAIN system and its acceptance.

Work performed from the beginning of the project to the end of the period covered by the report and main results achieved so far

The CAPTAIN project has been designed to develop and validate useful and easy to use radically new ICT based concepts and approaches for empowering and motivating people in need of guidance and care due to age related conditions. One of the first tasks in CAPTAIN was the creation of a network of loyal stakeholders and users coming from the pilot partner’s living lab networks which has been actively engaged in the many co-creation sessions since the first year of the project and has contributed by providing feedback on every new release, data collection and new ideas.
A clear methodology (modified version of SCRUM) has been defined and followed. The series of Sprints (slightly different from the one used in the software development, but applicable as concept) were the main events. Captain Sprints lasted for 3-4 months and resulted to the functional state of a product released version, while agile requirements elicitation and agile development enabled the researchers to elicit detailed requirements up front, or when needed. In this process, numerous issues regarding ethics had to be considered given the continuous data collection in living labs. Ethical applications were submitted, approvals were obtained, GDPR regulation was taken into account.
The hardware prototype and software development has evolved during the implementation of the project. CAPTAIN introduced a couple of new (i.e. not off-the-self) devices and the first prototypes of CAPTAIN were designed and manufactured by partners. At the same time the consortium implemented non-invasive and environment sensing algorithms, improved by additional datasets collected. The design and development of the coach behaviour was developed and a coaching (behaviour changing) framework has been established as the guideline for all the different components. An overall assessment of the CAPTAIN project was performed, although data collection was significantly impacted by the advent of the COVID-19 pandemic and associated public health restrictions.
During the project lifetime a lot of activities for increasing awareness of the general public and policy makers as well as the scientific community have taken place successfully. Of course, the COVID-19 pandemic put barriers on the approach to CAPTAIN’s dissemination, however, the consortium managed to communicate successfully the scope and achievements of the project and make them widely known to the community and interested parties.

Progress beyond the state of the art and expected potential impact (including the socio-economic impact and the wider societal implications of the project so far)

The main innovation potential of CAPTAIN lays in its multidisciplinary approach which leverages on a variety of state of the art technologies or methodologies such as artificial intelligence, projective interfaces, augmented reality, 3D sensing, deep learning, context and emotion detection, to name but a few.
CAPTAIN goes beyond usual participatory design and requirement elicitation techniques by applying agile requirements elicitation and development methodologies through participatory design throughout the lifecycle process of the development. The CAPTAIN Stakeholders’ community, consisting of those who will actively use CAPTAIN (older adults and their caregivers), and those who can give suggestions (service providers, nursing home management and patient associations), has been the only source of requirements throughout the project’s lifecycle, playing an important and substantial role of co-designers and co-creators. CAPTAIN's methodology (combination of Design Thinking, Lean Startup and a modified SCRUM agile framework) was specially designed to support the peculiarities of distributed and multidisciplinary teams. As this approach goes beyond the state of the art in similar projects, an evaluation tool has been designed to provide the value of this approach when applied in the H2020 funded CAPTAIN project.
Another domain where CAPTAIN brought innovation is the minimum intrusiveness of the technology. The main objective of CAPTAIN was to leave the environment as a cozy place for the older adults, without intervening too much, unless a coaching activity was required. Aligned with this, no wearable devices have been used, not even tablets or smartphones. The main monitoring device of CAPTAIN is a depth sensor (the CAPTAIN Satellite Detector) as well as the CAPTAIN Satellite Projector equipped with a camera, converting the indoor environment into a large tangible interface. The interaction with the system is delivered through projection systems (both of them designed and manufactured within the project's lifecycle).
The way the CAPTAIN system has been designed enabled its evolution into a system that the targeted user group will like to use. Focusing on nutrition, physical exercise and socialization, the CAPTAIN system is designed to support not only vulnerable older adults but also healthy ones. The consortium has managed to generate acceptance and trust, support the user-centered approach and the involvement of beneficiaries in its testing, even during a difficult pandemic emergence, which creates the conditions for up-taking the project results through exploitation of its achievements.

An older adult during pilot use of CAPTAIN

Family photo of the CAPTAIN consortium in the framework of a Plenary Meeting in Lausanne

CAPTAIN hardware

Design Thinking session with older adults in Greece

Family photo of the CAPTAIN consortium at the kick-off meeting in Thessaloniki - December 2017

CAPTAIN system in a home setting