Community Research and Development Information Service - CORDIS

H2020

PAL Report Summary

Project ID: 643783
Funded under: H2020-EU.3.1.

Periodic Reporting for period 1 - PAL (Personal Assistant for healthy Lifestyle (PAL))

Reporting period: 2015-03-01 to 2016-02-29

Summary of the context and overall objectives of the project

The growing burden of chronic illness on health and health care has globally led to health policy responses increasingly referring to self-management. This applies to the increasing number of children and adolescents in Europe with a chronic illness. For example, the incidence of childhood type 1 diabetes mellitus (T1DM) is rising rapidly, with a doubling time of less than 20 years. T1DM is associated with serious complications, which may appear sooner or later, cause high morbidity and mortality, affect the quality of life, and increase health-care costs. Complications can be prevented by performing self-management (e.g., monitoring blood glucose, recognizing symptoms and injecting insulin). However, self-management is not an easy goal to attain for young patients, since it requires motivation and long-term perseverance, in order to become ‘a way of life’. Self-management in children and adolescents is strongly affected by a diversity of personal and environmental factors, such as the child’s developmental stage, parent’s support and health care providers. Furthermore, children need not only to learn to self-manage their lifestyle-related diseases to improve their situated health-related habits, but also to be prepared for the physical and social changes at adolescence.
mHealth technology could play an important role in supporting self-management for children with a chronic condition such as T1DM. Children are getting more and more accustomed to mobile technology as an integral part of their daily lives and are developing a natural relationship with this technology. Various mHealth systems, services and applications have been devised, focusing on support of specific health professionals’ care activities, or on specific aspects of patient’s knowledge, skills and habits for coping with their illness. These applications have their own specific value, but are unable to deliver comprehensive and pervasive personalized support to the young patient and his or her caregivers over time and contexts (i.e., attuned to the patient’s developmental stage and the context in which self-management is supported). Our vision is to improve on this and make a difference.
The overall aim of the project is to develop a Personal Assistant for healthy Lifestyle (PAL), a system that will assist the child, health professional and parent to advance the self-management of children with type 1 diabetes aged 7 - 14, so that an adequate shared patient-caregiver responsibility for child’s diabetes regimen is established before adolescence. PAL will be composed of a social robot, its (mobile) avatar, and an extendable set of (mobile) health applications, which all connect to a common knowledge-base and reasoning mechanism.

1. To assess user’s (child, parent, health professional) needs and determinants to support long-lasting self-management behavior of the child from childhood into early adolescence, i.e.,
a. child’s behavioral determinants (e.g., knowledge, awareness, attitude, self-efficacy, social support, skills) that affect his self-management (e.g., medication adherence, glucose monitoring) and thus his health condition (e.g., glycemic control),
b. parent’s determinants (e.g., knowledge, attitude, skills, trust) to provide health-autonomy support and shared responsibility for their child’s self-management,
c. health professional’s perceived benefits and barriers of using PAL to provide personalised care and self-management support according to T1DM guidelines.
2. To develop an evolving transparent ontology for persistent knowledge-based multi-user assistance, i.e.,
a. for “user-in-the-loop” learning and reasoning,
b. for extracting and integrating semantics from available healthcare assets, and
c. for establishing a normative system with different support roles for the child, professional and parent.
3. To develop an agent-based reasoning mechanism for the personalised setting of child’s learning and behavioural goals, and engagement strategies.
4. To develop support tools for the caregivers, i.e.,
a. an authoring & control module for the health professional to tailor the PAL support to the child, to monitor and learn from child’s progress, and to operate the robot when needed, and
b. a monitor & inform module for the parents to improve their knowledge and attitude on child self-management, without violating the child’s values.
5. To develop a user model, and machine-learning and sentiment-mining methods that feed this model, for attuning the support to child’s developmental and behavioural change stage, condition, preferences, knowledge, skills and experiences.
6. To develop consistent personalized multimodal natural interactions for the physical (robot) and virtual (avatar) embodied conversational agent (ECA), fostering long-term engagement in a range of educational or assisting (mobile) health applications, and across a range of situations and interaction devices.
7. To further develop and integrate (mobile) health applications into an open infrastructure for comprehensive knowledge-based self-management support of children with diabetes.
8. To determine the costs and profits for the proposed self-management support.

Work performed from the beginning of the project to the end of the period covered by the report and main results achieved so far

"This subsection presents the work performed from the beginning of the project to the end of the period covered by the most recent progress reports. The activities and main results achieved so far will be summarized per work package.

Work package 1 has two general objectives. The first objective is to identify the user needs, human factors knowledge and technological opportunities for the project and to take care that these needs, knowledge and opportunities are well-addressed throughout the development process. The second objective is to develop and maintain a reusable, evidence-based, design knowledge-base that specifies these functions and their effects on self-management.
• In the first period, the initial data management plan (DMP) for the PAL project was created in DMPonline https://dmponline.dcc.ac.uk to ensure that the data are managed and preserved well. It considers data gathering, metadata generation, data analysis and data storage, encompassing both (1) the specifications of user and functional requirements and (2) the empirical evaluations with the children and (in)formal caregivers. The DMP supports a sound and efficient gathering of information and by structured refinement processes with the stakeholders.
• In the domain and support analysis, the stakeholders (children with diabetes, informal and formal caregivers) have been involved intensively: During interviews, focus groups and diabetes camps. They provided (1) detailed insights in the situated user values and needs, and (2) input and feedback for the PAL designs. For the human factors analysis, literature on behavioral change, developmental psychology and diabetes have been studied, this resulted, among other things, in the identification of key elements of the Self Determination Theory (SDT) for PAL-support. For the technological analysis, we provided a state of the art overview of apps and platforms for diabetic users.
• A semi-formal structure or template for design specifications was constructed for defining, maintaining and sharing a re-usable design rationale (i.e. capturing the evolving design knowledge). Personas, scenarios, use cases and storyboards were created to derive the first set of PAL requirements and corresponding claims (specified according to the template). Subsequently, these requirements have been prioritized for the functionality of the first PAL prototype (PAL Actor, MyPAL, PAL Control and PAL Inform).
• A network of connected ontologies ("frames") have been constructed, each consisting of general concepts and their relations: (1) roles and actors, (2) emotion and sentiment, (3) task, goal, activity and context, and (4) diabetes self-management. For these frames, existing knowledge models are being adapted and extended to meet the PAL scope and objectives. A specific ontology frame is being worked out for the PAL reasoning and querying.
• A formative prototype evaluation has been conducted to refine the initial design specifications and collect child-robot interaction data to feed the first modeling activities. The evaluation protocol for the first cycle evaluation in Italy and the Netherlands is being worked on.

Work package 2 focuses on the strategies or mechanisms to provide the child with a long-term and engaging experience: The modelling of learning goals, progress and cognitive-affective state, and the design of personalized style-based robot behaviors.
• The development of an authoring tool for care professionals, called PAL Control, was started. This tool enables health care professionals (HCP) to set learning goals for children during meetings. It further enables the HCP to enter child data including personal data and preferences such as sports and hobbies. Three important issues were tackled: How to design the user interface to facilitate easy data entering during an intake conversation, how to support gamification of goal achievements, and how to formalize the learning goals based on the medical protocols? Following the general incremental development approach, we have chosen to keep the tool as simple as possible, as to elicit needs from caregivers. As planned, the authoring interface will be evaluated during the first testing campaign.
• We studied how robots in educational settings need to be adapting there interaction style to individual children’s learning styles to maximize learning outcome and to maximize a constructive and pleasant experience for the children. Learning style and preferred interaction style of the virtual NAO robot can, in a later stage, be added to the goals of the authoring tool to steer strategic selection of learning goals (e.g., some children like to achieve learning goals by doing while others prefer to achieve the same goals by thinking).

Work package 3 focuses on the rapid personalization of PAL behaviors (typically, the robot or its avatar) to each of its users for two objectives: 1) increasing the engagement of the user in the PAL system, and 2) allowing the user to reach more effectively its personal goal(s) by adapting to his/her preferences. An action selection architecture is being developed, which allows us to experiment with and combine different predictive user models (encompassing user characteristics, preferences, level of skills or knowledge) and potential actions.
• A literature study provided review of the state of the art of action selection and personalization approaches in human-robot interaction. There proves to be a need for considering at the same time personalization (to fit the user's preferences) and adaptation (to follow the changes in the user's preferences over the time). Work package 3 aims to address both of these features in a single action selection framework, which is using as inputs the desired goals and the user cognitive state that are determined by modules in Work package 2 and the potential actions that are suggested by the interaction module, developed in Work package 4. The first prototype of this action selection module is based on the HAMMER architecture (Hierarchical Attentive Multiple Models for Execution and Recognition), developed by Imperial College.
• Two sets of experiments have been performed with the first prototype: 1) Validation of the implementation and 2) Stress test of the implementation. Based on the presented results, we demonstrate that the architecture successfully works with several concurrent models and manages to rapidly identify the most accurate one. The results also show that our current implementation of the architecture is able to simultaneously execute several thousands of concurrent simple models per second allowing the system to consider a large number of alternative actions without affecting the system reactiveness, even if the computational complexity of the underlying models increase greatly. These results demonstrate that this action selection architecture has the computational power and efficiency required for use in the PAL project.

Work package 4 is developing the means to conduct verbal communication, and to analyze textual data and extract relevant information, in such a way that it fosters sustainable long-term child-PAL interactions. This requires user-adaptive communication, coupling of verbal and non-verbal communication and grounding communication in long-term memory.
• We worked on a novel approach for dialogue management that treats the specification and storage of domain knowledge and interaction structure uniformly. This uniform knowledge layer will support almost all activities of WP4, such as natural language (NL) interpretation, NL generation, user and role adaptation, and the definition of dialogue policies. For the modelling and storage of the user and domain information, we developed an ontology that allows to represent interaction in a way that is natural and efficient to use. We heavily extended existing processing components, i.e., the reasoning engine and database layer, which make the data available to the interaction management and analysis. Besides, we connected the Google automatic speech recognition API to the common PAL infrastructure.
• We adapted and extended an existing component for verbal generation of Italian using a deep-generation approach, involving utterance content planning and grammar-based surface realization. Since the underlying system using this grammar is bi-directional, the grammar can later on also be used for interpretation. We also worked on a free Dutch text-to-speech voice for the Mary TTS system, in order to obtain a uniform voice for the virtual and the embodied robot.
• We built a prototype system from existing components for the first set of experiments. The purpose of this prototype was to provide a system for data collection and experimentation in short time, before having the new system architecture in place. The unavoidable adaptations were kept to a minimum, since this prototype will not be sustained, but be replaced by the new, more powerful and flexible architecture and components. To inform the development of the system, we analyzed the interaction data in order to evaluate how the children's perception of the robot is influenced by their individual interactions with the robot, and by specific relational verbal behaviors.

Work package 5 is developing the PAL architecture that comprises the general IT infrastructure and the integration of the main software modules. The software development is driven by the user requirements analyses of WP1. The main goals of WP5 are (1) the design of the overall IT architecture that is capable to fulfill the user requirements, (2) the identification of the software modules that compose the PAL IT infrastructure, (3) the design of the main data flows among the software modules, and (4) the identification of the teams and leaders in charge of the development of each software module of the PAL project (i.e., allocation of responsibilities).
• The general architecture has been established, which assembles the identified modules. Since the modules have tight and frequent interactions and several interdependencies, periods of face-to-face integration meetings have been planned and executed.
• PAL aims at Artificial Intelligence (AI) and user model–based support with advanced human-agent interactions. The innovative reasoning mechanisms have been defined and worked out into three main components. First, the Interaction Manager establishes the child-PAL interactions based on a dialogue model and ontology. Second, the Child model Agent tailors the possible actions based on the analysis of his estimated emotions and sentiments. Third, the Adaptive Action Selection Model, based on HAMMER, selects an appropriate action to propose to the child or to use to interact with the child.
• Since the PAL Architecture requires several hardware devices and software modules connected, one with the other, and frequently exchanging data, an online and real-time solution has been adopted. To exchange data among the several software modules a message dispatcher approach has been chosen.

Work package 6 focuses on the dissemination and valorization of the PAL project outcomes. The goal is to increase awareness about PAL's innovative role in supporting children with Type 1 Diabetes Mellitus (T1DM) and its ability to generate new ICT healthcare models, tuning the messages to be conveyed on the public to which they're directed.
• A general strategy with tasks among different dissemination channels has been created to achieve this goal, i.e., manage the sharing of knowledge, build and raise awareness on the project outside, produce appropriate communication material, disseminate knowledge, methodology, results & lessons learned organize demonstrations for healthcare professionals, technology players & industries, determine the health & economic impact of the PAL's solutions use for the project's end-users, and gather feedback from users and stakeholders to refine and improve service and solutions during and beyond the project's life.
• To measure the impact, a number Dissemination Indicators have been identified. These will be monitored over time and reported yearly, updating the present document, so that they can provide a brief overlook of the expected progresses made: PAL project website and social network Indicators, National and International events attended, Publications, users who access to the PAL online tools, feedback from interested people. This list may be edited and extended, during the project, if specific developments of the project ask for it or on the basis of the software utilized for the stats analysis."

Progress beyond the state of the art and expected potential impact (including the socio-economic impact and the wider societal implications of the project so far)

In this first period, the PAL achieved initial progress beyond the state of the art, which is expected to have potential impact. The first foundation was established for achieving the project objectives in the three design and test cycles, setting the foci on realistic and pregnant health outcomes and self-management behavior per cycle. Furthermore, a common architecture and research approach was agreed upon for a systematic and incremental development of the integrated system from the start on. Due to the emphasis on the identification and integration of PAL system components, these components have yet limited (intelligent) functionality in the first year prototype. However, it provides the foundation to work on these components intelligence the coming years, ensuring adequate integration.
In addition to the establishment of the PAL foundation, there are initial project results that are expected to have impact in the future.

First, an innovative tool was developed: A structured interview and questionnaire based on the know and do-objectives that were defined in 2011 by the Dutch diabetes nurses association.

Second, a coherent set of co-design methods have been worked and applied to identify, refine and test user requirements and interaction designs with children. In the fields of human-computer interaction, virtual agents and social robots, there is a clear need for such methods to develop personalized support that children will use in a beneficial way and over longer periods of time.

Third, an initial uniform knowledge layer was specified, an ontology, that provides a univocal and consistent conceptual framework for the PAL project. A specific part has been worked out, which represents interaction in a way that is natural and efficient to use, for the modelling and storage of the user and domain information.

Fourth, a rich set of dissemination activities were performed. We participated and presented at international conferences (among other things, organized two child-robot interaction evaluation workshops), showed the project to SMEs, and were present during camps and world-diabetes day meetings where children, parents and caregivers came.

Related information

Record Number: 190331 / Last updated on: 2016-11-14
Follow us on: RSS Facebook Twitter YouTube Managed by the EU Publications Office Top