Skip to main content

Sensor Enabled Affective Computing for Enhancing Medical Care

Periodic Reporting for period 2 - SenseCare (Sensor Enabled Affective Computing for Enhancing Medical Care)

Reporting period: 2018-01-01 to 2019-12-31

SenseCare has the primary aim of enhancing and advancing future healthcare processes and systems using sensory and machine learning technologies to provide emotional (affective) and cognitive insights into patient’s well-being so as to provide them with more effective treatment across multiple medical domains. The specific focus is to develop technologies and methods that will lessen the enormous and growing health care costs of dementia and related cognitive impairments that burden European citizens. Dementia is a hugely challenging social issue. People who develop dementia, as well as their carers, face a significantly stressful, even traumatic future. Creating new practices that support them is essential in future health services, and, given the scale of the challenge, technology will need to play a significant role. Affective computing, though still an emerging technology, should be explored for its potential in this context and SenseCare has taken on the role of building a body of know-how to enable this.

More specifically SenseCare had the following overall Objectives:
• Development of the SenseCare affective computing operating system platform as an extensible multi-layered architecture based on open standards.
• Develop, test and deploy various prototype software modules in the SenseCare software platform.
• Specify and enforce security, legal and ethical guidelines in relation to the deployment and use of SenseCare powered applications in the healthcare field.
• Implement selected use cases as test pilots for the dementia care and connected health domain using the SenseCare platform.
• Trial, test and evaluate feedback on the SenseCare above test pilots with patients, care givers and healthcare professionals.

In terms of outputs to date, SenseCare has:
• developed a SenseCare architecture and platform design, with prototyped elements.
• created a portal to host a repository of affective computing knowledge.
• built a strong culture of research, led by the secondees.
• built several use cases upon the architectural design and will use this to engage with a range of healthcare professionals.
• successful engagement with key stakeholders, people with dementia and their carers
• developed a significant framework for development of prototype solutions.
• developed approaches to apply this and develop new tools, built upon the platform design, that support the development of new computing practices.

The SenseCare consortium is: Cork Institute of Technology [CIT], University of Ulster [Ulster], Forschungsinstitut für Telekommunikation [FTK], GLOBIT and INMARK
The project was broken down into the following Work Package (WP) activities :

WP1. Use Case Requirements [Ulster] to identify, define and specify the most relevant use cases in which the SenseCare platform will ultimately be deployed. This WP produced some of the key use case descriptions and framed the design approach with projects key participants

WP2. Affective Computing & Machine Learning [CIT] to develop and apply affective computing algorithms to form the intelligent nucleus of the SenseCare platform This WP produced novel multi sensory fusion solutions and tested them to validate the foundation of the SenseCare concept.

WP3. Big Data Fusion & Machine Learning [FTK] to develop a number of input interfaces for specific sensory devices such as cameras, wearables and Internet of Things. This WP developed the overall architecture from which the SenseCare ecosystem can be created and developed.

WP4. Psychology of AC [INMARK] to apply the most current thinking, models and algorithms from psychology and will form an integral input to other work packages. This WP provided the core practical knowledge on the behavioral requirements for the SenseCare system specification and the key ethical insights

WP5. Medical Informatics [GLOBIT] responsible for the configuration, interface, deployment and testing of the use case SenseCare applications that have been specified. This WP provided the framework for the user interaction and a design baselinse from which development of the SenseCare prototypes could be implemented

WP6. Project Management [CIT] to enable the consortium to deliver on the target objectives. This WP provided the framework to support the evolution of the project in the context of the required objectives

WP7. Dissemination and Communication [CIT] to enable engagement with a range of stakeholders relevant to the SenseCare concepts. This WP supports the dissemination approach and the community development initiatives in adapting to the SenseCare development work
SenseCare targeted the following impacts for the project
• Enhancing research- and innovation-related human resources, skills, and working conditions to realise the potential of individuals and to provide new career perspectives
• To develop new and lasting research collaborations, to achieve transfer of knowledge between research institutions and to improve research and innovation potential at the European and global levels
• Develop a way to build the self-sustainability of the partnership after the end of the project.
• Contribute to the improvement of the research & innovation potential within Europe and/or worldwide.
• Position SenseCare solutions effectively in a futuristic marketplace and one in which Europe must be involved with/in for sustained economic prosperity and overall EU citizen well-being.

Progress towards achieve these impacts have largely seen successful outcomes. CIT developed a suite of prototypes in working with FTK, Globit and ULster and shared a series of novel studies on emotion recognition. CIT is currently working through regionally funded feasibility initiatives to explore the commercial potential of these systems.

FTK and Globit have developed SenseCare use cases and build prototypes with CIT. FTK, CIT and ULster have been successful in winning further EU funding together. The publications strategy of the project worked well, with secondees publishing at workshops, conferences and peer-reviewed publications. Importantly, a culture of collaboration developed and this has resulted in many joint publications.

Through use cases and the prototypes, the tools to explore the development of new practices with healthcare professionals now exist. SenseCare has explored the paradigm of communities of practice to develop engagement and made progress in Ireland, Germany and the UK in developing capacity to leverage the project work. ULster has built a community of partners within their region to further support the exploration of affective computing solutions. ULster has also progressed the development of technological solutions in the form of Chatbot solutions leveraging the SenseCare ecosystem

In terms of impact to date, SenseCare has:
• developed a significant framework for development of prototype solutions based around the concept of a SenseCare ecosystem
• In particular built a novel foundation for further research through the development of successful AC multi-sensor fusion techniques
• built a number of innovative use cases upon the architectural design and will use this to engage with a range of healthcare professionals.
• developed approaches to apply this and develop new tools, built upon the platform design, that support the development of new computing practices.
Figure 1 - SenseCare Multi-Layered Platform Architecture