Periodic Reporting for period 1 - PhilHumans (Personal Health Interfaces Leveraging Human-Machine Natural Interactions) Reporting period: 2019-01-01 to 2020-12-31 Summary of the context and overall objectives of the project The goal of the PhilHumans project is to train a next generation of young researchers in innovative Artificial Intelligence (AI) and establish user interaction with their personal health devices in an advanced and intuitive way. The project explores cutting-edge research topics related to AI-supported human-machine interfaces for personal health services. PhilHumans will commit to responsible research and innovation to establish disruptive and innovative technology for AI-assisted human-machines interfaces, employing language technology, cognitive computing, computer vision, and machine learning (ML). The technology can be applied in a number of personal health contexts and extend or being coupled with Home healthcare, as well as in additional healthcare fields such as population health management and provide several benefits to users making sure science and research is conducted with and for society. Research objectives: In order to realize the overall research goals, the training and research network of the project will explore AI knowledge and expertise from Natural Language Generation (NLG) & Processing (NLP), Cognitive Computing, Computer Vision, ML focusing on 5 research objectives. Based on sound career development plans, and coached by experienced supervisors a training is offered by leading image analysis research groups from Philips (global leader in medical imaging) and the Eindhoven university of Technology (worldwide recognized authority in education and research on image analysis, esp. on MRI) and supported by researchers from leading universities like University of Cagliari, University of Catania and University of Aberdeen. Work performed from the beginning of the project to the end of the period covered by the report and main results achieved so far In the first half year the ESRs started their introduction in the organisation and literature search relating to their research topics. Overall, during the first two years of the project, the work carried out has been in line with the Annex 1 to the Grant Agreement. The progress achieved in the training of ESRs and in the research activity is satisfactory.The project started on 1/1/2019, the focus in the first year of the project was onto the recruitment of the ESRs, this included the recruitment preparation, the vacancy publications, the interviews and the actual recruitment, contract discussion and signature. All activities have been developed according to the initial plan, except for the time for recruitment of ESR8, with a delayed contract start at 13 January 2020. Due to the start of the COVID-19 pandemic in Europe from February 2020 onwards most secondments could not be implemented in the normal way as a physical secondment, they were replaced by virtual secondments, other secondments have been postponed in case they were not yet time critical in the ESR project. Additionally also due to COVID-19 the planned network-wide training events (summer schools) have not been implemented yet.A summary of the activities in the WPs is provided here:WP1 - Conceptualization: The ESRs worked together to write a literature reviews, and also propose initial conceptualisations and hypotheses for their projects. Initially each ESR wrote the relevant section of the deliverable him/herself, and then the deliverable as a whole was discussed and (where appropriate) updated by the ESRs. WP2 - Development & customization: In WP2 the data collection using heterogeneous sources is executed, and additionally the data analysis and algorithms, formulation of results, as well as the result validation. The original plan was not to integrate the developed technologies into a common prototype platform or unified robotic demonstrator as a part of the project because that would have been a very large, and risky initiative also considering restrictions of physical presence due to the on-going COVID-19 epidemic. However, we plan to use virtual reality and simulation tools, used already by some of the ESRs, to integrate the technologies, e.g. robotic navigation and situated conversational interfaces, into a VR demonstrator that can be used in small-scale end-user studies. WP3 - Evaluation & definition of future scenario. In WP3 the ESRs will identify sustainable and effective innovation priorities in AI-assisted human machine interfaces, and formulate strategies and actions to enact them, focussing to the EU market context (including further potential sectors where to leverage expertize on the proposed technology). WP4 - Interdisciplinary enhancement. The research activity is supported by a set of Interdisciplinary actions (WP4), like training events and meetings. For the first year, actions related to the interdisciplinary enhancement have been taken by the project beneficiaries. There has been an enhancement approach that has been carried out by all the ESRs and an interdisciplinary workshop where all the ESRs participated. The details of each of them is given in section 1.2.4.WP5 - Training. The ESRs have presented their project to the 1st Workshop On Smart Personal Health Interfaces (Smartphil) which has been organized by the consortium on March 17-20, 2020 (Virtually due to COVID-19).WP6 - Dissemination, Communication & engagement. For WP6 the main communication channels, the webpage and the twitter account were created and Dissemination and Communication Plan was submitted. Progress beyond the state of the art and expected potential impact (including the socio-economic impact and the wider societal implications of the project so far) Despite of the problems with the corona epidemic we have seen good progress in all ESR projects. In fact, the increase in online conferences has saved a lot of time and made it possible to participate to more events. In the ESR1 project on insight mining we have been able to develop and propose a generic framework and several variations from conventional data mining and text generation methods and demonstrated the results in conferences. The work in ESR2 project on therapeutic conversational systems the research on assessment of empathy, and generation of empathetic responses by a dialog system has been demonstrated in first publications and real-time demonstrations. One of the contributions of this activity is an annotated corpus therapeutic conversations that will be made publicly available for the research community. The ESR3 project on deep program induction is advancing the science in automatic health program control and has produced several publications and open-source software packages. The ESR 4 work has focused on integration of knowledge models into clinical text classification using the recently introduced K-BERT language model. The first results are promising. The work of ESR5 in first person vision has produced first publications and software. The current plan is to test the solution in the HomeLab environment at Philips Research. One of the most important results of the ESR6 work on scene understanding is an extensive literature survey paper, with more than 330 citations, which has been accepted for publication in a prestigious journal in the robotics and automation domain.ESR7 project on personalized health communication has led to publications and a concrete demonstration of the envisioned solution as a conversational interface for nutritional tracking and counseling.The ESR8 project produced an initial literature survey but unfortunately the project has been terminated and we are currently looking for a new ESR to the role. img-20190129-154419-small.jpg
