Skip to main content

Multimedia Authoring and Management using your Eyes and Mind

Deliverables

D3.2 Multi-modal interaction with meso and high level control paradigms

Prototypes of multi-modal algorithms for meso-level and high-level control combining eye-tracking, EEG and bio-measurements. These algorithms will also become part of the Interaction SDK.

D2.3 - Final integration and optimization of multi-modal sensors

Delivery of the final version of MAMEM’s sensor installation that compared to D2.2 will be further fine-tuned and adapted to the user’s specific disabilities, incorporating the results of the first pilot studies conducted in WP6.

D4.3 – Initial implementation of MAMEM’s middleware and interaction SDK

Delivery of the software (together with its documentation) implementing the first version of MAMEM’s middleware and interaction SDK that incorporates the first-phase results of WP2 and WP3.

D2.2 - Initial integration and optimization of multi-modal sensors

Delivery of the initial version of MAMEM’s sensor installation, integrating and optimizing the three sensor modalities (eye-tracker, EEG recording system and bio-measurements).

D4.4 – Final implementation of MAMEM’s middleware and interaction SDK

Delivery of the software (together with its documentation) implementing the final version of MAMEM’s middleware and interaction SDK that incorporates the second-phase results of WP2 and WP3. The technical validation and verification tests of the updated algorithms are also included in this deliverable.

D3.3 Final implementation for the control paradigms of multi-modal interaction

Delivery of the final version of the MAMEM’s algorithms for multi-modal interaction that compared to D3.2 will also incorporate the results of the first-stage experiments conducted in WP6.

D5.2 – Initial design and implementation of the prototype interface applications

This deliverable will incorporate the mock-ups for the design, as well as an initial implementation for the prototype interface applications that will be used during the pilot studies. The prototypes of interface applications will be evaluated on effectiveness (lab studies in changing relevant user behavior and/ or attitudes). This deliverable will also encompass a description of the training cycles (of interface use) necessary for optimizing user acceptation and behaviour change.

D5.3 – Final design and implementation of the prototype interface applications

This deliverable will incorporate the final implementation of the prototype interface applications for managing and authoring multimedia content through the user’s eyes and mind. Compared to D5.2 the emphasis will be more on the development side and the integration with existing tools for managing, creating and sharing multimedia content. Moreover, the results of the first stage pilot trials will be also taken into consideration for improving the interface design.

D8.1 – Project Communication Kit

Initial project publicity material consisting of the project web-site, poster, leaflet and factsheet, as well as the project’s social media accounts (i.e. Twitter, LinkedIn and Facebook).

D1.4 - Data management plan

This deliverable will determine the strategy by which the research data generated by the project will be made open for maximizing their re-use.

D5.1b Update of D5.1 based on the revised requirements of the first-phase pilot trials

This is an update of the D5.1 based on the revised requirements that will result from the first-phase pilot trials.

D5.1 – Report on persuasive design principles, user models and profiles

This deliverable will incorporate a description of the most effective set of principles to derive a specific motivator for the behaviour or attitude change targets, incorporating social inclusion, increasing trust, and diminishing reactance effects. It will also describe MAMEM’s end-users on relevant characteristics as (dis-)abilities, interaction behavior, emotions, intentions, social abilities, extend to which training is needed, and social network, and sensitivity to persuasive strategies. The profiles for the different personas will be also part of this deliverable.

D6.2: Definition of pilot trials with the participation of patients

The report will describe detailed methods for conducting pilot studies with the participation of patients.

D7.1 – Methodology for measuring social integration

This deliverable will report on the results of our literature review about the existing methodologies for quantifying and monitoring social integration. Moreover, by employing the selected methodology for the cases of MAMEM this deliverable will also incorporate the complete portfolio of indicators for assessing the achieved impact on social integration.

D8.5 – Exploitation status report and updated plan

Report summarizing the status of the exploitation activities, the resulting paths for generating revenue and the activities that will be carried out to make the project results sustainable after its completion.

D4.1 Report on the middleware architecture and technical requirements

Description of the designed architecture and specifications including the technologies, the programming environment and the range of functionalities that will be adopted in the implementation of MAMEM’s middleware and SDK.

D6.1: Clinical requirements for the MAMEM platform for each of the patient cohort

The report will describe: (1) The special physical requirements for actual wearing and operation of the platform for each cohort of subjects; (2) Definitions of user interface software requirements, such as menus, screen buttons etc. (3) analyses of questionnaires and focus groups. This report will also include the ethical approvals obtained from the corresponding committees in Greece and Israel.

D6.5: Final report on pilot trials with the participation of patients

The report will describe: (1) Three sets of comprehensive results and analysis of experiments conducted in three clinical sites (2) Recommendations on future clinical use of the platform in terms of additional cohorts of subjects, additional disabilities and additional targets of use.

D7.3 – Study about the connection of natural interfaces and multimedia with the social integration of people with disabilities

This deliverable will accumulate and critically analyze the evidence that will be collected during the course of the project. Analysis will be performed with respect to bringing disabled people back to the society by means of managing and authoring multimedia content through natural human-computer interfaces.

D6.4: Interim report on pilot experiments based on half the subjects, including updated clinical requirements and definitions of clinical trials

The report will describe: (1) Three sets of results and analyses of experiments conducted in three clinical sites (2) Recommendations for modifications of methods for the second set of pilot studies with the participation of patients (3) recommendations for trials related to WP7 (Monitoring social inclusion).

D8.4 – Impact assessment and updated dissemination plan

Report summarizing the project dissemination activities, assessing their impact and suggesting potential corrections to the adopted dissemination strategy.

Searching for OpenAIRE data...

Publications

Assessing the usability of gaze-adapted Interface against conventional eye-based input emulation

Author(s): Kumar, Chandan; Menges, Raphael; Staab, Steffen
Published in: Issue 1, 2017
DOI: 10.5281/zenodo.583405

Chromium based Framework to include Gaze Interaction in Web Browser

Author(s): Kumar, Chandan; Menges, Raphael; Müller, Daniel; Staab, Steffen
Published in: Issue 1, 2017
DOI: 10.5281/zenodo.583340

Using tailoring to increase the effectiveness of a persuasive game-based training for novel technologies

Author(s): Fountoukidou, S.; Ham, J.R.C.; Midden, C.J.H.; Matzat, U.
Published in: CEUR Workshop Proceedings, 1833, 91 - 96, Issue 1, 2017
DOI: 10.5281/zenodo.583436

Analyzing the Impact of Cognitive Load in Evaluating Gaze-Based Typing

Author(s): Korok Sengupta, Jun Sun, Raphael Menges, Chandan Kumar, Steffen Staab
Published in: 2017 IEEE 30th International Symposium on Computer-Based Medical Systems (CBMS), 2017, Page(s) 787-792
DOI: 10.1109/CBMS.2017.134

Enhanced representation of web pages for usability analysis with eye tracking

Author(s): Raphael Menges, Hanadi Tamimi, Chandan Kumar, Tina Walber, Christoph Schaefer, Steffen Staab
Published in: Proceedings of the 2018 ACM Symposium on Eye Tracking Research & Applications - ETRA '18, 2018, Page(s) 1-9
DOI: 10.1145/3204493.3204535

eyeGUI - A Novel Framework for Eye-Controlled User Interfaces

Author(s): Raphael Menges, Chandan Kumar, Korok Sengupta, Steffen Staab
Published in: Proceedings of the 9th Nordic Conference on Human-Computer Interaction - NordiCHI '16, 2016, Page(s) 1-6
DOI: 10.1145/2971485.2996756

An Error Aware SSVEP-based BCI

Author(s): Kalaganis, Fotis; Chatzilari, Elisavet; Georgiadis, Kostas; Nikolopoulos, Spiros; Laskaris, Nikos; Kompatsiaris, Yiannis
Published in: IEEE 30th International Symposium on Computer-Based Medical Systems (CBMS), Issue 2, 2017, ISSN 2372-9198
DOI: 10.5281/zenodo.583386

GazeTheKey - Interactive Keys to Integrate Word Predictions for Gaze-based Text Entry

Author(s): Korok Sengupta, Raphael Menges, Chandan Kumar, Steffen Staab
Published in: Proceedings of the 22nd International Conference on Intelligent User Interfaces Companion - IUI '17 Companion, 2017, Page(s) 121-124
DOI: 10.1145/3030024.3038259

Detection of Mental Task Related Activity in NIRS-BCI systems Using Dirichlet Energy over Graphs

Author(s): Petrantonakis, Panagiotis; Kompatsiaris, Ioannis
Published in: 40th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC'18, Issue 1, 2018
DOI: 10.5281/zenodo.1294001

Usability Heuristics for Eye-controlled User Interface

Author(s): Sengupta, Korok; Kumar, Chandan; Staab, Steffen
Published in: COGAIN Symposium: Communication by Gaze Interaction, Issue 2, 2017
DOI: 10.5281/zenodo.820917

Hands-Free Web Browsing: Enriching the User Experience with Gaze and Voice Modality

Author(s): Sengupta, Korok; Ke, Min; Menges, Raphael; Kumar, Chandan; Staab, Steffen
Published in: ACM Symposium on Eye Tracking Research and Applications (ETRA 18), Issue 1, 2018
DOI: 10.5281/zenodo.1293980

Steady state visual evoked potential detection using Subclass Marginal Fisher Analysis

Author(s): Anastasios Maronidis, Vangelis P. Oikonomou, Spiros Nikolopoulos, Ioannis Kompatsiaris
Published in: 2017 8th International IEEE/EMBS Conference on Neural Engineering (NER), 2017, Page(s) 37-41
DOI: 10.1109/NER.2017.8008286

Sparse Kernel Machines for motor imagery EEG classification

Author(s): Oikonomou, Vangelis; Nikolopoulos, Spiros; Petrantonakis, Panagiotis; Kompatsiarios, Ioannis
Published in: 40th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC'18), Issue 1, 2018
DOI: 10.5281/zenodo.1294000

Sparse Bayesian Learning for Multiclass Classification with application to SSVEP- BCI

Author(s): Oikonomou, Vangelis P.; Liaros, Giorgos; Nikolopoulos, Spiros; Kompatsiaris, Ioannis
Published in: 8th International IEEE/EMBS Conference on Neural Engineering (NER), Issue 2, 2017, ISSN 1948-3554
DOI: 10.5281/zenodo.583431

A comparison study on EEG signal processing techniques using motor imagery EEG data

Author(s): Oikonomou, Vangelis P.; Georgiadis, Kostas; Liaros, George; Nikolopoulos, Spiros; Kompatsiaris, Ioannis
Published in: IEEE 30th International Symposium on Computer-Based Medical Systems (CBMS), Issue 2, 2017, ISSN 2372-9198
DOI: 10.5281/zenodo.583387

GazeTheWeb - A Gaze-Controlled Web Browser

Author(s): Raphael Menges, Chandan Kumar, Daniel Müller, Korok Sengupta
Published in: Proceedings of the 14th Web for All Conference on The Future of Accessible Work - W4A '17, 2017, Page(s) 1-2
DOI: 10.1145/3058555.3058582

Sparse Bayesian Learning for subject independent classification with application to SSVEP-BCI

Author(s): Vangelis P. Oikonomou, Anastasios Maronidis, George Liaros, Spiros Nikolopoulos, Ioannis Kompatsiaris
Published in: 2017 8th International IEEE/EMBS Conference on Neural Engineering (NER), 2017, Page(s) 600-604
DOI: 10.1109/NER.2017.8008423

Combining the Benefits of CCA and SVMs for SSVEP-based BCIs in Real-world Conditions

Author(s): Elisavet Chatzilari, Georgios Liarios, Kostas Georgiadis, Spiros Nikolopoulos, Yiannis Kompatsiaris
Published in: Proceedings of the 2nd International Workshop on Multimedia for Personal Health and Health Care - MMHealth '17, 2017, Page(s) 3-10
DOI: 10.1145/3132635.3132636

A Collaborative Representation Approach to Detecting Error-Related Potentials in SSVEP-BCIs

Author(s): Fotis P. Kalaganis, Elisavet Chatzilari, Spiros Nikolopoulos, Nikos A. Laskaris, Yiannis Kompatsiaris
Published in: Proceedings of the on Thematic Workshops of ACM Multimedia 2017 - Thematic Workshops '17, 2017, Page(s) 262-270
DOI: 10.1145/3126686.3129334

Computer uses and difficulties in Parkinson’s disease

Author(s): Z. Katsarou, M. Plotnik, G. Zeilig, A. Gottlieb, R. Kizony, S. Bostantjopoulou
Published in: The MDS 20th International Congress of Parkinson's Disease and Movement Disorders, 2016

Computer use aspects in patients with motor disabilities

Author(s): Bostantjopoulou, Sevasti; Plotnick, Meir; Zeilig, Gabi; Gottlieb, Amihai; Kizony, Amihai; Chlomissiou, Sissy; Nichogiannopoulou, Ariana; Katsarou, Zoe
Published in: 2nd Congress of the European Academy of Neurology (EAN'2016), 2016

MAMEM – A novel computer brain interface platform for enhancing social interaction of people with disabilities – Clinical requirements resulting from focus groups and literature survey

Author(s): Zeilig, Gabi; Gottlieb, Amihai; Kizony, Rachel; Katsarou, Zoe; Bostantzopoulou, Sevasti; Nichgiannopoulou, Ariana; Chlomissiou, Sissy; Plotnik, Meir
Published in: 20th European Congress of Physical and Rehabilitation Medicine, 2016

Parkinson’s disease impact on computer use. A patients’ and caregivers perspective

Author(s): Zoe Katsarou, Gabi Zeilig, Meir Plotnik, Amihai Gotlieb, Rachael Kizony, Sevasti Bostantjopoulou-Kambouroglou
Published in: The American Academy of Neurology 69th Annual Meeting, 2017

Clinical Research Example – MAMEM – Multimedia Authoring and Management Using Your Eyes and Mind

Author(s): Dr. Meir PLOTNIK, Mr. Amihai GOTTLIEB, Dr. Rachel KIZONY, Dr. Zoe KATSAROU, Dr. Sevasti BOSTANTZOPOULOU, Ms. Ariana NICHOGIANNOPOULOU, Ms. Sissy CHLOMISSIOU, Dr. Gabi ZEILIG,
Published in: Rehab science and technology update (RSTU), 2016

The importance of using computers in populations with Parkinson’s disease and spinal cord injury: a patients’ and caregivers’ perspective

Author(s): Meir Plotnik, Zoe Katsarou, Amihai Gotlieb, Adam Grinberg, Rachel Kizony, Gabi Zeilig, Sevasti Bostantjopoulou-Kambouroglou
Published in: SfN’s 47th annual meeting in Neuroscience, 2017

Finding Kairos: The Influence of Context-Based Timing on Compliance with Well-Being Triggers

Author(s): Jaap Ham, Jef van Schendel, Saskia Koldijk, Evangelia Demerouti
Published in: Symbiotic 2016, 2017, Page(s) 89-101
DOI: 10.1007/978-3-319-57753-1_8

Using personalized persuasive strategies to increase acceptance and use of HCI technology

Author(s): Sofia Fountoukidou, Jaap Ham, Peter Ruijten, and Uwe Matzat
Published in: Adjunct Proceedings of the 11th International Conference on Persuasive Technology, 2016

Using an Artificial Agent as a Behavior Model to Promote Assistive Technology Acceptance

Author(s): Sofia Fountoukidou, Jaap Ham, Uwe Matzat, Cees Midden
Published in: PERSUASIVE 2018, 2018, Page(s) 285-296
DOI: 10.1007/978-3-319-78978-1_24

Schau genau! A Gaze-Controlled 3D Game for Entertainment and Education

Author(s): Menges, Raphael and Kumar, Chandan and Wechselberger, Ulrich and Schaefer, Christoph and Walber, Tina and Staab, Steffen
Published in: COGAIN Symposium, 2017

Ambiguous Agents: The Influence of Consistency of an Artificial Agent’s Social Cues on Emotion Recognition, Recall, and Persuasiveness

Author(s): Peter A. M. Ruijten, Cees J. H. Midden, Jaap Ham
Published in: International Journal of Human–Computer Interaction, Issue 32/9, 2016, Page(s) 734-744, ISSN 1044-7318
DOI: 10.1080/10447318.2016.1193350

EEG-Based Brain–Computer Interfaces for Communication and Rehabilitation of People with Motor Impairment: A Novel Approach of the 21st Century

Author(s): Ioulietta Lazarou, Spiros Nikolopoulos, Panagiotis C. Petrantonakis, Ioannis Kompatsiaris, Magda Tsolaki
Published in: Frontiers in Human Neuroscience, Issue 12, 2018, ISSN 1662-5161
DOI: 10.3389/fnhum.2018.00014

A multimodal dataset for authoring and editing multimedia content: The MAMEM project

Author(s): Spiros Nikolopoulos, Panagiotis C. Petrantonakis, Kostas Georgiadis, Fotis Kalaganis, Georgios Liaros, Ioulietta Lazarou, Katerina Adam, Anastasios Papazoglou-Chalikias, Elisavet Chatzilari, Vangelis P. Oikonomou, Chandan Kumar, Raphael Menges, Steffen Staab, Daniel Müller, Korok Sengupta, Sevasti Bostantjopoulou, Zoe Katsarou, Gabi Zeilig, Meir Plotnik, Amihai Gotlieb, Racheli Kizoni, Sofia Foun
Published in: Data in Brief, Issue 15, 2017, Page(s) 1048-1056, ISSN 2352-3409
DOI: 10.1016/j.dib.2017.10.072

Eye-Controlled Interfaces for Multimedia Interaction

Author(s): Chandan Kumar, Raphael Menges, Steffen Staab
Published in: IEEE MultiMedia, Issue 23/4, 2016, Page(s) 6-13, ISSN 1070-986X
DOI: 10.1109/MMUL.2016.52

The influence of social cues in persuasive social robots on psychological reactance and compliance

Author(s): Ghazali, Aimi Shazwani; Ham, Jaap; Barakova, Emilia; Markopoulos, Panos
Published in: International Journal of Human–Computer Interaction, Issue 1, 2018, ISSN 1044-7318
DOI: 10.5281/zenodo.1294005

Discriminative codewaves: a symbolic dynamics approach to SSVEP recognition for asynchronous BCI

Author(s): K Georgiadis, N Laskaris, S Nikolopoulos, I Kompatsiaris
Published in: Journal of Neural Engineering, Issue 15/2, 2018, Page(s) 026008, ISSN 1741-2560
DOI: 10.1088/1741-2552/aa904c

Effects of Robot Facial Characteristics and Gender in Persuasive Human-Robot Interaction

Author(s): Aimi S. Ghazali, Jaap Ham, Emilia I. Barakova, Panos Markopoulos
Published in: Frontiers in Robotics and AI, Issue 5, 2018, ISSN 2296-9144
DOI: 10.3389/frobt.2018.00073

An error-aware gaze-based keyboard by means of a hybrid BCI system

Author(s): Fotis P. Kalaganis, Elisavet Chatzilari, Spiros Nikolopoulos, Ioannis Kompatsiaris, Nikos A. Laskaris
Published in: Scientific Reports, Issue 8/1, 2018, ISSN 2045-2322
DOI: 10.1038/s41598-018-31425-2

Single-Trial NIRS Data Classification for Brain–Computer Interfaces Using Graph Signal Processing

Author(s): Panagiotis C. Petrantonakis, Ioannis Kompatsiaris
Published in: IEEE Transactions on Neural Systems and Rehabilitation Engineering, Issue 26/9, 2018, Page(s) 1700-1709, ISSN 1534-4320
DOI: 10.1109/TNSRE.2018.2860629

Comparative evaluation of state-of-the-art algorithms for SSVEP-based BCIs

Author(s): Oikonomou, Vangelis P.; Liaros, Georgios; Georgiadis, Kostantinos; Chatzilari, Elisavet; Adam, Katerina; Nikolopoulos, Spiros; Kompatsiaris, Ioannis
Published in: Issue 1, 2016

The MAMEM Project - A dataset for multimodal human-computer interaction using biosignals and eye tracking information

Author(s): Nikolopoulos,Spiros; Georgiadis, Kostas; Kalaganis, Fotis; Liaros, Georgios; Lazarou, Ioulietta; Adam, Katerina; Papazoglou-Chalikias Anastasios; Chatzilari, Elisavet; Oikonomou, P. Vangelis; Petrantonakis, C. Panagiotis; Kompatsiaris, I.; Kumar, Chandan; Menges, Raphael; Staab, Steffen; Müller, Daniel; Sengupta, Korok; Bostantjopoulou, Sevasti; Katsarou, Zoe; Zeilig, Gabi; Plotnin, Meir; Gottlie
Published in: 2017
DOI: 10.5281/zenodo.834154