Skip to main content

Smart, User-friendly, Interactive, Tactual, Cognition-Enhancer that Yields Extended Sensosphere - Appropriating sensor technologies, machine learning, gamification and smart haptic interfaces

Deliverables

Driving and control units for the textile III

Based on work in T6.1 electromechanical driving, actuating and control units will be done in iterations generating more mature technical solutions along while the project is proceeding.

Prototypes for psychophysical studies with one touch dimension I

Based on Task 5.3 prototypes for T6.1 will be delivered having one significant touch, heat or olfactory dimension.

Driving and control units for the textile I

Based on work in T6.1 electromechanical driving, actuating and control units will be done in iterations generating more mature technical solutions along while the project is proceeding.

Prototypes for psychophysical studies with one touch dimensions II

Based on Task 5.3 prototypes for T6.1 will be delivered having one significant touch, heat or olfactory dimension.

Sensor array incorporating object recognition

A revised version of the initial sensor system incorporating a camera system for object recognition.

Prototypes for user studies

Based on Task 5.3 prototypes (null series) for user studies will be delivered having a realistic number of significant touch, heat or olfactory dimension and being of realistic robustness.

Demonstrators/Prototypes - iteration 3

Iteration 1 (M24- DEM) [Lead HSO]; Iteration 2 (M30 - DEM) [Lead HSO]; Iteration 3 (M36) [Lead HSO]. Working demonstrators /prototypes with multiple iterations.

Deliver the detailed dissemination plan

Deliver the detailed dissemination plan (M4, and continues updates) - LDQR, HARPO, HSO - Deliver the basic guidelines for a clear and accessible communication of the project’s results to different publics, (HARPO) - Establish a list of stakeholders stating possible collaboration opportunities, (LDQR) - Make a selection of possible academic events to attend and coordinate with the other WP to apply and participate. (HSO)

Driving and control units for the textile II

Based on work in T6.1 electromechanical driving, actuating and control units will be done in iterations generating more mature technical solutions along while the project is proceeding.

Prototypes for psychophysical studies with two touch dimensions

Based on Task 5.4 prototypes for T6.2 will be delivered having two significant touch, heat or olfactory dimension.

Prototype HIPI

The final revision of the sensor system, such that the sensors are incorporated with the haptic interface and signals are provided to trigger haptic feedback as required.

Demonstrators/Prototypes - iteration 2

Iteration 1 (M24- DEM) [Lead HSO]; Iteration 2 (M30 - DEM) [Lead HSO]; Iteration 3 (M36) [Lead HSO]. Working demonstrators /prototypes with multiple iterations.

Initial sensor system

The initial sensor system will comprise an array of distance sensors, and potentially IMU and GPS locator, attached to a Raspberry Pi, and housed in such a way that the sensor system can be worn for testing purposes.

Demonstrators/Prototypes - iteration 1

Iteration 1 (M24- DEM) [Lead HSO]; Iteration 2 (M30 - DEM) [Lead HSO]; Iteration 3 (M36) [Lead HSO]. Working demonstrators /prototypes with multiple iterations.

Define the project Identity I

Define the project identity: Delivery of templates to be used by team members and an information package that will be delivered to all partners, which will be updated every 6 months including latest results and news (M3, M9, M15, M21, M27, M33) - LDQR

Define the project identity II

Define the project identity: Delivery of templates to be used by team members and an information package that will be delivered to all partners, which will be updated every 6 months including latest results and news (M3, M9, M15, M21, M27, M33) - LDQR

Define the project identity III

Define the project identity: Delivery of templates to be used by team members and an information package that will be delivered to all partners, which will be updated every 6 months including latest results and news (M3, M9, M15, M21, M27, M33) - LDQR

Define the project Identity VI

Define the project identity: Delivery of templates to be used by team members and an information package that will be delivered to all partners, which will be updated every 6 months including latest results and news (M3, M9, M15, M21, M27, M33) - LDQR

Project website

Create the project website with organized basic information of each WP.

Project publicity material

Creation of initial project publicity material to achieve early project awareness consisting of the posters, leaflet and factsheet, as well as the project’s social media accounts (i.e. Twitter, LinkedIn and Facebook) which will be active and constantly updated during the projects life-time.

Define the project identity V

Define the project identity: Delivery of templates to be used by team members and an information package that will be delivered to all partners, which will be updated every 6 months including latest results and news (M3, M9, M15, M21, M27, M33) - LDQR

Define the project identity IV

Define the project identity: Delivery of templates to be used by team members and an information package that will be delivered to all partners, which will be updated every 6 months including latest results and news (M3, M9, M15, M21, M27, M33) - LDQR

Requirements for HIPI

This will formalise the goals and constraints for the HIPI, specifying both functions that it should perform and constraints its design should obey to be acceptable to its intended users. These requirements will be prioritised, identifying those that are critical (and must be satisfied), and those that are desirable (and should be satisfied, but not necessarily). This will serve both as a guide for technical development in WP5, and as the basis for evaluating the final device in WP6.

Public Report

This report will summarise the overall project achievements, results and conclusions for a public consumption.

Dissemination activities report III

Dissemination activities report (M12, 24, 36) – HSO, LDQR, HARPO

Impact Measurement Methodology

Develop and report on an Impact Measurement Methodology

Report on psychophysical experiments with 2D haptic stimuli

Report on psychophysical experiments with sighted and hearing human participants on the discriminability of 2D haptic signals. This will provide input to T5.3 and T5.4.

Refined recommendations

Refined recommendations for gamified solutions and social interaction based on group-specific variables and preferences.

Report on design and implementation of participatory events

Report on design and implementation of participatory events (workshops with the stakeholders, training on the use of prototype, organisation of seminar/congress to share the project’s results and future perspectives).

Data Management Plan

Data Management Plan.

Dissemination activities report I

Dissemination activities report (M12, 24, 36) – HSO, LDQR, HARPO

Final exploitation plan and report on IPR issues

Final exploitation plan and report on IPR issues.

Report on initial psychophysical experiments II

Report on psychophysical experiments with sighted and hearing human participants on the usefulness of specific haptic signals. Evaluation of first prototypes. This will provide input to T5.1, T5.3 and T5.4.

Report on initial psychophysical experiments I

Report on psychophysical experiments with sighted and hearing human participants on the usefulness of specific haptic signals. Evaluation of first prototypes. This will provide input to T5.1, T5.3 and T5.4.

Comparative Policy Report

This will synthesise the policy findings from the participating consultants, contrasting them to identify areas of best practice that could be adopted; relating them to the views and experiences expressed in the user interviews from T2.2 to identify their impact, and to identify how they might support or restrict the implementation of the HIPI.

Report on psychophysical experiments with multi-dimensional haptic stimuli

Report on psychophysical experiments with sighted and hearing human participants on the discriminability and identification of multi-dimensional haptic signals. Evaluation of prototypes.

Report on technologies used for existing (hard) solutions for haptic communication

Based on work in T5.1 and T6.1 the other tasks in WP5 will be informed on the choices for technologies for textile-based haptic devices.

Initial exploitation plan and report on IPR issues

Initial exploitation plan and report on IPR issues.

1st version of face and object recognition algorithms, dimensionality reduction algorithms, ontologies and semantic reasoning

This deliverable will report on: (a) The basic version of face and object recognition algorithms evaluated in benchmark datasets; (b) v1 of the algorithms that achieve discrete low dimensional representations for different high dimensional concepts evaluated in benchmark datasets; (c) Preliminary report on the semantic models for the mapping of communication’s conceptualisations on ontology-related constructs. In addition, it will describe the functionality of the first version of interpretation and reasoning techniques.

Report on textiles for haptic communication

Based onT5.6summarizing the benefits and obstacles using textiles for haptic communication.

Industrial production protocols

Based on T5.7 industrial production protocol, patterns etc. will be summarized into a report. A 3D scan will be produced and a garment based on this made.

2nd version of face and object recognition algorithms, dimensionality reduction algorithms, ontologies and semantic reasoning

This deliverable will report on: (a) An advanced version of the visual analysis components presented in D3.1; (b) v2 of the algorithms that achieve discrete low dimensional representations for different high-dimensional concepts and lead to perceptible haptic information; (c) Advanced version of semantic models adapted or extended according to the project’s requirements for semantic annotation and reasoning and the advanced functionality of techniques.

Best practice recommendations for engaging deafblind participants in research

This will summarise the key findings of the literature review carried out in T2.1., and a summary of the experiences of conducting the interviews in T2.2 to provide advice for future researchers on conducting research with deafblind participants.

Dissemination activities report II

Dissemination activities report (M12, 24, 36) – HSO, LDQR, HARPO

Recommendations

A set of recommendations for gamified solutions and social interaction based on the state of the art.

Documentation of agreed concepts

Documentation of agreed concepts.

Final version of face and object recognition algorithms, dimensionality reduction algorithms, ontologies and semantic reasoning

This deliverable will report on the final version and refinements of the algorithms presented in D3.1 and D3.2.

Report on psychophysical experiments on navigation

Report on psychophysical experiments with sighted and hearing human participants on stimulation for navigation, and final evaluation of suitability of intelligent haptic interface for navigation purposes.

Personas, environments and use scenarios

This will provide a tool to guide the technical aspects of the project during its development, to provide a practical context for assessing impact on users without the need for placing constant demands on their time. It will also be made more widely available at the end of the project, to support inclusive design for deafblind people in other projects.

Publications

Spatial Awareness for the Deafblind in Natural Language Presentation using SPIN Rules: A Use Case in the SUITCEYES Platform

Author(s): asileios Kassiano, Thanos G. Stavropoulos, Spiros Nikolopoulos, Ioannis Kompatsiaris, Marina Riga
Published in: eTELEMED 2020 : The Twelfth International Conference on eHealth, Telemedicine, and Social Medicine, 2020, ISBN 978-1-61208-763-4
Publisher: IARIA

Static and Dynamic Haptograms to Communicate Semantic Content

Author(s): Darányi, Sándor; Olson, Nasrine; Riga, Marina; Kontopoulos, Efstratios; Kompatsiaris, Ioannis
Published in: SEMAPRO 2019, The Thirteenth International Conference on Advances in Semantic Processing, 2019, Page(s) 16-20, ISBN 978-1-61208-738-2
Publisher: IARIA
DOI: 10.5281/zenodo.3491505

Emulating Social Haptic Communication with Vibration patterns

Author(s): Myrthe A. Plaisier and Astrid M.L. Kappers
Published in: World Haptics Conference 2021, 2021
Publisher: IEEE Xplore

A Dimensionality Reduction Method for Data Visualization using Particle Swarm Optimization

Author(s): Panagiotis Petrantonakis, Ioannis Kompatsiaris
Published in: Proceedings of the 12th International Joint Conference on Computational Intelligence, 2020, Page(s) 131-138, ISBN 978-989-758-475-6
Publisher: SCITEPRESS - Science and Technology Publications
DOI: 10.5220/0010020601310138

Keep Your Distance: A Playful Haptic Navigation Wearable for Individuals with Deafblindness

Author(s): James Gay, Moritz Umfahrer, Arthur Theil, Lea Buchweitz, Eva Lindell, Li Guo, Nils-Krister Persson, Oliver Korn
Published in: The 22nd International ACM SIGACCESS Conference on Computers and Accessibility, 2020, Page(s) 1-4, ISBN 9781450371032
Publisher: ACM
DOI: 10.1145/3373625.3418048

Co-Designing Assistive Tools to Support Social Interactions by Individuals Living with Deafblindness

Author(s): Arthur Theil, Lea Buchweitz, Mauricio Fuentes, Oliver Korn
Published in: Companion Publication of the 2020 ACM Designing Interactive Systems Conference, 2020, Page(s) 79-83, ISBN 9781450379878
Publisher: ACM
DOI: 10.1145/3393914.3395869

Exploring Low-Cost Materials to Make Pattern-Based Lock-Screens Accessible for Users with Visual Impairments or Deafblindness

Author(s): Lea Buchweitz, Arthur Theil, James Gay, Oliver Korn
Published in: The 22nd International ACM SIGACCESS Conference on Computers and Accessibility, 2020, Page(s) 1-4, ISBN 9781450371032
Publisher: ACM
DOI: 10.1145/3373625.3418020

Regulating Personal Cameras for Disabled People and People with Deafblindness: Implications for HCI and Accessible Computing

Author(s): Sarah Woodin and Arthur Theil
Published in: 23rd International ACM SIGACCESS Conference on Computers and Accessibility (ASSETS '21), October 18--22, 2021, 2021
Publisher: ACM

Information worlds of people with deafblindness

Author(s): Nasrine Olsson, Elena Maceviciute
Published in: Proceedings of ISIC: the information behaviour conference Pretoria, South Africa, 28th September to 1st October, 2020, vol. 25 no. 4, December, 2020, 2020
Publisher: University of Borås,
DOI: 10.47989/irisic2012

Empowering Persons with Deafblindness - Designing an Intelligent Assistive Wearable in the SUITCEYES Project

Author(s): Oliver Korn, Raymond Holt, Efstratios Kontopoulos, Astrid M.L. Kappers, Nils-Krister Persson, Nasrine Olson
Published in: Proceedings of the 11th PErvasive Technologies Related to Assistive Environments Conference on - PETRA '18, 2018, Page(s) 545-551, ISBN 9781-450363907
Publisher: ACM Press
DOI: 10.1145/3197768.3201541

Activity Recognition from Wearable Cameras

Author(s): Panagiotis Giannakeris, Konstantinos Avgerinakis, Stefanos Vrochidis, Ioannis Kompatsiaris
Published in: 2018 International Conference on Content-Based Multimedia Indexing (CBMI), 2018, Page(s) 1-6, ISBN 978-1-5386-7021-7
Publisher: IEEE
DOI: 10.1109/cbmi.2018.8516553

Sensor technology, gamification, haptic interfaces in an assistive wearable

Author(s): Olson, N., Urbański, J., Persson, N.-K., Starosta-Sztuczka, J., Fuentes, M.
Published in: The Journal on Technology and Persons with Disabilities, Volume 7, 2019, Page(s) 79-87
Publisher: LIBRARY OF CONGRESS * U.S. ISSN CENTER ISSN Publisher Liaison Section

Tactile Board: A Multimodal Augmentative and Alternative Communication Device for Individuals with Deafblindness

Author(s): Arthur Theil, Lea Buchweitz, James Gay, Eva Lindell, Li Guo, Nils-Krister Persson, Oliver Korn
Published in: 19th International Conference on Mobile and Ubiquitous Multimedia, 2020, Page(s) 223-228, ISBN 9781450388702
Publisher: ACM
DOI: 10.1145/3428361.3428465

Learning the Vibrotactile Morse Code Alphabet

Author(s): Myrthe A. Plaisier, Daphne S. Vermeer, Astrid M. L. Kappers
Published in: ACM Transactions on Applied Perception, 17/3, 2020, Page(s) 1-10, ISSN 1544-3558
Publisher: Association for Computing Machinary, Inc.
DOI: 10.1145/3402935

Hands-free Devices for Displaying Speech and Language in the Tactile Modality - Methods and Approaches

Author(s): Astrid M.L. Kappers, Myrthe Plaisier
Published in: IEEE Transactions on Haptics, 2021, Page(s) 1-1, ISSN 1939-1412
Publisher: Institute of Electrical and Electronics Engineers
DOI: 10.1109/toh.2021.3051737

First-person activity recognition from micro-action representations using convolutional neural networks and object flow histograms

Author(s): Panagiotis Giannakeris, Panagiotis C. Petrantonakis, Konstantinos Avgerinakis, Stefanos Vrochidis, Ioannis Kompatsiaris
Published in: Multimedia Tools and Applications, 80/15, 2021, Page(s) 22487-22507, ISSN 1380-7501
Publisher: Kluwer Academic Publishers
DOI: 10.1007/s11042-020-09902-6

Representing Numerosity Through Vibration Patterns

Author(s): Myrthe A. Plaisier, Raymond J. Holt, Astrid M.L. Kappers
Published in: IEEE Transactions on Haptics, 13/4, 2020, Page(s) 691-698, ISSN 1939-1412
Publisher: Institute of Electrical and Electronics Engineers
DOI: 10.1109/toh.2020.2988211

On the Talent vs. Luck-Based Evaluation of the Classification Process

Author(s): Panagiotis C. Petrantonakis, Ioannis Kompatsiaris
Published in: IEEE Access, 7, 2019, Page(s) 37565-37574, ISSN 2169-3536
Publisher: Institute of Electrical and Electronics Engineers Inc.
DOI: 10.1109/access.2019.2905049

Thermal Perception and Thermal Devices used on Body Parts other than Hand or Face

Author(s): Astrid Maria Louise Kappers, Myrthe Plaisier
Published in: IEEE Transactions on Haptics, 4/year, 2019, Page(s) 1-1, ISSN 1939-1412
Publisher: Institute of Electrical and Electronics Engineers
DOI: 10.1109/toh.2019.2925339

Perception of vibrotactile distance on the back

Author(s): Myrthe A. Plaisier, Lotte I. N. Sap, Astrid M. L. Kappers
Published in: Scientific Reports, 10/1, 2020, ISSN 2045-2322
Publisher: Nature Publishing Group
DOI: 10.1038/s41598-020-74835-x

Perception of Vibratory Direction on the Back

Author(s): Astrid M. L. Kappers, Jill Bay, Myrthe A. Plaisier
Published in: Haptics: Science, Technology, Applications - 12th International Conference, EuroHaptics 2020, Leiden, The Netherlands, September 6–9, 2020, Proceedings, 12272, 2020, Page(s) 113-121, ISBN 978-3-030-58146-6
Publisher: Springer International Publishing
DOI: 10.1007/978-3-030-58147-3_13

Physical Add-Ons for Haptic Human-Surrounding Interaction and Sensorial Augmentation

Author(s): Eva Lindell, Arthur Theil, Li Guo, Nasrine Olson, Oliver Korn, Nils-Krister Persson
Published in: Human Interaction, Emerging Technologies and Future Applications III - Proceedings of the 3rd International Conference on Human Interaction and Emerging Technologies: Future Applications (IHIET 2020), August 27-29, 2020, Paris, France, 1253, 2021, Page(s) 183-188, ISBN 978-3-030-55306-7
Publisher: Springer International Publishing
DOI: 10.1007/978-3-030-55307-4_28