Audio-Visual System for the Blind Allowing Visually Impaired to See Through Hearing

Project Information

EyeSynth

Grant agreement ID: 757202

DOI 10.3030/757202

Closed project

EC signature date 4 February 2017

Start date 1 February 2017

End date 31 January 2019

Funded under

• INDUSTRIAL LEADERSHIP - Leadership in enabling and industrial technologies - Information and Communication Technologies (ICT)

Total cost € 1 478 950,00

EU contribution € 1 035 265,00

Coordinated by

EYESYNTH SL
Spain

Periodic Reporting for period 2 - EyeSynth (Audio-Visual System for the Blind Allowing Visually Impaired to See Through Hearing)

Reporting period: 2018-02-01 to 2019-01-31

Summary of the context and overall objectives of the project

"There is a paradoxical circumstance in the technological world: there are plenty of resources and techniques available, but that potential is far from being fully exploited to assist blind people. Since the use of the white cane and the guide dog, there’s no technological standard for blind mobility.
Eyesynth is an audiovisual system for the blind. It consists of a pair of glasses connected to a microcomputer. The system records the surrounding environment in three dimensions. Then, the
collected data is converted into understandable audio for the blind, which allows them to identify the position of objects and obstacles, as well as their size and shape, with unprecedented accuracy.
The audio signals are non-verbal and abstract, so it is a universal product in terms of language.
Eyesynth is initially focused on the European market as they have received funding from the European Union’s Horizon 2020 Research and Innovation Program under a grant agreement. The patent process
around the globe is on the way, preparing the terrain for global deployment.

The main features of this technology are:

- Unparalleled levels of detail. The user can distinguish spaces, obstacles and objects, their shape and distance. With enough training you can distinguish small objects (such as coins) on a table.

- Easy to use. The Eyesynth system is tremendously easy to learn and use. It is very intuitive. After an hour of training, the user is able to use the system in an environment with large obstacles, such as chairs or tables. With a longer training (in terms of days) you can already distinguish much subtler nuances of shape, size and distance.

- It's universal. The sound system does not use words, but an abstract sound (very similar to the sound of the sea) that ""changes its shape"" and intensity according to what the cameras record. There is no language barrier.

- Cochlear sound. The audio system (integrated in the frame of the glasses) transmits the sound by vibrations through the cranial bone. It’s absolutely non-invasive, and It does not cause auditory fatigue.


"

Work performed from the beginning of the project to the end of the period covered by the report and main results achieved so far

We have done a lot of work toward creating our final prototype for it to be ready for mass manufacturing.
For example, we have redesigned the hardware following the Design For Manufacturing (DFM) criteria. We have also been working on the audio-visual algorithm optimization, ensuring the performance will be maxed out maintaining low levels of power consumption.
We have completed the Cloud platform, which is an information hub regarding device upgrading and technical assistance. Likewise, we have set up mobile apps which will add extra features to the Eyesynth system.
We have established initial agreements with key distributors in our target markets. We have focused on large organizations since we know that they are the most relevant channel for initial market traction.
Furthermore:
• We have performed a complete circuit redesign taking into account as main driver cost and reliability considerations
• We have performed a technology watch by updating a report with new findings in terms of relevant research work being performed by other companies or entities.
• We have completed a Web update with new creative material implementation, including a new convenient look & feel. All new content, text, videos on the website have been designed within the accessibility standards
• We have completed Deliverables concerning Ethics Requirements that our project must comply with.
• Ophthalmological Study: We’ve set up alongside Ophthalmologists of the Provincial Hospital of Castellón (Spain) a series of protocols, training and analysis software. The medical trials have proven the efficiency of our system with low vision patients. They’ve experimented a clear enhancement of spatial perception using Eyesynth. We’ll publish the full results soon.

Progress beyond the state of the art and expected potential impact (including the socio-economic impact and the wider societal implications of the project so far)

The combination of Eyesynth technologies is nourished by different fields: Machine Vision (artificial vision and robotics), software algorithms of image processing in three dimensions, sound synthesis in real time and specific electronic designs for high intensity calculation tasks. The video-audio conversion algorithm is our own and with intellectual registration granted. The set of hardware and software is in the process of obtaining an international patent.
Technological innovation
In addition to the amount of information that the sound of the system presents, it allows to identify shapes and spaces with a very high precision, allowing to distinguish, for example, bottles, glasses and dishes on a table. Another huge benefit is the speed and ease of learning. In the tests performed, the basic operation is assimilated in the first hour of use.
Eyesynth includes a demanded feature: an emergency button (with GPS location and emergency call) bringing notable improvements in day-to-day security.
The use of Eyesynth is positioned as a Premium experience, distancing itself from orthopaedic products.
Finally, a great advantage of Eyesynth is its capacity for expansion. Through software updates, new navigation functions will be implemented; recognition of texts, faces or bank notes, or new functions that our clients demand through the forum that we will implement.
Worldwide, visual impairment has slightly decreased since the 1990s, mainly due to the special emphasis and resources that governments and NGOs have implemented to prevent the disorder in developing regions, which were the main contributors to blind cases.
However, the situation in developed regions (Eyesynth´s target market due to purchasing power reasons), is different. We see that the future trend is an increase in the number of blind people because of 1) Aging population and 2) Increase in the prevalence of diabetes.
In the past 20 years, organizations such as the WHO or the International Agency for the Prevention of Blindness (IAPB) have emphasized the rising situation of blindness worldwide. This has contributed to a greater social awareness, prompting the implementation of preventative measures and actions to address specific problems or needs, for example the Eyesynth solution.

Glasses 1

glasses 2

Glasses 2