Skip to main content

Development of 3D digital glasses for enhancing mobility of visually impaired people to open strategic product lines for participant SMEs

Final Report Summary - DIGIGLASSES (Development of 3D digital glasses for enhancing mobility of visually impaired people to open strategic product lines for participant SMEs)

Executive Summary:
According to the European Blind Union an average of 1 in 30 Europeans experience sight loss. Partially sighted persons are 4 times more than blind people and altogether there are over 30 million blind and partially sighted persons in geographical Europe (nearly 10% of the total visually impaired population). One in three senior citizens over 65 struggles with sight loss as the likeliness of most important eye diseases grow progressively by age. Most cases their vision loss cannot be recovered by common eyeglasses. Furthermore, the population of Europe is getting older in parallel older generations would also like to live an active life it is a crucial task to increase the standard of living of these visually impaired people.

Currently available visual aids provide only limited solution for the visually impaired people, one tool may provide help in a specific case (for example reading a newspaper), but for a different situation another tool is required. Also there is no real solution to help people in outdoors scenarios, for example walking on the street which may seem very easy for a healthy person, but for an impaired person it is a great challenge. The aim of DIGIGLASSES project was to develop a digital tool for the visually impaired with existing but low vision, which will be able to provide stereoscopic vision for the users, corrected and customized for the special symptoms of the user’s eye disease.

The Consortium has developed a prototype digital glass, which includes two separate image capturing device (input device), which are connected to a data processing unit, where the specifically tailored image processing algorithms are enhancing the captured images in real time for each input device separately and these modified images are transferred to the corresponding electronic display devices. The image information of the display devices are transferred to the user’s eyes through a specially constructed optical interface for each eye separately. The whole construction is integrated into one frame and wore by the visually impaired user on the head. The user is receiving two separate image for both eyes enabling the brain to create stereoscopic information and enable depth perception, these images are specifically enhanced for the user’s visual impairment.

The device is suitable to help people with different eye diseases like: Myopia, Macular degeneration, Glaucoma, Cataract etc. It offers integrated solutions for several indoor and outdoor situations like: reading (newspaper, bank-automat display), walking inside/outside, zebra crossing and traffic light enhancement, determining the direction of a moving staircase, stair-case enhancement. All these functions are integrated into a single stylish device.

Project Context and Objectives:
In DIGIGLASSES project a group of SMEs aimed to develop a marketable digital tool for the visually impaired with existing but low vision, which will be able to provide stereoscopic vision for the users, corrected and customized for the special symptoms of the user’s eye disease. By this combined solution, not only the symptoms of the unique disease will be decreased, but the stereoscopic vision allows judging distances within the environment.

In conjunction with this aim, the project has developed several additional results which can be used in different applications. The hardware is a digital smart glass, which is current technology trend, and also several useful software application was developed for the visually impaired that can be applied to different platforms like mobile phones or different smart glasses.

Main objective

• To create a marketable stereoscopic vision aid device to enhance the mobility of the visually impaired people with existing, but low vision.

Societal objectives

• To increase the standard of living of the visually impaired people by increasing their mobility.
• To increase the employment rate of the visually impaired people in Europe, by providing them a device, that assists them to travel to work places safely and individually.

To answer the need for the development of a novel visual aid tool the scientific and technological objectives are summarized below. The scientific objectives are to support the technological development to achieve the defined measurable objectives.

Scientific objectives

• To gain in-depth understanding of the sight mechanism of a person suffering from visual diseases.
• To develop an image manipulation algorithm to improve the vision of a people suffering from glaucoma or from cataract.

Technical objectives

• Low level implementation of high speed image processing and manipulation algorithms specifically for DSP.
• To create a video capturing system working with suitable resolution and speed.
• To use digital image processing to modify the captured picture in order to give higher quality vision for visually impaired people.
• To develop a personal computer based control software to define unique picture modification algorithms to be able to give unique solution for the injured person.
• To design the display system that is able to transform images for a visually impaired person.
• To validate the usability of the prototypes.

Project Results:
The project was two years in duration. It started on the 1st of August 2012.
During the project a survey on user and market needs was performed, and Consortium partners conducted an in depth literature search to obtain information necessary for the DIGIGLASSES system specification. Based on the results, the functional system specification was outlined. The main requirement of the system was defined, the necessary functions to be implemented was agreed according to the need of the SME partners, and the feedback of the end users.
After the Consortium partners agreed on the main points of the system, the RTD partners continued investigations on the possible technical realization of the system. Hardware and Software considerations were made. The possible technologies for each subsystem were investigated and parts were selected. The required algorithms were investigated that allow implementing the functions envisioned.
Based on the system architecture, the Consortium started development of the electrical design of DIGIGLASSES. The design consist high complexity printed circuit boards, fitting in very small dimensions. All electronic components are integrated into the device balancing between demanding size and functionality requirements. The developers achieved the final design through several iterations and faced many difficulties, but finally achieved the working prototype.
Examination software was developed which enables the researchers to create the desired functions and allows the optimization and fine tuning of the given functions. The defined functions were realized with OpenCV algorithms and fine-tuned for the visually impaired in terms of usability. In parallel with this a hardware specific software environment was established on the target signal processing unit. The developed algorithms were than deployed to the system and the path of optimization possibilities were demonstrated with some of the algorithms through low level implementation. Also a configuration GUI was developed to enable fine tuning of the algorithms for the specific requirements of the end-user.
Researchers designed the optical part of the system which will enable the user to see the picture ahead of him in the proper distance. The selected optical system was than manufactured and optimized. The partners aimed for a mechanically small design that will enable the realisation of a stylish pair of glasses. Through an iterative development process, several design approaches were investigated and finally selected a compromise between the demanding size and style requirements, achieving an acceptable mechanical structure which was manufactured.
Consortium prepared the Data management principles of the system to make sure that all data collected during the End user validation tests are properly handled in line with the national and European rules. Submitted Ethical clearance petition to the responsible Hungarian authorities and received their approval enabling the final tests with the visually impaired people.
All the separate parts were integrated together and passed several laboratory tests before the researcher were able to wear the device. Before enabling the target group to try out the device, partners spent a lot of effort to establish the required circumstances, test procedures. Based on the first test results the device has positive effect.
Consortium delivered a Video Animation presenting the concept of the project, it was disseminated through online mediums like YouTube and Facebook, and also it was uploaded to the project website, as well as the video about the final prototype and the achieved results.
During the period the state of the art was continuously monitored, and information was shared within the Consortium.

As a summary, the main results achieved are as follows:
• Consortium defined a system which can greatly help the visually impaired people in everyday life, within affordable price range
• An effective research tool was developed to implement the desired software functions of the glasses
• Optical system was designed, and manufactured which is small and provides the necessary resolution to realize the DIGIGLASSES project
• A digital smart glasses hardware was developed and demonstrated
• A set of software algorithms were developed and demonstrated that can help the visually impaired people
• Low level optimization of the algorithms were demonstrated
• A PC based GUI was developed enabling configuration of the algorithms for the user
• The usability of the device was tested through laboratory and end- user tests with positive results

Potential Impact:
The DIGIGLASSES project successfully completed its tasks and demonstrated the complete DIGIGLASSES system -hardware and software containing HMD display system, HMD camera, optics system, image processing DSP (hardware and embedded software), the PC based DIGIGLASSES configuration software and documentation is considered to be the most important foreground. The technology has already been tested and validated by visually impaired end users . The second and third main results of the project are also marketable on their own, such as the know-how of the head-mounted display system and the configuration software containing the knowledge of applicable image modification processes.

The aim of the DIGIGLASSES project was to provide a marketable stereoscopic vision aid device to enhance the mobility of the visually impaired people with existing, but extremely low vision. Furthermore, this technology will provide European SMEs not only a highly marketable product, but will open larger markets in Assistive Technology (AT) market.

Economic and EU wide impact

According to the the European Blind Union an average of 1 in 30 Europeans experience sight loss. Partially sighted persons are 4 times more than blind people and altogether there are over 30 million blind and partially sighted persons in geographical Europe (nearly 10% of the total visually impaired population). One in three senior citizens over 65 struggles with sight loss as the likeliness of most important eye diseases grow progressively by age. Most cases their vision loss cannot be recovered by common eyeglasses. Furthermore, the population of Europe is getting older in parallel older generations would also like to live an active life it is a crucial task to increase the standard of living of these visually impaired people. According to United Nations’ data from 2012 the proportion of the world population over age 60 is projected to increase from 11 percent today to 22 percent by 2050. Population pyramids, especially in Europe, the U.S. and Canada reports that while elderly population is increasing young people is shrinking. As the population ages, more people will need care and fewer people will be available to provide and pay for the patient-centered care characteristic of Western societies today. Consequently, new solutions will be needed from researchers that develop technologies to assist in elder care (Peterson et al., 2012; Broekens et al., 2009).

Although diseases causing sight loss have sharply different symptoms, the fact that they go hand-in-hand requires complex solutions. The main aim of DIGIGLASSES project was to ensure this complexity by the customizable image processing applications.

Since the device is competing at the Assistive technology (AT) market it is important to highlight the predicted situation of this segment. AT market is projected to grow in Europe in US and also in China mostly due to the above mention tendency of ageing population.

The U.S. disabled and elderly assistive technologies market (including glasses and contacts) was valued at $45.9 billion in 2013 and predicted to reach $60.5 billion in 2018, after increasing at a five-year compound annual growth rate (CAGR) of 5.7%. As a result of these developments the affected groups are living longer independent or semi-independent lives with the help of assistive technologies . The same tendency can be followed in China where the country's 12th five-year plan for the elderly care industry forecast that, between 2011 and 2015, the 60+ sector would increase by an average of 8.6 million a year, topping 221 million (16% of the population) by 2015. Over the next two decades, the average age will increase still further, with the number of over 60+ expected to double by 2030.

DIGIGLASSES perfectly fits to the expectations of the changing demographic tendency and as a newly developed high-tech product will serve the well-being and independent living of elderly people.

Social impact

The prior market of DIGIGLASSES has been identified already before the project launch and was strengthened by project partners such as ADAPTOR and INFOALAP that have daily contact with the visually impaired customers.

Today visually impaired people hardly can afford to buy an expensive tactile device due their limited financial situation. The unemployment statistics implies a correlation between visual impairment and poverty rate. Although in theory the target market is huge since there are 285 million people with visual impairment in the world the consortium had to face with the fact that standard of living of visually impaired people is significantly lower than the average and the employment rate of the visually impaired is fewer than 30% especially in Eastern European countries. The use of DIGIGLASSES contributes to reach a significantly higher employment rate and to a better social integration of visually impaired people.

Furthermore, it is also an important fact that there is a huge difference between the countries national health system and its subsidizing net. While in Scandinavia almost 90% of the AT devices for visually impaired people are paid, in Hungary there are only some essential devices that get state support. Therefore, the researchers had to take into consideration the price level of the device, and keep the prices among 1000-1500€ that proved to be an accepted price for the glasses.

The benefits of DIGIGLASSES technology are evident from both economic, and, in particular, from social point of view. Better access to daily activities such as walking, reading, etc. and the learning of new technologies is critical for the visually impaired.

Dissemination:

The dissemination plan was defined in terms of the platforms and tools to be used and prepared and relevant events have been identified. A database of the target audience was created and continuously updated. The dissemination actions implemented were concentrated on two aspects, promoting the FP7 funded project itself and spreading information on the planned product as part of its marketing campaign.

Main dissemination actions so far:

The main dissemination event was the Vision 2014 Conference in Melbourne, Australia (31st March – 3rd April 2014). During this Conference Mr. Haugen from ADAPTOR had a presentation on the DIGIGLASSES project.
Another significant event was the Sight City 2014 Conference in Frankfurt, Germany (14-16 May, 2014) where ADAPTOR also represented the DIGILASSES project and continued the discussions started on Vision 2014.
During the project execution the Consortium and ATEKNEA has received several email and phone requests mainly from the visually impaired people to give more details about the project, and the target market launch.
The project website at www.digiglasses.eu was maintained on a regular basis ensuring that interested parties receive up-to date information on project progress.
The basic dissemination tools such as the project logo, leaflet, project poster, newsletters (available in electronic and printed versions) of DIGIGLASSES were prepared.
A database of target audience was prepared and has been continuously updated by all partners. It includes various blind associations, companies manufacturing or distributing assistive technology as well as relevant multiplicator organizations who could assist the dissemination and exploitation process.

The SMEs agreed to prepare a video about DIGIGLASSES –The video is available on the website www.digiglasses.eu under the new section “video” and on the official Facebook page of ATEKNEA (http://www.facebook.com/pages/MFKK-Invention-and-Research-Center-Services-Co-Ltd/217464818283451 ). A second video was prepared focusing on the prototype itself this was also uploaded to different social media sites.

All dissemination materials clearly state that the DIGIGLASSES project has received funding from the European Union’s Seventh FrameworkProgramme for research, technological development and demonstration under Grant Agreement No.315127.

List of Websites:
www.digiglasses.eu; digiglasses@ateknea.com