Final Report Summary - ANAGRAPHS (Anaglyptic Refreshable Photo-Haptic Screen)
The overall industrial objective of the ANAGRAPHS project was to develop an anaglyptic or tactile display that can be connected to a personal computer or used in stand-alone mode as a Braille e-book reader. The device will also be adaptable as a graphic tactile display able to display both Braille characters and simple graphical information. The system will be a major contribution to the integration of blind and visually impaired individuals into a society of knowledge. The proposed development aimed to deliver a system which better meets the needs of blind and sight-impaired people by providing a device which operates by thermo-hydraulic micro-actuation. Each of the Braille dots will be activated by a heating system under microprocessor control using specially developed software.
The development of an initial specification for the characteristics of the ANAGRAPHS system as a whole and for the software component was developed. The technical development of the concept design for the thermo-hydraulic actuator mechanism that employs the expansion characteristic of paraffin waxes at phase change from solid to liquid has been developed by using a combination of analytical modelling, finite element method (FEM) simulations and actuation data and was followed by extensive technical trials.
ANAGRAPHS user interface software was developed to convert ASCII text to Braille format and send the binary data of image(s) to the ANAGRAPHS device. An algorithm was implemented for the translation process, which creates bitmap image(s). The user interface allows the operator to disconnect the display device once the required images are transferred to the device.
Finally an assembly method and prototype casing was developed to house the control PCBs with the display screen. The housing components and assembly were developed on a 3D CAD system which ensured all the parts fitted together, and it also brings the right CAD data for machining prototypes and manufacturing of SLS rapid prototype models. The assembly of the full scale ANAGRAPHS prototype demonstrator device was carried out and all the sub-components were successfully integrated together. A cost model has been developed for the ANAGRAPHS device and initial forecasts look very competitive and a risk analysis for the development and commercialisation has been produced.
To ‘demonstrate proof of principle’ of the ANAGRAPHS system a number of trials were carried out. These trials using pre-loaded EEPROM images, and tested the software conversion of ASCII to Braille and Braille image creation and transfer to the display screen to show that correct and acceptable pixel actuation was achieved.
A detailed test regime for ‘End User’ trials was successfully developed with input from personnel from Royal National Institute for the Blind (RNIB) and the University of York to assess the functionality, performance and market needs for the ANAGRAPHS device. ‘End User’ trials involved experienced blind Braille readers were carried out and suggested that there was a real market for such a device and especially if the new technology used can drive the cost of a Braille pin down to an acceptable manufacturing cost that would make a Braille device affordable to the ‘End Users’. It was also stressed that Braille is very important for developing learning in reading and writing, especially helpful if such a device was available and low cost.
The ANAGRAPHS device was described has the ‘Holy Grail’ for the visually impaired and blind Braille users
Project Context and Objectives:
The aim of the ANAGRAPHS project is to develop an anaglyptic or tactile display that can be connected to a personal computer or used in stand-alone mode as a Braille e-book reader. The device will provide reading of any kind of textual or graphic information and be a major contribution to the integration of blind and visually impaired individuals into a society of knowledge.
Our development aims to deliver a system which better meets the needs of blind and sight-impaired people by providing a device which operates by thermo-hydraulic micro-actuation. Each of the Braille dots will be activated by a heating system under microprocessor control using specially developed software.
The scientific objective is to enable the technological developments through the extension of the current knowledge on the following areas:
The scientific objectives of the project were to:
• Enhance the scientific understanding of phase-change micro-actuators to enable the development of a display which will dramatically improve the functionality and reliability of tactile interfacing.
• Aim to achieve a vertical displacement of at least 20µ in the phase-change material which will be amplified by at least a factor of 10 by the micro-hydraulic layer.
• Enhance the scientific understanding of IR laser - micro-mirror scanning systems to enable the development of a heat distribution system which has sufficient degree of control to enable the heating of the phase-change micro-actuators in a full-page display.
The technological objectives of the project are to:
• Develop an anaglyptic display which will provide a tactile and optical output having a minimum dot elevation of 0.5mm.
• Power the ANAGRAPHS display for a duration of at least 8 hours before battery charging is required.
• Conform to ISO 10993 and meet the criteria for device type approval
• Lead to the manufacture, distribution and sale of the ANAGRAPHS device at an ex-factory cost of less than €7000 (with intensive post- project development) with a service lifetime of at least 5 years.
The technical work over the period (1st January 2011 – 31st December 2012) has been spread over the tasks in the following Work Packages:-
• Work Package 1: New Scientific Knowledge
• Work Package 2: Development of Thermo-hydraulic screen
• Work Package 3: Development of a Resistive Heating System
• Work Package 4: Development of Software
• Work Package 5: System Integration, Validation and Testing
The concept design of the thermo-hydraulic actuator mechanism that employs the expansion characteristic of paraffin waxes at phase change from solid to liquid has been developed by using a combination of analytical modelling, finite element method (FEM) simulations and actuation data. The energy requirements calculated for a screen with 6000 Braille dots adds up to a considerable amount of energy. Due to the energy constraints, the mass of paraffin wax had to be kept at a minimum which led to several micro-actuator concept models being developed, but after reviewing with micro-manufacturing industrial experts who raised a number of serious production issues the concept focused on a more simple wax actuator model. Base on this ‘direct expansion’ concept design substrate, membrane and wax materials have been trialled and the most suitable selected for the development of the thermo-hydraulic display screen.
Characterisation of the IR laser scanning system has been carried out which is used as a heat source for the activation of the thermo-hydraulic actuated Braille dots in the display screen. The main objective of the IR scan system is to in-coupling light into the absorption material layer of the thermo-hydraulic actuator. With the absorbed optical intensity the phase change material will heat up and expands. Using single light sources (e.g. IR lasers) the light beam has to be moved over thousands of absorption layers of the thermo-hydraulic screen. This can be done with a modulated laser and the use of double resonant scanning micro mirrors. Based on some assumptions mainly analytical calculations were performed for the determination of the overall IR scan system requirements.
The initial specification for the software component has been broken into three main sections, software for converting text or images to Braille, converting Braille to monochrome image software and control software for information control on the ANAGRAPHS screen.
The development of an initial specification for the characteristics of the ANAGRAPHS system as a whole was an extremely complex task, largely due the numerous, and severe technical issues encountered during the early months of the project. The key issues defined were the screen resolution, laser scanning system, thermal management and power restrictions. Following a detailed joint technical review, a modified concept was selected which satisfies the fundamental needs of the project. This has been explored in detail and a newer draft specification has been proposed. Prior to formal submission for approval the concept shall be submitted to a user focus group to obtain additional feedback and more extensive user requirements. The specification shall be updated as appropriate.
For the design of the thermo-hydraulic display computational fluid dynamics analysis determined the need for a bulk thermal insulator such as a polycarbonate substrate which will be one piece and contain >6000 cell features These cells will be filled with wax for the thermo-hydraulic actuator mechanism and sealed both sides with a membrane. Initial research focused on each of the Braille cells being heated by in-coupled infrared radiation. However after an extensive review and careful consideration by the consortium regarding the IR laser design, a number of urgent concerns were raised. It was therefore agreed that the IR laser system would not be suitable as a heat source for the development of the ANAGRAPHS system. An alternative technology has been defined and three micro-heater concepts are currently being researched. But since there has been a change in heating technology the requirement for an absorption layer is no longer required.
After initial research a paraffin wax was selected as suitable phase change material because it exhibits volumetric expansion of c. 15 % upon phase change. It had a melt temperature which was an ideal starting point as it provides a reasonable level of thermal stability for the initial trials.
The selection of an elastomer membrane material was carried out which was based on a three-fold functional criteria. Primarily the membrane provides a method of sealing the wax, in its molten state, into the individual Braille dots within the screen. Secondly it’s tensile and elongation properties enable the expanding wax (upon phase change) to form a Braille dot. Lastly the membrane, being the display’s tactile surface, provides enough wear resistance to protect the display’s function throughout the lifetime of the device. Based on these requirements membranes from the silicone family and polyurethane family were further investigated. After extensive trials an elastomer membrane material was selected which provides the best combination of elastic recovery, elastic modulus, adhesion and mechanical properties for subsequent development.
In order to evaluate the characteristics and performance of the elastomer membrane to form a Braille dot a suitable coating methodology was first required in order to produce reproducible membranes. A coating rig was designed and developed for the trials to produce membranes reproducibly of different thicknesses. The performance of the membrane was evaluated visually by noting the uniformity and shape of the Braille dots when heat was applied to the model. The effect of cooling rate on the performance of the Braille cell was also evaluated visually. The trials resulted in determining the best deposition technique and optimum thickness for the membrane providing the best mechanical integrity producing uniform Braille dots with no evidence of bleeding.
A small scale screen comprised of six active Braille dots (i.e. one character) was developed at MatRI to demonstrate the basic principal of using via thermal expansion of the paraffin wax. A design was created for the screen in accordance with the research, known technical and manufacturing restrictions found to date. Also an interim micro heater system was developed to enable the “controlled testing of the 6 dot array and demonstrate the feasibility of using resistive heating. The heater design was discussed in collaboration with IPMS and will facilitate the development of the advanced heating and control system. The six dot prototype test system included independent control switches, variable temperature and a heating array board and successfully produced dots with the appropriate shape and size characteristics. The test system was also used to examine the various candidate membranes and substrates and shall be used to construct a detailed materials performance specification.
From these extensive trials a display screen concept was developed using paraffin wax reservoirs where the side walls use electroless plating to increase the contact area between heater and the wax which enhances the rate, and consistency of heat transfer within each wax pot. A resistive heater array was developed which was used for the control of the ANAGRAPHS thermo-hydraulic Braille actuator cells. The design was a passive array, divided in to 9 sub-arrays that work with pairs of resistors and diodes as well as column and row switches. Using a resistor geometry and OhmegaPly technology 4x5 matrix dot small resistive heater array samples were manufactured. Extensive electrical and thermal measurements with both static and pulsed power in-coupling were performed and the electrical and electro-thermal measurements showed good, reliable and stable results and enable a good understanding of the fundamental physical behaviour of the developed and manufactured resistive heater array. With these results the operation of a passive heater array and the control of 6120 thermo-hydraulic cells used for Braille reading as well as graphical representations enabled the developed technological approach integrating the resistive heater array and the wiring of the 6120 dot array with a multi-layered PCB. Furthermore, we decided to integrate the paraffin wax reservoirs also into the PCB. Initial system-assembly and testing using a thermography system successfully demonstrated that the development of the electronic system including firmware provides activation of the correct selected dots and patterns tested, although thermal transfer between the dots and heat distribution over the full board was an issue.
ANAGRAPHS user interface software was developed to convert ASCII text to Braille format and send the binary data of image(s) to the ANAGRAPHS device. An algorithm was implemented for the translation process, which creates bitmap image(s). The user interface allows the operator to disconnect the display device once the required images are transferred to the device. Translation operation is performed with four iterations since translated Braille will have four rows. All iterations are operated until the end of the text. In other words, ASCII text to be translated is analysed four times in total, for each row of the translated Braille. After the translation is done and saved in file, it is analysed and formatted to 68 x 90 pixels in order to create bitmap images. The user interface software allows the user to input text which is then converted to Braille images and shown to the user in the Braille image box. To this, the software also allows users to add pre-generated Braille images to be sent together to the Braille display device. Images can be sent individually in single mode or in batches of 10 in the multiple image mode of the software. Once read on the Braille display device, more images can be requested using the buttons on the screen. Additionally, PictureBraille has been used to convert graphic files into tactile graphic files suitable for the ANAGRAPHS display board. The software package has been enhanced and functionality added to save the converted tactile image to a monochrome bitmap tactile image with the right dimensions (90x68) as required by the ANAGRAPHS display.
Once the software programming was complete, the user interface software was tested with one of the test boards to check correct functionality and operation of software. The tests carried out involved:
• Typing different text paragraphs in the software and converting them to Braille - verify text conversion to Braille
• Generation of Braille images – verify the Braille images generated were of the right size and structure as required by the ANAGRAPHS display
• Sending several different Braille images over the communication link to ensure the right mage were being sent and received without any corruption or data loss in the process.
The preliminary tests confirmed correct operation of the features in the user interface software. Final tests were carried out to check the operation of the software and the firmware after the display device had been fully integrated and all software functions implemented. Tests carried out included:
• Connection and disconnection of the display device
• Communicating with the display device and getting its status
• Send images to the display device and verify pin movement on the display.
Tests proved that the operation of the user interface software and the firmware developed for the ANAGRAPHS system with images were being successfully transferred from the software to the display device and pins rising on the display.
From the prototyping and trials of the 4x5 ‘dot’ array demonstrator devices of which there were different design versions one was chosen to further characterise and on develop into the full scale 6120 dot device. The full size ANAGRAPHS display screen was designed and several were successfully produced which consisted of a top layer that contained the reservoir holes and a bottom layer which had the resistive heater plates and a matrix of SMT diode components.
To fill the 6120 holes with paraffin wax investigations were carried out into the automation of the paraffin wax filling process where industrial dispensing systems and pick and place robots with the capability to dispense small deposits with excellent repeatability and increased productivity and cost savings were considered. Using these technologies and after a number of challenging trials the wax dosing technique and set-up parameters were defined and the wax reservoirs were filled and the application of an elastomeric membrane over the filled holes successfully achieved. To complete the ANAGRAPHS display screen assembly a concept of an ‘End-User Interface’ clamping plate which provides a ‘clamping’ effect on the screen’s membrane to reduce delamination around the wax reservoir holes and uses the wax actuation to lift the pins to be ‘felt’ by the Braille user was designed and prototype manufactured.
Finally an assembly method and casing was developed to house the control and resistive heating PCBs with the display screen. The housing components and assembly were developed on a 3D CAD system which ensured all the parts fitted together, and it also brings the right CAD data for machining prototypes and manufacturing of SLS rapid prototype models. The assembly of the full scale ANAGRAPHS prototype demonstrator device was carried out and all the sub-components were successfully integrated together. To carry out wear trials a wear test simulator device was designed and constructed to facilitate the simulation of typical usage to quantify the degree of wear on the elastomer by providing accelerated cyclic stress on the membrane.
To ‘demonstrate proof of principle’ of the ANAGRAPHS system a number of trials were carried out. These trials using pre-loaded EEPROM images, and tested the software conversion of ASCII to Braille and Braille image creation and transfer. The pre-loaded image trials to the display screen showed that the correct pixels were heating up and produced acceptable membrane actuation. However the pin actuation was not as good when the ‘End User Interface’ clamping plate was fitted. Using the ANAGRAPHS software and converting ASCII text to Braille characters and transferring to the ANAGRAPHS device was successfully carried out. It was observed that the pin actuations for the Braille characters sent to the device were both in the right position on the display and the correct pins were raised. Pre-prepare Braille images were successfully created, then converted to monochrome bitmap tactile image with the right dimensions (90x68) and sent to the device. Picture images, shapes, graphs and charts were trialled. These validation trials successfully demonstrated ‘proof of principle’ of the following:
• Text and image Braille conversion and transfer to a 90 x 68 pixel display
• The correct pins were being actuated on the display
• Successful pin actuation using cost effective wax and resistive heating technology
This demonstrated that Braille text and images together can be sent to and displayed on a screen format. The validation results demonstrated that the ANAGRAPHS system worked successfully but the performance of the pin actuation and thermal transfer issues needs further improvement and recommendations were made for commercial development.
A detailed test regime for the ‘End User’ trials was successfully developed from discussions and in collaboration with personnel from Royal National Institute for the Blind (RNIB) and the University of York to assess the functionality, performance and market needs for the ANAGRAPHS device. ‘End User’ trials involved experienced blind Braille readers were carried out and suggested that there was a real market for such a device and especially if the new technology used can drive the cost of a Braille pin down to an acceptable manufacturing cost that would make a Braille device affordable to the ‘End Users’. A trials review with experts in interactive Braille and tactile systems was also carried out and they suggested that this development which converts, transfers and displays both Braille text and images on a display screen and would be a very valuable device if it was available, as there is nothing commercially available like this on the market today.
Generally the ANAGRAPHS device was well received by all, the first prototype worked and ‘demonstrated proof of principle’, but performance of the pin actuation needed some improvement. However there is most definitely a commercial opportunity for such a Braille tactile device, especially in education and higher learning that would help to support delivery of new information to students. But it was also stressed that Braille is very important for developing learning in reading and writing, especially helpful if such a device was available and affordable. The ANAGRAPHS device was described has the ‘Holy Grail’ for the visually impaired and blind Braille users
To aid dissemination of the ANAGRAPHS project a website (http://www.ANAGRAPHS.eu/) has been created which gives the public information on the project goal, partners, project objective and press releases. This website has since been further developed to raise awareness and interest in the ANAGRAPHS product, targeted at organisations for the blind, ‘End Users (blind and partially sighted individuals), media and strategic partners such as investors and manufacturing companies. A dissemination plan has been produced to inform the scientific and industrial communities through papers and articles being published in major relevant international journals and through conferences and trade shows. A range of target customers has been defined from educational establishments, through to Braille associations and municipal organisations. There are a number of routes to market – through awareness and online product portals of national associations, through conferences and exhibitions, via assisted living companies, educational establishments and existing distributors/resellers. National associations for the blind often host an online product directory or shop where individuals can purchase products for the blind or partially sighted. An exploitation agreement to build on the rights granted by the Consortium Agreement has been developed and agreed by the project beneficiaries
The market for haptic (or tactile) displays has been defined and the opportunity for marketing development, sales targets are estimated to be in the range of 10% market penetration of Braille readers by 2017 with potential additional opportunities in the device’s application in commercial and municipal settings, such as public transport and retail environments. Sales and marking routes have been proposed for the device such as educational establishments, distributors/resellers, direct, associations (e.g. RNIB) and municipal authorities
To assess the novelty of such a system, literature searches using databases dedicated to engineering looked for published technical papers and patent searches using the Derwent World Patent Index (WPI) database were carried out. The results revealed no relevant patents that would infringe the ANAGRAPHS project and enables ‘freedom to operate’.
A cost model has been developed for the ANAGRAPHS device and initial forecasts look very competitive and a risk analysis for the development and commercialisation has been produced. Existing competition has been researched in detail with costs and highlights that what is needed, is an actuator that is energy-efficient enough to be used without a locking mechanism, yet is cost effective per dot and fits in a standard Braille cell profile such as the ANAGRAPHS device. From the economic perspective a suitable technology has also to be attractive enough to cover a wider market to make its development worthwhile from an entrepreneurial point of view.
List of Websites:
The ANAGRAPHS project website (http://www.ANAGRAPHS.eu/)
For further details, contact:
Hobart Lasers Limited,
Little Market Row,
Leybourne, West Malling,
Kent ME19 5QL,
Direct: 01732 252053
Mobile: 07814 000540 +34 617 215 537