Making technology more inclusive
Digital technology seems to have permeated nearly every aspect of our lives, radically transforming how we connect, work and interact. But this isn’t the case for everyone. Because digital devices rely almost exclusively on visual and auditory feedback, they raise inherent accessibility issues. In fact, access to digital content by the visually impaired – especially low vision and blind people – remains challenging, even for content as elementary as text. “Considering our increasing reliance on technologies, dematerialisation and the use of buttonless touchscreens, a significant portion of the population is at risk of being denied access to content and becoming stigmatised even more because of their disabilities,” explains Sabrina Panëels, a researcher on interactive systems at CEA-List(opens in new window). With the support of the EU-funded ABILITY(opens in new window) project, Panëels is leading a group of partners(opens in new window) working to help close this digital accessibility gap. “By using and developing new technologies and software and by leveraging haptic solutions, we aim to make digital content and devices more tangible for everyone,” she says.
The first multisensory tablet
Having directly engaged with 285 sighted and visually impaired users to identify actual user needs, the project went on to achieve a lot of firsts. This includes the first multisensory tablet able to render textures on a standard OLED screen. The 10.5-inch tablet prototype runs on Windows and features a screen equipped with surface haptics. “The prototype uses localised and multitouch vibrations that can render textures depending on the underlying content, for both static and moving fingers,” notes Panëels. “It can also be felt independently by different fingers, meaning that when a user for example explores a map, one finger can feel the waves of the sea while another can feel the texture of forests and other topological elements.” The project further investigated design strategies to render content haptically, as well as the benefits of using a combined 2D pin display and the multisensory tablet with artificial intelligence (AI) for converting images, PDFs and graphs into touchable content in real time.
Using AI to make digital content more accessible
The project also made advances in developing machine learning algorithms tailored to the unique needs of the visually impaired. This work includes creating new software that uses AI for image analysis and predictive writing to turn digital content into accessible visual, audio and touch alternatives. “We are particularly proud of our use of an image analysis pipeline and conversation AI solutions to deliver interactive descriptions of an image tailored to the individual user, as well as our predictive writing app that allows users to include emojis in their communications,” remarks Panëels.
Raising the bar for inclusive technology
Despite encountering many challenges, the project successfully demonstrated that an affordable, AI-powered, multisensory tablet is technically achievable, laying the groundwork for a commercial product and for a new standard for user-centred, inclusive technology development in Europe. “Our work moves the needle on inclusive design and inclusion in general, benefiting not only the visually impaired, but also people with other disabilities, the elderly and even children,” concludes Panëels. But before that can happen, more work needs to be done. That’s why researchers are currently working to secure the funding and partnerships needed to further develop and market its inclusive solutions.
