SignON aims to go beyond the state-of-the-art by (i) providing an unprecedented translation service and mobile application that are user-driven and co-created with users; (ii) researching and developing new methods, methodologies and models for sign and speech recognition, as well as machine translation; (iii) delivering a new, customisable 3D virtual signer; (iv) collecting and unifying existing (in terms of formatting and organisation) and generating new data (synthetic data as well as sign language specific lexicons to drive the 3D virtual signer); (v) integrating machine learning capabilities in a large-scale framework to allow new languages, user preferences and use-cases to be integrated; as well as (vi) bringing deaf, hard of hearing and hearing communities closer together through co-creation, collaboration and open communication. We would also hope to provide ethical guidelines and principles that would help ensure fair and respectful data collection and curation, as well as the research and development of open and unbiased communication and collaboration for future projects of the scale and scope of SignON.
The project ran over 36 months with all tasks that were specified in the Grant Agreement completed and all deliverables submitted on time. The SignON communication framework and the SignON SLMT and ML applications were developed following co-creation principles. The applications achieved Technology Readiness Level 6.
The expectation we set at the beginning of the project, is that the project will continue to be publicised and spoken about via continuous communication and dissemination activities which will lead to a continuously growing community, even beyond the lifespan of SignON. In the 3 months after its end we see its impact on academic and user communities; SignON resonates in new publications, presentations, events, satellite and spin-off projects, and has the potential to continue doing so in the future.
Appropriate use-cases and usage domains were identified and investigated within the SignON Sustainable Exploitation, Innovation and IPR plan. These have been ethical and privacy monitored.
In addition to this, and of more fundamental importance, is that SignON has made a large step towards bridging the communication gap between deaf, hard of hearing and hearing individuals. We believe that the co-creation process in our project instills confidence in the project and its results, and allows the respective communities to communicate more easily and more freely. Through SignON’s success we demonstrated the effectiveness of the adopted co-creation process and we advocate that this should be the default methodology in future projects involving DHH communities. We also see the need for a sign language data repository and draw plans towards the establishment of such a repository in the future.
The impacts for this project are huge. Besides the impact on scientific research in terms of technical advancements in multilingual speech processing and sign language recognition on mobile devices, the societal impact of having access to an easily accessible communication service in lieu of interpreters would create a huge benefit to society. The SignON application is not presented as a tool to replace interpreters and current means of communication, nor it is an off-the-shelf ready-to-use tool, but it sets a stepping stone towards a more open and easily accessible translation option available to the general public if it is required and appropriate. There has also been a noticeable impact on the various partners in the consortium, not least, an improved awareness of the interests, needs and requirements of DHH people as well as deaf culture and communities.
