Description du projet
Cognitive Systems, Interaction, Robotics
Dicta-Sign's goal is to develop the necessary technologies that make Web 2.0 interactions in sign language possible.
Dicta-Sign addresses the need for communication between deaf individuals and communication via natural language by deaf users with various HCI environments. It will research and develop recognition and synthesis engines for sign languages (SLs) at a level of detail necessary for recognising and generating authentic signing. Research outcomes will be integrated in three laboratory prototypes (a Search-by-Example tool, a SL-to-SL translator and a sign-Wiki) leading to a practical project demonstrator.Dicta-Sign will be based on research novelties in sign recognition and generation exploiting significant linguistic knowledge and coded SL resources. Interoperation of several scientific domains is required in order to combine linguistic knowledge with computer vision for image/video analysis for continuous sign recognition, and with computer graphics for realistic avatar animation. Coding of SL linguistic knowledge requires exploitation of annotated parallel video corpora for four SLs (BSL, DGS, GSL and LSF), linked to common grammar and lexicon modules created to feed both recognition and synthesis engines.These resources will also be applied, by exploiting multilingual syntactic and lexical interrelations, to domain-specific SL-to-SL machine translation. Sign input will be extended by integrating continuous sign recognition with multi-modal fusion and signer adaptation. Sign recognition technology will also help with annotation procedures and tools developed within the project as a necessary support for adequate corpus construction and data access. Procedural sign synthesis will be extended by further development of current state-of-the-art representations of manual signing elements in order to achieve close-to-natural signing, including non-manual features of eye gaze, mouthing patterns and body postures. Prototypes and the demonstrator will validate the interaction between sign recognition, linguistic modelling, and sign synthesis.
Champ scientifique
Appel à propositions
FP7-ICT-2007-3
Voir d’autres projets de cet appel
Régime de financement
CP - Collaborative project (generic)Coordonnées du coordinateur
Coordinateur
151 25 Maroussi
Grèce