Skip to main content
European Commission logo print header

Sign Language Recognition, Generation and Modelling \n with application in Deaf Communication

Project description


Cognitive Systems, Interaction, Robotics
Dicta-Sign's goal is to develop the necessary technologies that make Web 2.0 interactions in sign language possible.

Dicta-Sign addresses the need for communication between deaf individuals and communication via natural language by deaf users with various HCI environments. It will research and develop recognition and synthesis engines for sign languages (SLs) at a level of detail necessary for recognising and generating authentic signing. Research outcomes will be integrated in three laboratory prototypes (a Search-by-Example tool, a SL-to-SL translator and a sign-Wiki) leading to a practical project demonstrator.Dicta-Sign will be based on research novelties in sign recognition and generation exploiting significant linguistic knowledge and coded SL resources. Interoperation of several scientific domains is required in order to combine linguistic knowledge with computer vision for image/video analysis for continuous sign recognition, and with computer graphics for realistic avatar animation. Coding of SL linguistic knowledge requires exploitation of annotated parallel video corpora for four SLs (BSL, DGS, GSL and LSF), linked to common grammar and lexicon modules created to feed both recognition and synthesis engines.These resources will also be applied, by exploiting multilingual syntactic and lexical interrelations, to domain-specific SL-to-SL machine translation. Sign input will be extended by integrating continuous sign recognition with multi-modal fusion and signer adaptation. Sign recognition technology will also help with annotation procedures and tools developed within the project as a necessary support for adequate corpus construction and data access. Procedural sign synthesis will be extended by further development of current state-of-the-art representations of manual signing elements in order to achieve close-to-natural signing, including non-manual features of eye gaze, mouthing patterns and body postures. Prototypes and the demonstrator will validate the interaction between sign recognition, linguistic modelling, and sign synthesis.

Call for proposal

FP7-ICT-2007-3
See other projects for this call

Coordinator Contact

Eleni EFTHIMIOU Ms.

Coordinator

ATHINA-EREVNITIKO KENTRO KAINOTOMIAS STIS TECHNOLOGIES TIS PLIROFORIAS, TON EPIKOINONION KAI TIS GNOSIS
Address
Artemidos 6 kai epidavrou
151 25 Maroussi
Greece

See on map

Region
Αττική Aττική Βόρειος Τομέας Αθηνών
Activity type
Research Organisations
Administrative Contact
Yanna Volti (Ms.)
Links
EU contribution
No data

Participants (7)