Skip to main content
European Commission logo print header
Contenu archivé le 2022-11-15

Multilingual Speech to Face-Movements Transformation for Use as a Training System in Lip-reading and Language Acquisition and as a Basis for a New Telecommunication Service

Objectif

The objective is to develop a telecommunication service for hearing impaired persons which converts the speech, issued from a telephone, in a synthetic animation of a face to allow lip-reading

Interpersonal communication between a hearing person and a hard of hearing person is only possible if the speech is converted into visual information which can be understood by the disabled person. The concept of SPLIT is to convert the speech signal provided by telephone lines into a synthetic animation of a face.

The aims are to design, develop and test two applications:

a) a prototype service of conversion (Relay Mediation Service) and the terminal
b) a PC station prototype for Lip-reading training and language acquisition with Cued Speech (Cued Speech is a coding system adapted to 42 languages which facilitates lip-reading)

Both systems are based on conversion of speech analogue signal into lip readable animation of a mouth image on the terminal screen. the 'SPLIT' terminal represents a significant aid for multilingual training, and enhanced telephone communications.

Technical Approach : Since the analogue speech signal reaching the 'SPLIT' terminal is corrupted, suitable filtering algorithms are to be devised to restore acceptable quality speech digital conversion. The algorithms for speech processing and lip movements will be defined, implemented, tested and tuned in the context of a constant interaction with the end user associations which will provide the necessary expertise and the specific know-how. Moreover, these conversion algorithms will be designed in such a way that they are able to work in the telephone environment ('Telephone speech signal'). That ensures the possibility of integration and testing in the Mediation Relay Service (Italian Language).

The main project outcomes will consist of software packages to perform the following speech-to-face movement transformation. This software package will be integrated on a PC architecture for training application and prototype Relay Mediation Service.

Impact and Expected Results : Multimedia approaches to interpersonal communication represent a formidable means of overcoming most of the impairments affecting communication with older and disabled people. The medium goal of this project is the deployment of the existing telephone network for new services oriented to hearing-impaired persons, by means of low cost acoustic-to-visual conversion.

This innovative conversion will be experimented during this project in training application to show the pertinence of such transformation for language acquisition and training of lip-reading.

Appel à propositions

Data not available

Régime de financement

CSC - Cost-sharing contracts

Coordinateur

Association Nationale des Parents d'Enfants Déficients Auditifs (ANPEDA)
Contribution de l’UE
Aucune donnée
Adresse
10 Quai de la Charente
75019 Paris
France

Voir sur la carte

Coût total
Aucune donnée

Participants (7)