The research and development in the TAPAS project are organized into three parts:
I. Pathological speech detection
The activities under this research direction have focused on: (a) development of novel neural architectures to capture word-level information and sentence-level information embedded in manual transcript and automatically generated transcripts for dementia detection, (b) development of a multi-instance learning framework for speech-based assessment, (c) integrating prior knowledge about speech production in raw waveform modeling neural networks to assess pathological speech and (d) development of deep learning-based approaches for sensing breathing signal and breathing parameters from the speech signal.
II. Pathological speech assessment and therapy
The activities under this research direction have focused on: (a) investigating different neural network architectures and training methods together with transfer learning to robustly estimate phonological features for pathological speech intelligibility assessment, (b) analyzing the internal representations learned by deep neural networks trained for speech recognition task for objectively assessing head and neck cancer voice intelligibility, (c) development of automatic methods for speech-based evaluation of Parkinson's disease and integrating those evaluations with movement information captured through inertial sensors for holistic assessment of neurological state of the patients following the Unified Parkinson’s Disease Rating Scale, (d) determining deviations in pathological speech that have most impact on speech intelligibility and finding good procedures for measuring intelligibility, with reduced workload, (e) conducting literature survey, online survey and expert interviews for development of a clinician-friendly low-level segmental acoustic measures-based speech assessment tool, and (f) development of speech data, exercises, linguistic targets and methods for development of a virtual articulation therapist, to guide patients through intensive treatment program for improving articulation and, consequently speech intelligibility.
III. Communication technologies for assisted living and rehabilitation
The activities in this research direction have focused on: (a) development of an articulatory to acoustic inversion system for demonstration of phenomena of pathological speech and collection of publicly available oral cancer speech data for explaining the difference between oral cancer speech and healthy control speech, (b) investigating the influence of language model trained with in-domain data and out-of-domain data on dysarthric speech recognition, (c) application of state-of-the-art sequence discriminative training methods for acoustic modeling and analysing the errors made by the automatic speech recognition system, when recognizing control speech and dysarthric speech, (d) improving recognition of children speech through transfer learning and data augmentation for children pathological speech assessment in the context of gamified speech therapy sessions and (e) development of tools to automatically assess speech production deficits/problems of cochlear implant users, such as through evaluation of articulation deficits at consonant-vowel and vowel-consonant transitions, developing models to distinguish between healthy speakers' speech and cochlear implant users' speech, developing methods to distinguish between "Pre-lingual" speech and "Post-lingual" speech.
Beside peer reviewed conference and journal publications, these activities have also resulted in open source software or tools such as, Phonet, Apkinson. The TAPAS project consortium has organized three training events: (i) Speech Pathologies and Therapies, (ii) Speech processing and machine learning and (iii) Data collection, management and ethical practices.
