OPTImization of the automated Fitting to Outcomes eXpert with language-independent hearing-in-noise test battery and electro-acoustical test box for cochlear implant users

Projektinformationen

OPTI-FOX

ID Finanzhilfevereinbarung: 262266

Abgeschlossenes Projekt

Startdatum 1 November 2010

Enddatum 31 Oktober 2012

Finanziert unter

• Specific Programme "Capacities": Research for the benefit of SMEs

Gesamtkosten € 1 747 897,40

EU-Beitrag € 1 274 928,80

Koordiniert durch

OTOCONSULT NV
Belgium

Final Report Summary - OPTI-FOX (Optimization of the automated fitting to outcomes expert with language-independent hearing-in-noise test battery and electro-acoustical test box for cochlear implant u

Patients suffering from sensori-neural hearing loss caused by damaged hair cells in the cochlea and diagnosed as being profoundly deaf, are potential candidates for cochlear implantation. Today, there are a number of important limitations with respect to the optimal use of these devices in deaf patients. Firstly, cochlear implant speech processors need to be adjusted so that sounds perceived by the patient are representational and at a comfortable level. Manual fitting, currently the norm, is technically demanding and time consuming and clearly sub-optimal, as it involves only two of the many electrical parameters in the speech processor. Secondly, manipulation of the implant settings is based on subjective judgments of the patient, which are often inconsistent and do not reflect the outcomes on psycho-acoustic measures. For the last few years, experts in the field have expressed the need for a new fitting process that optimises the patient's hearing in a more efficient and accurate way. For this to happen, the fitting procedure should change from a comfort-driven approach to an outcome-driven one. It should also address as many electrical parameters as possible.

Ideally, a cochlear implant should come with an assisted or (semi-)automated fitting procedure in which a large number of parameters may be adjusted, based on measured psycho-acoustic feedback from the implant user. Such an assisted fitting process would drastically reduce the number of man-hours of fitting during the lifetime of the device with qualitative with qualitatively better outcomes.

The main objectives of this collaborative research project were therefore:

(i) to turn an existing theoretical automated fitting model into a clinical application by means of various techniques from statistics, machine learning and optimisation; and
(ii) to develop an evaluation tool to measure functional hearing capacities, in casu the ability to understand speech-in-noise, representative for day-to-day listening situations.

In view of these objectives, three products have been developed which together will drastically change the technical fitting of cochlear implants:

(1) OTOspeech, a psycho-acoustical test to assess the auditory performance of cochlear implant users in a universal and automated way; universal in that it can be used with native speakers of a large number of different languages and that it is not influenced by dialectal variation; automated in the sense that it allows the scoring of speech intelligibility in an automated way.
(2) FOX, an artificial intelligence (AI) application which analyses the electrical parameters of a cochlear implant together with the psycho-acoustical test results obtained with it and which provides recommendations for changing the electrical parameters in such a way that the outcome will improve to target. In this project, an optimisation model has been developed to manipulate the many input variables as a function of the many output variables by using AI technology. In the future, the model will be integrated in the final application in such a way that it will be self-learning by continuous analysis of the growing data set.
(3) OtoCube, a portable desktop module with integrated amplifier, loudspeaker and soundcard, which allows psycho-acoustic testing and fitting of CI-users in standardised and calibrated conditions without the need of sound-treated test rooms with highly specialised and expensive audiological equipment.

Project context and objectives:

Project context

Hearing and deafness.

In healthy humans, the perception of sound is the result of a complex mechanism in which acoustic waves are transformed into nerve impulses on their way from the outer ear to the brain. The outer ear picks up sound waves that travel through the ear canal and cause the eardrum and the three bones within the middle ear to vibrate. In the cochlea or inner ear, a cavity filled with fluid, these vibrations lead to the displacement of a flexible membrane ('basilar membrane'). The tiny hair cells attached to this membrane are mechanoreceptors that release a chemical neurotransmitter when stimulated and cause neurons to fire and transmit information about the acoustic signal to the hearing nerve and the brain. If the link between sound and brain is broken at some point in the auditory chain, this may lead to different types, degrees and configurations of hearing loss.

Communicating with current hearing devices

The regular use of an adequate hearing aid increases the chance of keeping hearing impaired patients communicatively, socially and economically active. Today, advances in hearing devices enable intervention by means of a different types of hearing devices:

(i) individuals suffering from mild to moderate hearing loss may benefit from a classical hearing aid, a sophisticated acoustic amplifier which typically fits in or behind the user's ear, amplifying and modulating sounds. More severe hearing losses (> 70 dB HL) in the higher frequencies are usually beyond amplification range of classical acoustic stimulation;
(ii) patients suffering from sensori-neural hearing loss caused by damaged hair cells in the cochlea and diagnosed as being profoundly deaf, are potential users of a cochlear implant (CI).

The development of this device is based on the idea that in most deaf patients, in spite of the damaged cochlea, enough auditory nerve fibres are left that may be stimulated directly, bypassing normal hearing mechanisms. Through the CI, sound is picked up by an external microphone, converted into an electronic code in a speech processor, sent to an internal receiver and further on to an array of electrodes. The electrodes provide electrical stimulation of multiple loci in the cochlea, based on the characteristics of the code. High- and low-frequency signals stimulate respectively the base and the apex of the cochlea. The combination of amplitude and place of stimulation in the cochlea enables the brain of deaf individuals to interpret the incoming signal as having a particular pitch and loudness.

Fitting the implant: current issues

Although most patients perceive a major improvement in listening situations while using cochlear implants, there are a number of important limitations with respect to the optimal use of these devices in deaf patients. Firstly, patients have to be evaluated by an audiologist to have their speech processor 'fitted'. This fitting should be seen as creating a set of instructions that defines the specific characteristics of the code used to stimulate the electrodes of the implanted array. With advances in cochlear implant technology, speech processors have hundreds of electrical parameters that may be adjusted to the individual needs of the patient, to optimise his or her hearing. As such, each implant patient has an individualised program ('map'). The traditional approach for cochlear implant device fitting is to manually adjust a limited number of parameters in the speech processor component such that sounds perceived by the patient are representational and at a comfortable level. This manual fitting, currently the norm, is technically demanding and time consuming. It requires the presence of an expert audiologist or engineer who has to set the minimum and maximum current level outputs for each electrode in the array. The patient is required to answer questions in response to a series of audiological tests, the results of which are used to manipulate the implant settings in a stepwise up-down procedure. Several adjustments need to be made during the course of the first year after implantation, requiring at least 30 man hours (adult patients) or at least 60 hours (infant patients) of the fitter's time. Thereafter, patients need to have the device adjusted on an annual basis, consuming between 10 and 20 man hours per annum. Secondly, manipulation of the implant settings is generally based on subjective judgments of the patient (which sound do you like better?), which are often inconsistent and do not reflect the outcomes on psychoacoustic measures. It is also know that patient's comfort feeling changes over time because of habituation and adaptation. This causes the set levels to drift over time. Thirdly, in the current clinical practice, audiological tests are typically performed in sound-treated rooms with calibrated headphones. In CI-users, headphones cannot be used because of the fact that they are wearing an external speech processor with one or two microphones. The only alternative is currently to conduct the test with the stimuli presented through routine HiFi loudspeakers. As these test conditions are difficult to control, it would be very useful to have an acoustically insulated box that can contain the CI speech processor and that can produce exact acoustic replicates of an electric input signal.

Main objectives of the project

The main objectives of the project proposal addressed both the technological and scientific problem outlined above:

Objective 1: To improve the assisted fitting model with help of a self-learning and optimisation component

By means of an intelligent agent the map settings of the CI speech processor are adjusted in view of targeted (improved) hearing outcome measurements. The agent will act upon its environment and its actions are evaluated by comparing the targeted outcome to the observed outcome after map modification. The evaluation will allow the agent to learn from its own actions, thus evolving to a state of greater knowledge. In its final stage, the agent will have evolved into a true intelligent system, performing the fitting process without or with only minor human intervention.

Objective 2: To develop an evaluation tool for functional hearing capacities

More precisely, this objective involves the development of a test that provides a standardised measure with respect to the perception of speech in quiet and in noise conditions. As real life background noise is never stable but varies with the patient's environment, the test should be able to measure speech perception with different intensity levels of speech and noise. To be useful in the clinical practice, this test battery should be cross-linguistically interchangeable and provide direct acoustical feedback to the fitting software

Objective 3: To build a desktop audiological test booth with ideal conditions for the measuring of psycho-acoustic outcomes with cochlear implants

The box should be able to exclude external noise (acoustic insulation) as defined by ISO standards, and to produce an exact acoustic replicate of an electric input signal (a simulation of ANSI Type 1 free field testing).

Project results:

OTOspeech: An evaluation tool for functionl hearing capacities

Background

One of the objectives of this project was to develop a language and tester independent test battery that is able to assess speech perception of hearing (impaired) individuals in quiet and noisy conditions. The test battery, implemented in a software application that will be commercialised as OTOSpeech, is the outcome of the first two work packages (WP) in the project: WP1 mainly focused on the representativeness of the word lists to be used as speech stimuli in the test in view of the native language of the tested individual; WP2 was concerned with the automated scoring of the verbal response of the tested individual based on automated speech recognition techniques.

Main scientific and technological results

Context
Speech audiometric tests are used to assess the speech recognition abilities of hearing-impaired individuals. As these tests use speech stimuli, they give information about the communication abilities of hearing-impaired individuals in daily life. In this respect, tone audiometry is much more limited as it gives only information about the hearing-impaired individual's ability to detect sound. It is generally known that test outcomes are partially dependent on test stimuli. Therefore, it is important to gain insight in the speech stimuli that are currently used in clinical practice. First, a literature survey was performed on speech audiometry currently used in the Netherlands and Flanders.

The historical overview for the Dutch-speaking area has concentrated mainly on the development of different types of speech stimuli used in the tests for adults, such as sentences, spondees, monosyllables and digits. Sentences are known to have a high degree of informational redundancy in the sense that missing acoustic information does not necessarily lead to misperception, because context can be used to fill in the gaps. Therefore, often the less redundant speech stimuli are chosen as test items, involving spondees, monosyllables and digits.

To ascertain the representativeness of the speech stimuli to daily speech, the stimuli should be phonemically balanced. This means that the distribution of phonemes in the sentence- or word lists is highly similar to the distribution of phonemes in daily speech. A critical review of the existing tests included in our survey has shown that the tests under investigation were not phonemically balanced or that any scientific proof of such phonemic balancing was missing.

Building word lists that are representative for the sound system of the target language

One of the core tasks of WP1 was to establish a metric capable of determining the representativeness of a speech or text sample of a given language, based on the linguistic features of that language. As the metric will be used to build lists of words that serve as acoustical prompts in speech audiometry, priority was given to the sound system of the language. In agreement with original plan of work, the metric was first built on Dutch language data. In view of its commercialisation as a clinical application, it was required that the metric of phonemic balancing be able to:

1) establish speech stimuli sets that are representative of a language;
2) define a measure of representativeness; and
3) be used in other languages.

For the development of the metric, it was taken into account that all languages include two classes of words, i.e. open class words including nouns, verbs and adjectives and closed class words such as function words and that phoneme frequencies which are based a particular class may lead to an over- and/or underrepresentation of particular phonemes. Such over- and underrepresentation results from frequency effects of open and closed class linguistic elements, closed class words being more frequent than open class elements. To correct for this, it was necessary to counterbalance so-called types (different words) and tokens (all words). Extreme frequencies were corrected by establishing type (TY) and token (TO) frequency counts and counterbalancing them (TY/(TO/TY)).

Firstly, from an existing lexicon (Corpus Gesproken Nederlands, a corpus of 9 million words of spoken adult Dutch retrievable from the Dutch Language Union) a 'fingerprint' was established based on the count of each word's initial character, N-1 bigraphs and its final character. The metric involves both grapheme and phoneme frequencies and distributions. However, for many languages phonetic transcriptions of a reference corpus such as the one used for Dutch are currently not available. Therefore, it is not yet possible to establish a phonemic fingerprint of all languages. In order to overcome this problem, a distance measure was established calculating the representativeness of the Dutch language's sound system based on graphemes. Using graphemic transcriptions of words as a reference for speech sounds, a large correspondence between the orthographic and phonetic transcripts of the word lists was found.

Secondly, phoneme and grapheme frequencies were used to derive the optimum set of speech stimuli in terms of representativeness. In order to do, small selections (n = 25) of words were generated from the reference word list and the distance between the phonemic and graphemic fingerprints of both have been calculated. Statistical analysis revealed that with sample sizes of 25 words and based on initial, final and bigraphs a high correlation (r = 0.81) is obtained. Additional analyses of the data focused on lists of words that were constrained by word length. When using words of 3 - 5 graphemes (e.g. 'bal' ball, 'eend' duck), a decrease of the correlation (r = 0.72) was found, probably due to the fact that most words in Dutch are longer than 5 graphemes.

During the second year of the project, more combinations of different possible distance measures were explored, yielding correlations between 0.7 and 0.8. which were felt to be sufficiently strong to justify the use of the grapheme-based measure to assess sound representativeness of the target language. Further refinement of the measure included the analysis of language samples from different text types and sizes and of typologically different languages. The parameters of the final metric were adjusted to the properties of individual target languages. As such, it can be used for all 22 European languages.

Automated scoring of the verbal response of the tested individual

Expert assessment of speech is expensive, especially so for speech produced by patients with a pathological character, including speech produced by speakers with a hearing deficit. Automated speech analysis avoids assessment by humans and can therefore be made cheaper and, more importantly, is not affected by (inter and intra) examiner bias. One of the tasks of this project was to address some concerns regarding accuracy and robustness that might compromise the validity of automated stimulus / response similarity scoring in the present medical context.

At this background, inter- and intra-person variability was analysed on available speech corpora, and methods were explored to reliably assess differences between speech samples (within speaker across sessions, and between speakers). In view of the multilingual market for the final product under development, its threefold aim was specified as follows:

(i) to develop an automatic speech error analysis module;
(ii) to improve robustness of automatic error analysis (to various types of variation, such as language and dialectic variation); and
(iii) to improve robustness of automatic error analysis to articulation deficits.

Starting point was the design of an assessment method that was able to detect differences in pronunciations, especially between two tokens (acoustic realisations) of the same word, spoken by a single speaker. From the literature on automatic speech recognition several techniques were explored which are able to provide a 'dissimilarity measure' that give an indication of the difference (distance) between two tokens. Among these methods, dynamic time warping (DTW) is one of the major techniques. A much simpler approach is to uses a fixed-length vector representation of speech samples such as power spectral density estimation (PSD). The other extreme option was taken to be full-blown automatic speech recognition (ASR). By ASR, a continuous speech signal can be translated into a word sequence by looking for the best matching sequence of words, given the acoustic input signal. However, to that end one needs in general a number of knowledge sources (dictionary, acoustic models (AM), language models (LM)) in order to define the match function between signal and word sequence. Months of intensive experimentation showed that it was more feasible to promote the DTW option, at the cost of down-weighting the ASR option. A scoring algorithm judging the match / non-match of the pairs of verbal stimuli and the obtained responses was thus built on DTW technology. The outcomes obtained with this automated scoring algorithm have been compared to those obtained from manual scoring by an expert-audiologist. A validation study of this scoring algorithm included 249 native Dutch-speaking and 101 native German-speaking subjects.

OTOSpeech: The commercial product

One of the main requirements of the software tool was that it be usable in audiological centres around the world by operators who administer subjects. The final product OTOSpeech allows generating a personalised word list for an individual patient. This list is automatically sampled from the internet, it contains 300 words (can be changed in the settings) which belong to the lexicon of the patient and which have been sampled in such a way that they are phonetically representative for the given language (actually available for 22 European languages). These words are recorded once by the patient after visual prompting in a session of typically 30 - 45 minutes. During a speech audiometric test session, sets of 25 words (can be changed in the settings) are sampled from this word list and they are presented acoustically in the AE framework, either in quiet or together with noise (WP1). The hearing impaired patient then hears his own words and is asked to repeat them. These words are compared with the original words by means of speech recognition technology and this yields a score just like in conventional speech audiometry.

OTOSpeech consists of three components which have been developed in the Optifox project:

(1) the OTOSpeech builder
(2) the OTOSpeech runner and
(3) the OtoCube driver and calibration software.

The general functionalities of (1) enable the user to enter some text of choice that will serve as input for the recordings, to compute a representability metric of the text that has been entered, revealing whether the text is representative enough for the language of choice, to draw a sample from the text that contains the words to be recorded and to record the words. The functionalities of the OTOSpeech runner (2) are to enable the testing of a patient by playing the verbal stimuli, by recording the answers of the patient and by consulting the distances of all stimulus / response pairs. Test can be performed under noisy conditions.

2. Opti-Fox: Optimised automated feedback for cochlear implant fine-tuning

Background

The second main objective of this project was to develop a new, powerful methodology, algorithms and software tools for tuning cochlear implants that would overcome limitations of manual trial-and-error methods (technically demanding, time-consuming and sub-optimal, and taking into account only two of the hundreds of electrical parameters that may affect the patient's hearing). This new methodology is expected to maximise the quality of the fit of cochlear implant settings to patients, to reduce the number of 'trial-and-error' interactions with patients to a minimum, to allow for simultaneous tuning of multiple parameters, to provide customised tuning strategies for patients with different deficiencies of their hearing apparatus, to provide tools for re-training of the tuning strategies with help of new data, to be generic, i.e. be applicable to new situations where, for example, the format of test data (feedback from patients) is changed, or the set of adjustable parameters is modified.

Main scientific and technological results

Exploratory data analysis and modelling experiments

Starting point for the development of the optimised fine-tuning algorithm was an exploratory data analysis aiming at learning as much as possible about the quality of the data, about attributes, typical values, outliers, etc. After investigating simple statistics about individuals and assessing data quality, the fitting maps of each patient were further explored, drawing statistics for e.g. the number of days (sessions) spent at a centre, the time interval between the first and the last map, the time range between first and last record, and data about the device settings, impedance values or the number of active electrodes of the cochlear implant.

The objective of tuning a CI was defined as being driven by the constraints that are not met: whenever tests reveal that some requirements are not satisfied, a corrective adjustment (a map modification) should be implemented in hope of meeting all constraints. An evaluation framework was designed to measure the quality of various tuning strategies, including the one used so far, i.e. the manual tuning of two parameters. The evaluation framework included measuring the dependency between the number of tune-test iterations and the amount of improvement (measured on several dimensions).

The key question to be answered with help of data modelling was which CI parameters should be changed, and by how much, in order to obtain a desired change in the results of a specific test. Special attention was given to handling disabled electrodes. After preparing data, numerous experiments were run with three regression techniques: linear, logistic and robust, trying to model various output variables by various sets input variables. This led to hundreds of plots demonstrating the behaviour of various models.

The main conclusions from these modelling experiments were that:

(i) the 'delta approach' worked much better than the standard approach that uses original values;
(ii) data preprocessing is a key factor in obtaining satisfactory results. In particular, handling disabled electrodes, removal of outliers, dimensionality reduction play here an important role;
(iii) that non-linear modelling methods are not suitable due to the risk of data over-fitting; and
(iv) from the linear methods that were tried, the robust regression led to best results.

Development of optimal tuning strategy

Trying to build a model relating the physical parameters of the model (input) with the user's performance (output), several techniques were examined, including such as linear regression, stepwise linear regression, robust linear regression, logistic regression with various link functions, neural networks, principal component analysis, etc. However, the scarcity of data and the high number of variables involved in the problem showed that a pure data-driven approach was unfeasible. It became apparent that the correct approach should be based on a method combining expert knowledge and data. A model was then built that represents the parameters of the cochlear implant device, the electric activity in the auditory nerve and the results of the audiological tests. The model was built using OpenMarkov, a software tool that had been developed by one of the members of the consortium prior to this project. This software tool has been extended significantly by a new type of probabilistic model called 'tuning networks', by new models of interactions among variables, by new inference algorithms and new learning algorithms for refining the parameters of a probabilistic model with data. All of them were motivated by the needs encountered in this project. The initial prototype for cochlear implant speech processor fine-tuning has been tested with both hypothetical and retrospective patients, comparing its recommendations with those of human experts and those of earlier versions of an existing semi-automated fine-tuning system (FOX). This lead to many refinements of both the structure and the parameters of the model. By the end of the project, a comparison between the outcomes of an expert audiological (manual) fitting of a patient, to the automated fine-tuning of the cochlear implant speech processor by means of the optimised fitting model, showed that the latter was capable of bringing a poor performance in speech recognition within the range of normality.

From prototype of an optimal tuning strategy to a future commercial product

The model has been tested on a set of cases taken from a database of real CI users. The advantages of the new model with respect to the rule-based system used in earlier versions of the automated fine-tuning tool are that it is capable of complex reasoning instead of merely concatenating rules, and that it can handle uncertainty and allows to be fine-tuned by learning from data.

To develop into a mature to a commercial product, the prototype should be tested extensively on real patients with different types of poor hearing ability. Ongoing research and development activities include the further optimisation of the model. One particular aspect that will be targeted is the effect of small changes in the values of the parameters. Given that the goal is to improve the accuracy and the precision of the current model, each parameter should be presented as a continuous variable. This will require the development of new algorithms for inference and for learning the parameters of the network from data. Future work also includes the exploration of configurations to further improve the user's performance by using one of the most successful models for reinforcement learning (partially-observable Markov decision processes, POMDPs, an extension of Bayesian networks and influence diagrams). Turning a big cochlear implant speech processor fitting model into a POMDP is a tremendous scientific challenge that is, however, worth facing, since the power of such a model would exceed by far the cognitive capabilities of any human expert, and consequently its capability to determine the optimal sequence of tests and parameter adjustments would outperform significantly the expertise of the best human programmers of cochlear implants.

3. Otocude: A desktop electro-acoustic test box for cochlear implant audiological implant audiological testing

Background

The third main goal of the project was to develop an electro-acoustical test box with ideal conditions for the measuring of psycho-acoustic outcomes with cochlear implants. The particular requirements of this test box included an adequate acoustical insulation to insure precise test results and both digital-analog conversion and amplification of the test signal. Therefore, this box should be able to:

(i) exclude external noise (acoustic insulation); and
(ii) to produce an exact acoustic replicate of an electric input signal (a simulation of ANSI Type 1 free field testing).

Main scientific and technological results

Technical requirements and design

During the first four months of the project, an inventory was made of technical requirements of the test box describing the norms and necessary specifications for the test boxes. Also a list of materials to be used to build the first prototype of the test box was provided. The requirements regarding the acoustic insulation of the test box focused on efficiency within the set limits of weight, size and cost price. The norms were deduced from ISO8253-3 2009 §4.3 and §7 describing signal-to-noise ratios and ambient SPL in the test room. Test environments in a normal household situation were taken as a reference for the insulation of the test. In addition to acoustic insulation, other requirements such as consumer safety, choice of loudspeaker, microphone, sound cards, amplifiers and target weight have also been taken into account. A Box-in-box design was chosen in order to allow a high attenuation. The inner- and the outer-box were decoupled from each other so that the inner-box remain floating at all time.

Building the first prototype of the test box

The first prototype of the test box was built in agreement with the previously described requirements and design parameters. The box itself consists of an acoustics section, an electronics- and speakers-section, tunnels for headpiece and programming cables, and an electric feed-through panel section.

Testing revealed that the box worked as expected. The soundcard feeds the amplifier and captures the sound out of the speaker simultaneously. The computer software was able to record the produced sound. The measured isolation characteristics of the box, plus the accepted ambient noise level described in ISO 8253-2, were compared to the household norm. Based on this testing, a change of materials was proposed to even out the insulation.

Adapting the prototypes

The building of the inner-box out of the Merfoplex material used in the first prototype, appeared to be rather work intensive. It was reasoned that an inner-box made out of steel would be more cost-efficient.

Also, it was predicted that heat development within the box could become problematic. Therefore, the new prototypes used a smaller power supply and the side-wall of the box was replaced with an aluminium plate on which the electronic compartments were bolted.

Two of the three copies that were made of the second prototype went on tour. The boxes were thoroughly tested by universities and institutes in Belgium, the Netherlands, Germany, France, Morocco and Algeria. The changes for the new prototypes were partly based on the user feedback and on our own testing and measurements. Other changes to the box were implemented to improve the acoustic capabilities of the box, to reduce the eventual production costs of the box, and to make maintenance of the box easier to execute.

OtoCube: the final commercial desktop audiological test-booth for cochlear implant users

The final commercial product resulting from the above described research and development activities is an desktop electro-acoustical test box that enables audiologist to do precise measurements and test on cochlear implants using a soundproofed 'calibrated' box in which the CI sound processor and microphone are placed in a small acoustical camber and connected to the patient with an special extension cable to the transmitter system of the cochlear implant. With the test stimuli presented to the CI processor in the acoustically insulated OtoCube CI-patients can then be tested on their perception of 'speech in noise' in optimal testing conditions an excellent precision. The final product can be combined with the audiological test software OTOSpeech described above, and with the fitting assistant software (FOX) that enables the semi-automated fitting of cochlear implant speech processors.

OtoCube is designed to produce several acoustical stimuli an small anechoic test room in the box. With class D amplifier and matching loudspeaker in the box it can replicate an exact acoustic sound field of the electric input signal which is produced by the software on a laptop and digitally send to the 24 bit soundcard in the box. Before running the audiological test, the box is calibrated with the monitoring class-1 microphone and the known factory calibration of the complete measuring system (soundcard - amplifier - loudspeaker - anechoic test room specifications - microphone). The values of the acoustical stimuli in the box are therefore exactly the same as those intended and produced by the audiological test software with respect to intensity, frequency and phase.

Potential impact:

A scientific survey (Shield, 2006) shows that the regular use of an adequate hearing aid increases the chance of keeping hearing impaired patients communicatively, socially, psychically, physically healthy and economically active. In the EU, only about 17 % of the 55 million of people with hearing loss utilises a hearing device on a regular basis. To ensure that those who need hearing aids obtain and use them to a maximum benefit, the development and constitution of adequate hearing rehabilitation should meet the challenges posed by the current hearing impaired populations.

The OPTIFOX project has addressed a number of typical challenges to the non-use or the under-use of hearing devices:

(i) poor benefit and listening experiences with hearing aids: over 16 % of hearing aided patients report that their hearing aids are in the drawer (Kochkin, 2000) mainly due to the feeling that their bene?t is minimal or non-existent, especially in difficult listening situations such as in the presence of background noise;
(ii) changing needs of the hearing impaired individual: for hearing impaired individuals who have been hearing well for years, renewed access to sound at regular intensities may be uncomfortable. To reduce the risk of non-use of the hearing device due to auditory overstimulation, a gradual increase of auditory input by stepwise adjustment of the hearing device may be necessary;
(iii) deaf patients who get a cochlear implant need to be evaluated by an audiologist to have their speech processor 'fitted', i.e. to be equipped with a set of instructions that defines the specific characteristics of the code used to stimulate the electrodes of the implanted array. For the last few years, experts in the field have expressed the need for a new fitting process that optimises the patient's hearing in a more efficient and accurate way based on measured psycho-acoustic feedback from the implant user obtained from testing. More than often, the high number of man-hours required to obtain the necessary reliable hearing assessment data which serve as input to fine-tune the many electrical device parameters is an impediment to multiple fitting sessions and hence also to the optimal use of this hearing device.

Together, the three products resulting from the OPTIFOX project (the OtoSpeech perception test battery, the semi-automated fine tuning of CI speech processors and OtoCube, the desktop testbox) succeed in partially overcoming the challenges mentioned above.

Socio-economic impact

The outcomes of the OPTIFOX project consist in a set of innovative products that aim at the hearing rehabilitation of individuals with different types and degrees of hearing losses. Currently, there are no straightforward means of calculating the exact gain of adequate rehabilitation in this group of potential beneficiaries. In agreement with a European report of costs that have been calculated for unaided moderate to profound hearing loss (Shield, 2006), a hypothetic estimation of the socio-economic impact of this project can be made by means of:

(i) an approach that measures quality adjusted life years (QALY), a common practice in health economics by which the value allocated to one full quality year of life is adjusted by a factor determining the change of quality of life due to a health effect; and
(ii) the mitigation of hearing impairment related direct costs associated with employment, treatment or education thanks to adequate hearing rehabilitation.

Improvement of QALY

In the literature, unaided hearing loss has been associated with a reduced value on the Health Utility Index ranging from 0.05 to 0.25 a value which is increasing with the degree of hearing loss (HUI = 0.15 for moderate hearing loss and 0.25 for severe to profound hearing loss, Sorri et al., 2001). Mitigated costs of unemployment. Unaided hearing impairment has been claimed to cause an additional unemployment of 10 % over the existing unemployment rates. Based on these measures, the cost to Europe of hearing impairment of all grades is EUR 284 billion (reference year of estimation 2004, Shield, 2005), and EUR 224 billion for the EU.

Hearing: a vector for psycho-social well-being

Important benefits of the projects are to be expected for two particular groups of end-users of its products: elderly adults and congenitally deaf children.

The ageing population
A number of European studies show that 53 % of elderly persons over 70 years of age have some degree of hearing loss and that this portion increases up to 90% by the time they are 80 (Davis, 1995; Uimonen et al., 1999; Herbst and Humphrey, 1980). As such, hearing impairments are the third most prevalent chronic condition in the elderly. Socio-demographic trends suggest a further growth in magnitude and incidence of this handicap (Abend and Chen, 1985). It is expected that in 20 years' time, there will be approximately 100 million hearing impaired people in Europe.

Children
It is well known that children with hearing impairments have little access to the speech signal, making them particularly prone to deficits in the development of their oral language. As strong language skills are crucial to literacy development, which is in turn the foundation for further academic success (Chute and Nevins, 2003, Geers, 2003), the negative effect of hearing impairment on the developing child does not stop at the level of spoken language itself: hearing impaired children are known to have more difficulties to build up the strong oral language skills necessary for mainstreaming, for reading and writing tasks, and to access mathematical concepts (Spencer et al., 2003, Brannon, 2005).

Hearing loss is known to be related to psycho-social and emotional changes, as it affects the communicative abilities of the individual and may thus cause social withdrawal and eventually isolation. Recent investigations have shown that even mild hearing impairment increases the risk for psychotic experiences in normal aging populations (van der Werf, van Boxtel et al., 2007).

There is, however, overwhelming scientific evidence that adequate rehabilitation provides significant improvement of the communication skills of the hearing impaired: several studies report benefits in various types of conversational and listening situations (at work, at home, in the street, at social gatherings, etc. Robillard and Gillain, 1996; Golabek et al., 1988; Dillon et al., 1999). Indirectly, the sustained use of hearing aids is also reflected in improvements of various factors of psychological (self-concept and self-esteem, Harless and McConnell, 1982) and physical and social functioning (Crandell, 1998).

As such, we expect the three OPTI-FOX products to provide an improvement of the overall quality of life of the hearing aid user (more satisfaction with life, less depressions, less negativity in personal relationships, etc., Birdges and Bentler, 1998; Kochkin, 2002; Kochkin and Rogin, 2000).

Conclusion: overall potential impact of the project

The project clearly addresses objectives directly related to quality of life, health, working conditions and employment. As it is expected that the number of hearing impaired persons is ever more increasing (amongst youngsters and elderly people), the three products resulting from this R&D project, i.e. an advanced speech perception test tools, a semi-automated method for fine-tuning cochlear implant speech processors, and the desktop testbox, can evidently lead to improved life, health and working conditions of the hearing impaired users concerned.

New test methodologies such as psycho-acoustic measurements of hearing and the objective evaluation of patient responses thanks to the introduction of ASR techniques are expected to lower future social security costs and to improve the psycho-social and emotional well-being and the communicative abilities of the hearing impaired individual. Importantly, it is expected that hearing device users will be able to interact with their environment in much more natural ways. The optimised use of a hearing device will assist the hearing impaired in carrying out activities of daily living and may thus prevent them from social withdrawal and eventually isolation.

Exploitation of results

During the project, three products have been developed of which the first two are now ready for commercialisation:

(i) OTOSpeech, a test battery to measure auditory perception skills in difficult listening situations;
(ii) OtoCube, a desk-top test-box for hearing aid evaluation that is adapted to the individual needs of the cochlear implant user; and
(iii) an optimised automated fitting engine for cochlear implant speech processors.

The three products have particular features that aim at providing a solution for current needs in the audiological practice:

1. OTOspeech, a semi-automated speech (in noise) test

The first product which has been labeled OtoSpeech is an evaluation tool to measure functional hearing in quiet and in noise capacities, with automated speech error analysis. Current tests of hearing abilities in audiological centres generally involve the administration of an audiogram and word discrimination and speech reception tests in quiet. Although such tests are without any doubt useful components of the diagnostic audiological test battery, they do not provide information about the patient's functional hearing. In addition, they are not representative for the language, they do not control for the individual patient's linguistic skills or capacities, they often lack intensity control, etc.

The product provides a solution for a number of hitherto unmet needs:

- There is a huge need for well calibrated speech tests using linguistic material which is representative for the language, which belongs to the daily linguistic portfolio of the patients and which can be administered in an intensity-controlled and well calibrated way.
- Only few standard audiometry tests measure the ability to understand speech in noise. However, this knowledge is necessary to make realistic predictions of improvement of hearing in day-to-day conversations.
- Ccurrent commercially available speech perception tests have important limitations, as speakers of other languages or dialects than the ones available cannot be adequately tested. This is especially striking for Europe, with a population of 499 million inhabitants and a large number of official linguistic communities (23 official languages and 138 linguistic minorities according to Euromosaic studies I and II) of which none are represented in the commercially available tests.

OtoSpeech aims at tackling these problems and is therefore highly innovative by the following actions:

- Using a speech audiometry test specific for the individual patient: an individual profile is created based on a digital text, selected by the patient and used as a source of carefully selected, randomised test words (a subset of the source). Therefore the audiometry test is at the individual patients' level, well known to him / her and in his / her own language. OtoSpeech therefore provides a test method that enables the tester to use test materials for speakers / listeners with any native language.
- Patients can be tested in different noise conditions that are representative of day-to-day communication (including different types of background noise such as steady speech noise, spectral modulated speech noise and temporal modulated speech noise).
- Automated scoring of word repetition is integrated in the test, yielding not only absolute scores but also discretionary scores for a number of spectral bands.
- The scores will be fed back to the intelligent agent FOX that will use them to optimise the cochlear implant processor map settings in order to improve the general response profile. The spectral band information is potentially useful to improve the fitting of specific electrodes of frequency bands of cochlear implants.

2. OtoCube: A desktop electro-acoustical test box for measurement of psycho-acoustic outcomes with cochlear implants

The second product resulting from the project was labelled OtoCube. It is a soundproof calibrated boot consisting of a small acoustical compartment with a loudspeaker and a microphone. In this acoustical compartment, the cochlear implant sound processor will be inserted during testing and a special extension cable connects the processor to the transmitter system of the cochlear implant itself. The testbox also comes with an electronical compartment. As OtoCube comes with a class D amplifier and matching loudspeaker, it can replicate an exact acoustic sound field of the electric input signal.

OtoCube is expected to provide a solution for the need to physically disconnect the speech processor from the other parts of the cochlear implant for test purposes. This need results from the current state of the art of the testing conditions for cochlear implant users. To date, these require sound-treated rooms, dedicated audiometric devices and high end HiFi products. The typical cost of a fully equipped room varies from EUR 20000 - to EUR 50.000. Fully equipped rooms are rare available outside most Western countries. When available, they are often heavily booked for routine clinical work and it may be difficult to occupy them for cochlear implant testing purposes.

OtoCube provides a direct solution to this particular need, as it can be used as a desktop sound-treated box. The product weighs only 15.6 kg and measures 45.7 x 33 x 21,3 cm with a very robust metal casing (Peli case). Its insulation characteristics for ambient noise comply with the ISO norms for clinical audiological practice and are therefore suitable to be used in normal clinical rooms. It reduces the ambient noise in the test box to acceptable levels as described in the American National Standard (ANSI), British Standard (BSI), European Standard (EN) and International Standard (ISO). More in particular, OtoCube allows in situ measurements of the insulation levels at frequencies ranging from 125 - 8000 Hz. It is extremely accurate at frequencies ranging from 35 Hz to 16 000 Hz.

OtoCube's additional features include: a marked positioning for any of the commercially available CI processors, and connections and feed-troughs which allow input of information coming from clinical audiometers and soundcards and from electronic fitting interfaces with the firmware of the four commercial cochlear implant companies (Cochlear, Advanced Bionics, Med-El, Neurelec).

OPTI-FOX, an optimised automated fitting software tool for cochlear implant speech processor

The external speech processor of a cochlear implant needs to be adjusted to yield optimal auditory perception. This fitting process consists in setting a number of electrical parameters (a so-called 'map'), for which the optimal values must be found in the deaf individual. Accurately adjusted processors generally result in better outcomes with respect to speech perception, but are very time-consuming. The main shortcomings of the current processor fitting procedures lie in the fact that they are dependent on comfort indications by the patient, on the one hand, and on the expertise of the audiologist or other personnel performing the measurements and adjustments, on the other hand. Although it seems reasonable to rely on feedback provided by the patient, comfort indications, especially when obtained from patients with no or little auditory experience, may vary according to the methodology employed, instructions to the patient, etc. The expertise of the fitting personnel generally builds on training provided primarily by the cochlear implant manufacturers, but there exists no standardised methodology, making it very difficult to verify the quality of the fitting process.

Anecdotal reports from clinical specialists working with cochlear implant manufacturers suggest that patients with grossly inappropriate maps are occasionally encountered. In sum, is has been argued that after more than 20 years of cochlear implantation, fitting the implant processor still builds on setting a limited number of electrical parameters based on behavioural responses related to the level of detection and some level of comfort, a time-consuming operation of which the reliability can be questioned.

For the last few years, experts in the field have expressed the need for a new fitting process that optimises the patient's hearing in a more efficient and accurate way. The needs are met by the OPTI-FOX product in the following way:

- by providing a (semi)-automated fitting procedure by which a larger number of parameters may be adjusted, based on measured psycho-acoustic feedback from the implant user (outcome-driven approach);
- by introducing techniques known in artificial intelligence to optimise the earlier developed fitting-to outcome expert (FOX) fitting model.

Within the next couple of months the optimised fitting software is expected to be available on the market as a clinical application which is capable of autonomously analysing input data, identifying those implant parameters that are not yet optimally set and semi-automatically adjusting these parameters to obtain an improved hearing performance.

Major dissemination activities

Over 70 different dissemination activities have been organised during the project. These include, amongst other:

(i) a large number of presentations targeting the scientific community (workshops and conferences), the media (interviews), the industry (hearing aid and cochlear implant companies), etc;
(ii) scientific publications published both in A-rated journals and the popular press;
(iii) participation at conferences and workshops with research topics closely related to the project;
(iv) oral and poster presentations for the scientific community, targeting different international conferences in various scientific disciplines going from theoretical and applied linguistics, over applied mathematics and artificial intelligence to audiology and otorhinolaryngology;
(v) the transfer of project-related scientific and technological knowledge to graduate students (research schools, dedicated research meetings at the participating RTD institutes, BA-, MA- and PhD theses related to the project's research topics);
(vi) the organisation of workshops and a final international debate;
(vii) the dissemination of preliminary and final results of the project through a dedicated project website.

Project website: http://otoconsult.com/opti-fox/Default.aspx