Final Report Summary - LISTEN (LISTening Effort in the European population: a New innovative programme of research and training)
Approximately one third of the world’s population over 65 years of age is affected by a hearing disability. People’s ability to hear and understand speech plays a major role in daily life communication situations, especially when the auditory input is altered by hearing impairment. In particular, hearing-impaired listeners often report difficulties to understand speech in background noise and that listening is very hard, tiring, effortful or stressful even when audibility is provided. Hearing loss does not only impair people in communication situations, it also affects their general health and participation in society negatively. It is therefore essential that research aims to better understand the underlying mechanisms that modulate effortful listening and to develop hearing aid processing strategies that improve speech recognition and reduce listening effort.
Listen607373 aimed 1) to train two dedicated ESR’s to become experts in an emerging field combining cognitive, psychological and physiological perspectives on hearing aid technology; by 2) investigating which current and new hearing aid technologies decrease listening effort; 3) to further develop the method of pupillometry by examining additional pupil reflex parameters than the ones previously used in Audiological research. Two ESRs were recruited who received extensive training in scientific, technological and transferrable skills and who participated in dissemination and outreach activities.
In summary, the scientific work performed by ESR1 focused on the influence of individual characteristics (age, hearing status, cognitive abilities) and external factors (amount of speech degradation, type of background masker, hearing aid characteristics) on the resulting speech intelligibility and listening effort as assessed using pupillometry. Pupillometry is the measurement of the pupil size of the eye and reflects the amount of mental effort evoked by a (listening) task. First, a systematic review was conducted to identify the state-of-the art and the available evidence related to the impact of hearing impairment and hearing aid amplification on listening effort. Systematic evidence across different measurement methods suggested that hearing impairment increases listening effort during speech perception. There was no scientific evidence suggesting that hearing aid amplification can reduce listening effort.
Additionally, two experimental studies were carried out in which commonly applied speech recognition measures were combined with measuring the pupil dilation response as an indication of listening effort. In the first experimental study, speech recognition and pupil dilation response data were obtained for normal-hearing and hearing-impaired listeners in a variety of degraded listening conditions. One would expect to see larger pupil dilation in listeners with hearing impairment than normal hearing listeners during speech understanding in noise, as larger pupil dilation is associated with higher level of listening effort. However, the results indicate a more complex relationship. The allocation of listening effort in everyday listening conditions differs between normal-hearing and hearing-impaired listeners, but importantly, the effect of hearing status interacted with degradation level. Namely, in difficult listening conditions, listeners with hearing impairment show smaller pupil dilation responses than listeners with normal hearing. In less challenging conditions, in general, the opposite pattern of results is observed. This means that the difference between listeners may vary depending on the listening conditions assessed. This has important implications for the interpretation of pupillometry data in future academic or industrial studies when assessing individual differences and the effect of devices like hearing aids. It underlines the need for the assessment of speech intelligibility and listening effort across a wide range of listening conditions.
In a second experimental study, experienced hearing aid users participated in speech recognition tests in which pupil response measures were again obtained. Participants used commercial hearing aids and the effect of different signal processing strategies on intelligibility and effort were evaluated. The results suggest that commercial hearing aids can help to improve speech recognition and thereby reduce listening effort in a variety of degraded listening conditions. Improved speech recognition in challenging conditions may improve hearing impaired listener’s ability for social interactions and their overall life quality.
The research of ESR2 focused on the relationship between autonomic nervous system functioning, as assessed by pupil measurements, hearing loss and need for recovery. There is mounting evidence showing that for listeners with hearing impairment, listening is more effortful than for their normally-hearing peers. People with hearing impairment are likely to experience higher levels of fatigue due to effortful listening in daily communication. This hearing-related fatigue might not only constrain their work performance, but also result in withdrawal from major social roles. Therefore, it is important to understand the relationships between fatigue, listening effort, and hearing impairment. In a first study, as systematic review was performed to assess 1) the existing evidence for the Pupil Light Reflex (PLR) being a sensitive method to evaluate parasympathetic dysfunction, 2) the relationship between hearing impairment and parasympathetic activity and 3) evidence of possible connections between hearing impairment and PLR. The results indicate that the PLR is a sensitive tool to assess parasympathetic functioning. Maximum constriction velocity and relative constriction amplitude appeared to be the most sensitive parameters.
Measurements of the pupil dilation response during speech recognition tasks have been widely used as an indication of listening effort. As described above, studies have shown interactions between hearing status and listening condition on the task-evoked pupil dilation response. The aim of the first experimental study performed by Wang was to further assess this finding by investigating the relationship between daily-life need for recovery and the pupil dilation response to speech recognition in noise. Results indicated that need for recovery and hearing ability contributed independently to the pupil dilation response. Higher levels of need for recovery and worse hearing acuity were associated with smaller pupil dilation. In a second experimental study, the pupil dilation response was measured in dark and light conditions, which allows the disentanglement of the parasympathetic and sympathetic contribution to the response. Interestingly, a lower pupil dilation response was observed in listening with hearing loss, but only in a bright test condition. In darkness, the differences between the groups disappeared. This suggested that the origin of the difference between individuals relate to parasympathetic activation. Similarly, the relationship between need for recovery and the pupil dilation response was only observed in light conditions, indicating that these are also related to parasympathetic nervous system activation. This system is known to govern the ‘rest and digest’ response of the human body. In order to gain more insight into how parasympathetic nervous system activity may differ between normally-hearing and hearing-impaired groups, we developed a sensitive measurement method of parasympathetic nervous system activity in a follow-up study. In this study, a PLR measurement method was evaluated that uses a computer screen to generate the light stimuli. Validity of the screen set-up has been tested by comparing the results with the PLR generated by a more conventional LED method.
In conclusion, the research performed in the context of LISTEN provides a better understanding on the underlying mechanism of the interaction between listening effort, task demands, fatigue and hearing impairment. It also has provided essential knowledge on the components of listening effort and the future development of hearing aid processing. LISTEN also resulted in the development of equipment and software to measure the pupil response, including the light reflex. A tutorial related to the application of pupillometry in hearing science is currently being written. The tutorial and also a review of the current state of knowledge related to the use of pupillometry to assess auditory processing will be part of a Special Issue in Trends in Hearing that will be published in 2018. This special issue is the product of the International Pupillometry in hearing science Workshop organized by the LISTEN team in Sept. 2017.
Advancement of knowledge and broader implications for future academic and industrial work focusing on the assessment of listening effort and/or any work aiming to assess the PLR and/or interpret pupil response data include: 1) the insight that effort measurements need to include a broad range of performance levels. 2) The finding that the effect of hearing status on effort can be observed even in conditions where performance levels are high. 3) The finding that around 50% intelligibility, listeners may start to give up listening in some of the cases. This may have broad implications for speech perception tests applied in academic and clinical settings that often target such challenging listening conditions. 4) The methodological advancements made in the assessment of the PLR.
The socio-economic impact of LISTEN relates to the identification and further development of a new type or dimension of outcome measures of hearing rehabilitation: listening effort. Applying this measure in the development of new hearing rehabilitation techniques will, in the long term, contribute to the development of hearing aids that optimize performance while minimizing listening effort. This will have huge impact on the daily life of millions of hearing-impaired persons in Europe by improving their daily life communication and listening experiences.
Project website: http://www.ac-vumc.nl/onderzoek/cognition.htm#_D3
Listen607373 aimed 1) to train two dedicated ESR’s to become experts in an emerging field combining cognitive, psychological and physiological perspectives on hearing aid technology; by 2) investigating which current and new hearing aid technologies decrease listening effort; 3) to further develop the method of pupillometry by examining additional pupil reflex parameters than the ones previously used in Audiological research. Two ESRs were recruited who received extensive training in scientific, technological and transferrable skills and who participated in dissemination and outreach activities.
In summary, the scientific work performed by ESR1 focused on the influence of individual characteristics (age, hearing status, cognitive abilities) and external factors (amount of speech degradation, type of background masker, hearing aid characteristics) on the resulting speech intelligibility and listening effort as assessed using pupillometry. Pupillometry is the measurement of the pupil size of the eye and reflects the amount of mental effort evoked by a (listening) task. First, a systematic review was conducted to identify the state-of-the art and the available evidence related to the impact of hearing impairment and hearing aid amplification on listening effort. Systematic evidence across different measurement methods suggested that hearing impairment increases listening effort during speech perception. There was no scientific evidence suggesting that hearing aid amplification can reduce listening effort.
Additionally, two experimental studies were carried out in which commonly applied speech recognition measures were combined with measuring the pupil dilation response as an indication of listening effort. In the first experimental study, speech recognition and pupil dilation response data were obtained for normal-hearing and hearing-impaired listeners in a variety of degraded listening conditions. One would expect to see larger pupil dilation in listeners with hearing impairment than normal hearing listeners during speech understanding in noise, as larger pupil dilation is associated with higher level of listening effort. However, the results indicate a more complex relationship. The allocation of listening effort in everyday listening conditions differs between normal-hearing and hearing-impaired listeners, but importantly, the effect of hearing status interacted with degradation level. Namely, in difficult listening conditions, listeners with hearing impairment show smaller pupil dilation responses than listeners with normal hearing. In less challenging conditions, in general, the opposite pattern of results is observed. This means that the difference between listeners may vary depending on the listening conditions assessed. This has important implications for the interpretation of pupillometry data in future academic or industrial studies when assessing individual differences and the effect of devices like hearing aids. It underlines the need for the assessment of speech intelligibility and listening effort across a wide range of listening conditions.
In a second experimental study, experienced hearing aid users participated in speech recognition tests in which pupil response measures were again obtained. Participants used commercial hearing aids and the effect of different signal processing strategies on intelligibility and effort were evaluated. The results suggest that commercial hearing aids can help to improve speech recognition and thereby reduce listening effort in a variety of degraded listening conditions. Improved speech recognition in challenging conditions may improve hearing impaired listener’s ability for social interactions and their overall life quality.
The research of ESR2 focused on the relationship between autonomic nervous system functioning, as assessed by pupil measurements, hearing loss and need for recovery. There is mounting evidence showing that for listeners with hearing impairment, listening is more effortful than for their normally-hearing peers. People with hearing impairment are likely to experience higher levels of fatigue due to effortful listening in daily communication. This hearing-related fatigue might not only constrain their work performance, but also result in withdrawal from major social roles. Therefore, it is important to understand the relationships between fatigue, listening effort, and hearing impairment. In a first study, as systematic review was performed to assess 1) the existing evidence for the Pupil Light Reflex (PLR) being a sensitive method to evaluate parasympathetic dysfunction, 2) the relationship between hearing impairment and parasympathetic activity and 3) evidence of possible connections between hearing impairment and PLR. The results indicate that the PLR is a sensitive tool to assess parasympathetic functioning. Maximum constriction velocity and relative constriction amplitude appeared to be the most sensitive parameters.
Measurements of the pupil dilation response during speech recognition tasks have been widely used as an indication of listening effort. As described above, studies have shown interactions between hearing status and listening condition on the task-evoked pupil dilation response. The aim of the first experimental study performed by Wang was to further assess this finding by investigating the relationship between daily-life need for recovery and the pupil dilation response to speech recognition in noise. Results indicated that need for recovery and hearing ability contributed independently to the pupil dilation response. Higher levels of need for recovery and worse hearing acuity were associated with smaller pupil dilation. In a second experimental study, the pupil dilation response was measured in dark and light conditions, which allows the disentanglement of the parasympathetic and sympathetic contribution to the response. Interestingly, a lower pupil dilation response was observed in listening with hearing loss, but only in a bright test condition. In darkness, the differences between the groups disappeared. This suggested that the origin of the difference between individuals relate to parasympathetic activation. Similarly, the relationship between need for recovery and the pupil dilation response was only observed in light conditions, indicating that these are also related to parasympathetic nervous system activation. This system is known to govern the ‘rest and digest’ response of the human body. In order to gain more insight into how parasympathetic nervous system activity may differ between normally-hearing and hearing-impaired groups, we developed a sensitive measurement method of parasympathetic nervous system activity in a follow-up study. In this study, a PLR measurement method was evaluated that uses a computer screen to generate the light stimuli. Validity of the screen set-up has been tested by comparing the results with the PLR generated by a more conventional LED method.
In conclusion, the research performed in the context of LISTEN provides a better understanding on the underlying mechanism of the interaction between listening effort, task demands, fatigue and hearing impairment. It also has provided essential knowledge on the components of listening effort and the future development of hearing aid processing. LISTEN also resulted in the development of equipment and software to measure the pupil response, including the light reflex. A tutorial related to the application of pupillometry in hearing science is currently being written. The tutorial and also a review of the current state of knowledge related to the use of pupillometry to assess auditory processing will be part of a Special Issue in Trends in Hearing that will be published in 2018. This special issue is the product of the International Pupillometry in hearing science Workshop organized by the LISTEN team in Sept. 2017.
Advancement of knowledge and broader implications for future academic and industrial work focusing on the assessment of listening effort and/or any work aiming to assess the PLR and/or interpret pupil response data include: 1) the insight that effort measurements need to include a broad range of performance levels. 2) The finding that the effect of hearing status on effort can be observed even in conditions where performance levels are high. 3) The finding that around 50% intelligibility, listeners may start to give up listening in some of the cases. This may have broad implications for speech perception tests applied in academic and clinical settings that often target such challenging listening conditions. 4) The methodological advancements made in the assessment of the PLR.
The socio-economic impact of LISTEN relates to the identification and further development of a new type or dimension of outcome measures of hearing rehabilitation: listening effort. Applying this measure in the development of new hearing rehabilitation techniques will, in the long term, contribute to the development of hearing aids that optimize performance while minimizing listening effort. This will have huge impact on the daily life of millions of hearing-impaired persons in Europe by improving their daily life communication and listening experiences.
Project website: http://www.ac-vumc.nl/onderzoek/cognition.htm#_D3