Final Report Summary - ECHOUTCOME (European consortium in healthcare outcomes and cost-benefit research)
Executive summary:
The 'European consortium in healthcare outcomes and cost-benefit research' (ECHOUTCOME) is an interdisciplinary European research platform with the aim of assessing methodological properties of healthcare outcome used in health technology assessment (HTA). Led by Dr Ariel Beresniak (Data Mining International, Geneva, Switzerland) acting as scientific coordinator, the first general objective of the ECHOUTCOME project was to study how the 27 European health systems organisations use health outcomes in the frame of HTA. The second general objective was to scientifically assess the robustness of key health indicators used in Europe, such as the quality-adjusted life years (QALY) indicator. Finally, the third general objective was to propose guidelines and recommendations for conducting scientifically robust cost-effectiveness assessment in the frame of HTA.
The systematic scientific methodologies selected by the ECHOUTCOME initiative led to three major advancements in HTA improving the knowledge about the use of key health outcomes in Europe:
(1) European multi-criteria comparisons of health systems organisations and use of health outcomes in the frame of HTA in the 27 European member states. The results of this first multi-criteria assessment conducted on European Union (EU) health care systems across the 27 Member States thru three categories of stakeholders (government, academia, industry) showed a large spectrum of various understanding about HTA basic concepts according to stakeholders categories, suggesting that achieving the overall mission of HTA agencies would require continued educational efforts, as well as the full support from health authorities towards building scientific excellence to guide health care decision making in Europe. This would also suggests to develop and implement more robust HTA methodologies that would allow to establish best HTA practices for the optimal and timely allocation of limited resources.
(2) European validation test of the scientific robustness of the QALY indicator in health decision making
QALY has been proposed in the 80s as a standard metric in cost-utility assessment but is currently fuelling an active international debate about ethical and equity issues, leading few countries such as the United States of America (USA) and Germany to officially reject the use of such indicator for health decision making. On the other hand, HTA agencies such as National Institute for Health and Care Excellence (NICE) in the United Kingdom (UK), PBAC in Australia and CADTH in Canada are still requiring cost-utility studies expressed in costs / QALY to assess how an innovative products could be reimbursed and then access to the market. However, very few studies have tested the validity of the strong methodological assumptions of the QALY indicator. The ECHOUTCOME project carried out the largest ever experimental study specifically dedicated to test the scientific robustness of the QALY indicator. The experiment has been conducted in 1 361 subjects from France, Italy, Belgium and the UK, which expressed their preferences about various combinations of time duration and health states. Results indicate that the preference of respondents were not in line with the theoretical assumptions of the QALY approach, leading to divergent costs / QALY estimates and suggesting that Health decisions based on costs / QALYs would be flawed.
(3) European guidelines for conducting cost-effectiveness assessment in HTA
Reimbursement recommendations being primarily shaped by national or regional HTA bodies across the EU-27 Member States, this has led to different approaches, guidelines, interpretations and decisions. For instance, health authorities in France (HAS), Germany (IQWIG) and in the UK (NICE) are just a few examples illustrating how HTA processes deployed in Europe use different HTA approaches and methods, including the use of the QALY indicator potentially leading to inconsistent and divergent decisions. The ECHOUTCOME initiative proposes 5 recommendations for conducting robust cost-effectiveness to support health decision making:
(i) clear distinctions between cost-benefit, cost-effectiveness and cost-utility analyses should be established;
(ii) QALY assessment for health decision making should be abandoned;
(iii) cost-effectiveness analyses should be expressed as costs per relevant clinical outcome;
(iv) cost-effectiveness analyses should be validated by an interdisciplinary research team;
(v) cost-effectiveness analyses should use a tool box of various robust modelling techniques, to be selected on the case-by-case basis.
The dissemination of the ECHOUTCOME results and guidelines will contribute to develop innovative HTA methodologies that promise to bring about more transparency, robustness and consistency to assist meaningful, suitable and equitable health care decision-making, with the aim of optimising patient access to innovative medicines and to maximising health outcomes in Europe.
Project context and objectives:
The ECHOUTCOME consortium is led by Dr Ariel Beresniak (Data Mining International, Switzerland), project leader.
The consortium is composed by 8 partners including academic international experts in Health Economics and Outcome Research (Claude Bernard University, France, Free University of Brussels, Belgium and Bocconi University, Italy), the French Society in Health Economics, the CYKLAD Group (a small and medium-sized enterprise (SME) specialised in advanced mathematics, France), Data Mining International (a SME specialised in modelling, Switzerland) and Lyon Ingenierie Project (a SME specialised in European project management, France). This consortium has been created to leverage an extensive collaboration in outcome research which has generated state-of-the-art publications in the field of outcome research methodology. The general objective of the ECHOUTCOME consortium was to study the ability of European health systems to handle Health priorities thru efficient HTA organisations. In this context and during the 30 last years, most of European decision makers used to manage health system by financial regulation, without taking into account equity, quality of care or patient accessibility. Considering the growing population expectation for a better accessibility of innovative technologies, decision makers have now to take into account not only the cost, but also concepts such as health benefits, quality of life and quality of care. Then the robustness of healthcare outcomes is critical to accurately assess technologies and compare European health systems performances.
However, there are a number of possible healthcare outcomes, from the simplest one (i.e. morbidity criteria) to complex ones such as QALY or disability adjusted life years (DALY). Since the choice of the outcomes can directly impact decisions to be made, there is a strong expectation from European HTA agencies to get scientific guidance about the properties and limitations of health outcomes.
Relationship between quality of care with costs, efficiency and accessibility are classically investigated by cost-benefit studies. The 'cost-benefits' terminology is often use as a generic term to describe analyses linking costs and outcomes such as cost-effectiveness analyses (relation between costs and effectiveness expressed in clinical outcome), cost-utility analyses (relation between costs and quality of life expressed in utility values) and 'strict' cost-benefit analyses (relation between costs and health benefit expressed in monetary values).
Because of the complexity of research in health care outcomes and the number of assumptions underlying some approaches such as cost-utility analyses, most of European HTA agencies do not recommend any particular indicator as reference case. However, the National Institute for Heath and Clinical Excellence NICE in the UK has chosen to use the QALY indicator as reference case for cost-utility analyses expressed in costs / QALY. New EU countries tend to be influenced by the NICE approach in the UK in viewing cost-QALY as a useful tool in decision making treatment that is required to yield all these associated benefits.
Various approaches have been proposed to investigate the relationship between costs, quality of care, efficiency and accessibility. Clinical experts are familiar with healthcare outcomes expressed in clinical measures (number of life years saved, remission rate, number of case avoided, etc.). This approach leads to the construction of 'cost-effectiveness analyses' linking costs and outcomes expressed in clinical measures. This approach is robust and clinically meaningful to compare various interventions in the frame of one disease. However, cost-effectiveness analyses are not designed to perform any cross-disease comparisons.
It is the reason why some economists have recommended the use of synthetic and standard healthcare outcomes, which allow performing cross-disease comparisons such as the QALY. 'Cost-utility' studies expressed in costs / QALY are often presented in the frame of 'cost-effectiveness' studies but remain quite different because of the use of utility assessment. Many techniques exist to assess utility outcomes (standard gamble, time trade off, visual analogic scale, utility questionnaire, etc). Each technique is based on a set of underlying assumptions, such as those of the multi-attribute utility theory. When, combined with survival indicator, utility outcomes are aggregated in QALY which allow cross disease comparisons. This explained why the QALY approach has been selected as reference case by HTA agencies in the UK, Canada and Australia in charge to recommend whether new treatments should be publicly funded. For example, on the basis of this calculation, recommendations are made about whether treatments are offered by the UK's National Health Service (NHS). If the incremental cost per QALY (cost of one additional year in perfect health) is below GBP 30 000 then the treatment is usually recommended to be accessible to patients. Whilst Germany has explicitly rejected it, some European countries are currently considering replicating the NICE model. If equity and ethical issues of the QALY approach are well known, very few studies have tested the methodological robustness of this indicator. The issue of their validity and relevance have raised an active international debate, questioning the level of scientific evidence generated in studies using them. It is the reason why some countries do not recommend the use of such indicator in decision making.
1. In USA, health reform legislation prohibited use of cost-per-QALY thresholds.
2. In Germany, the Institute for Quality and Efficiency in Health Care implemented a new system for evaluating the value of medical technologies but rejected the use of QALY for ethical and methodological reasons.
3. Other countries such as France, Spain, and Italy have opted for other approaches. Acknowledging this international controversy, the United Kingdom has proposed that the NICE, which have always recommended the use of cost-per-QALY for economic appraisal of innovative products, will not in the future use such information to make dichotomist yes/no recommendations. A new concept entitled 'Value based pricing' is still in discussion in February 2013 in the UK.
Project results:
QALY has been proposed in the 80s as a standard health outcome measure for conducting cost-utility analyses. This synthetic indicator which combines the quantity of life lived with the quality expressed as a preference score, is currently recommended as reference case by some HTA agencies such as the NICE in the UK. While most of the critics about the QALY approach concern equity and ethical issues, surprisingly, very few studies have tested the basic methodological assumptions supporting the QALY equation so to establish its scientific validity. The results of the ECHOUTCOME project are derived from a large experiment conducted in 1361 healthy subjects from France, United Kingdom, Italy and Belgium. The subjects were asked to express their preferences regarding various hypothetical health states derived from combining different time durations and health status. The S&T results show that the preferences expressed by the respondents were not consistent with the QALY theoretical assumptions, which confirms how the QALY multiplicative model is flawed. The S&T results also explain why the QALY approach can lead to highly divergent costs/QALY estimates and to inconsistent recommendations relevant to providing access to innovative medicines and technologies. The ECHOUTCOME project has confirmed that the preferences suggested by using one utility instrument cannot be reproduced using another (all other parameters being identical). That is, one utility technique instrument may result in bias towards a particular intervention, when another instrument would suggest quite the opposite. This comparison between disease states becomes even more problematic when publishing and utilising QALY league tables. Many authors have urged that considerable caution should be exercised in interpreting such findings because of the non-comparability of methods, the use of inappropriate comparators and the non-generalisability of results.
These experimental results suggest that the hypotheses that found the QALY approach do not correspond to behaviour patterns observed in real populations. It is then fair to question the representativeness of the study population. However, in mathematical logic, a theory is invalid if one single counter example exist. In this case, at least one population has been found not fulfilling with the QALY theory, which confirms that this theory is not validated. The ECHOUTCOME experimentation is the largest ever study specifically dedicated to test the assumptions of the QALY indicator. The results clearly confirm that the assumptions of the QALY multiplicative model are not validated, which explain the inconsistencies of the approach.
European HTA agencies should consider other cost-effectiveness approaches for health care decision-making.
Potential impact:
Using both the descriptive and the experimental approaches, the ECHOUTCOME consortium was able to investigate the robustness of the QALY health indicator recommended by some HTA agencies in Europe, to assess existing cost effectiveness approaches, and to recommend new cost-effectiveness approaches for health decision making.
The consortium delivered new knowledge and new tools in Healthcare Outcome Research, allowing to significantly improve the quality of economic evaluations. ECHOUTCOME specifically focused on identifying existing approaches and recommending relevant and robust new approaches in outcome research. Considering the fact that some Health agencies in Europe such as the NICE in the UK recommend to use cost per QALY as a requirement for population access to innovative health technologies, it was urgent that EU considers the properties and consequences to use one reference case, before spreading or recommending any specific approach in other member states.
This interdisciplinary effort was a unique combination of Outcome Research, Health Economics and mathematics experts, allowing to make major advancements to improve the knowledge in health outcomes and cost-effectiveness for health decision making. The area of the ECHOUTCOME research is of increasing importance in health decision making. The scientific impact is based on the:
- creation of a unique European database of health system organisations,
- generation of new knowledge to better understand properties and limitations of existing healthcare outcome and cost-benefit approaches,
- implementation of specific experimental surveys to test the scientific validity of the QALY indicator,
- recommendations for conducting robust cost-effectiveness approaches.
Surprisingly, the ECHOUTCOME research is the only European research project integrating all these aspects into one consistent research program, which appeared necessary before recommending any particular approaches for health regulation in member states. NICE is probably the most well-known HTA agency in the world, due to a very efficient communication strategy and because the UK is considered as the gatekeeper for access to the market of innovative products in Europe. Then NICE has directly or indirectly been the main reason of the publication of hundreds of cost / QALY studies. By testing the validity of the QALY indicator recommended by NICE, the ECHOUTCOME will have a major economic impact, including the consequences of the choice of one Outcome on the innovation accessibility. The fact that the ECHOUTCOME project has demonstrated that the QALY indicator is not a valid health outcome will be very useful for European public authorities before recommending any particular methodologies.
The ECHOUTCOME research platform will have a major societal impact in the Outcome Research area. It will enhance the competitiveness of each involved research centre, through the synergy between complementary skills and disciplines. We believe that the scientific assessment study of the QALY indicator is going to be considered as a reference case in HTA. In addition, the ECHOUTCOME deliverable proposed alternative robust approaches which could be used to express the added value of new treatments, and then their acceptability. Therefore the ECHOUTCOME project will, though indirectly, contribute to the equity and accessibility of innovation.
In this perspective, it appears obvious that the insights gained via this consortium will be valuable to the Biopharma industry devoted to the development of new treatments.
Thus, the ECHOUTCOME project is contributing to community social objectives by focusing to the properties of Healthcare Outcomes, the acceptability of potential health interventions, and the performance of Health.
Dissemination activities and exploitation of results
Given the nature of the issue addressed by the project, raising stakeholders awareness and understanding is a fundamental ambition of this project. Because of the current international debate about the use of the QALY outcome in HTA, ECHOUTCOME has generated a huge interest by European stakeholders. A dedicated dissemination plan aimed to address the broader research community, European stakeholders including HTA agencies and health Industry that will profit from the project.
Dissemination activities included:
(1) Scientific communications in international congresses: International Society of PharmacoEconomics and Outcome Research (ISPOR) 2010 and 2012 Latin Association for the Advancement of Health Systems (ALASS 2011).
(2) Dedicated presentations to stakeholders: EUCOMED, EFPIA.
(3) Dedicated presentations to other European projects: HEALTH-NCP-NET, EUROSEARCH, FIT FOR HEALTH, EUROREACH.
(4) Announcement of the project initiation to European media.
(5) Organisation of the special conference about QALY controversy.
(6) Announcement of key results to European media.
(7) Distribution of guidelines and recommendations for conducting cost-effectiveness studies to European stakeholders.
(8) Preparation of scientific manuscripts.
(9) Project website.
Audience monitoring of ECHOUTCOME project website in January 2013, showing visit picks at the date of the final conference presenting the key project results is available in the attachment.
Every dissemination activities and publications in the project acknowledged the EU's Seventh Framework Programme (FP7) funding.
Finally, the exploitation of the results will be made during the 3 next years in conjunction with the continuation of dissemination activities at different levels (within and outside the project). This includes contacts and coordination/networking with HTA agencies, scientific organisations, stakeholders, national programs, industrial actors. Importantly is that the ECHOUTCOME key results presented during the final conference organised on 25 January in Brussels have been referenced by numerous general, economic and scientific media including high impact journals such as Nature medicine and The Economist (in press).
List of websites: http://www.echoutcome.eu
The 'European consortium in healthcare outcomes and cost-benefit research' (ECHOUTCOME) is an interdisciplinary European research platform with the aim of assessing methodological properties of healthcare outcome used in health technology assessment (HTA). Led by Dr Ariel Beresniak (Data Mining International, Geneva, Switzerland) acting as scientific coordinator, the first general objective of the ECHOUTCOME project was to study how the 27 European health systems organisations use health outcomes in the frame of HTA. The second general objective was to scientifically assess the robustness of key health indicators used in Europe, such as the quality-adjusted life years (QALY) indicator. Finally, the third general objective was to propose guidelines and recommendations for conducting scientifically robust cost-effectiveness assessment in the frame of HTA.
The systematic scientific methodologies selected by the ECHOUTCOME initiative led to three major advancements in HTA improving the knowledge about the use of key health outcomes in Europe:
(1) European multi-criteria comparisons of health systems organisations and use of health outcomes in the frame of HTA in the 27 European member states. The results of this first multi-criteria assessment conducted on European Union (EU) health care systems across the 27 Member States thru three categories of stakeholders (government, academia, industry) showed a large spectrum of various understanding about HTA basic concepts according to stakeholders categories, suggesting that achieving the overall mission of HTA agencies would require continued educational efforts, as well as the full support from health authorities towards building scientific excellence to guide health care decision making in Europe. This would also suggests to develop and implement more robust HTA methodologies that would allow to establish best HTA practices for the optimal and timely allocation of limited resources.
(2) European validation test of the scientific robustness of the QALY indicator in health decision making
QALY has been proposed in the 80s as a standard metric in cost-utility assessment but is currently fuelling an active international debate about ethical and equity issues, leading few countries such as the United States of America (USA) and Germany to officially reject the use of such indicator for health decision making. On the other hand, HTA agencies such as National Institute for Health and Care Excellence (NICE) in the United Kingdom (UK), PBAC in Australia and CADTH in Canada are still requiring cost-utility studies expressed in costs / QALY to assess how an innovative products could be reimbursed and then access to the market. However, very few studies have tested the validity of the strong methodological assumptions of the QALY indicator. The ECHOUTCOME project carried out the largest ever experimental study specifically dedicated to test the scientific robustness of the QALY indicator. The experiment has been conducted in 1 361 subjects from France, Italy, Belgium and the UK, which expressed their preferences about various combinations of time duration and health states. Results indicate that the preference of respondents were not in line with the theoretical assumptions of the QALY approach, leading to divergent costs / QALY estimates and suggesting that Health decisions based on costs / QALYs would be flawed.
(3) European guidelines for conducting cost-effectiveness assessment in HTA
Reimbursement recommendations being primarily shaped by national or regional HTA bodies across the EU-27 Member States, this has led to different approaches, guidelines, interpretations and decisions. For instance, health authorities in France (HAS), Germany (IQWIG) and in the UK (NICE) are just a few examples illustrating how HTA processes deployed in Europe use different HTA approaches and methods, including the use of the QALY indicator potentially leading to inconsistent and divergent decisions. The ECHOUTCOME initiative proposes 5 recommendations for conducting robust cost-effectiveness to support health decision making:
(i) clear distinctions between cost-benefit, cost-effectiveness and cost-utility analyses should be established;
(ii) QALY assessment for health decision making should be abandoned;
(iii) cost-effectiveness analyses should be expressed as costs per relevant clinical outcome;
(iv) cost-effectiveness analyses should be validated by an interdisciplinary research team;
(v) cost-effectiveness analyses should use a tool box of various robust modelling techniques, to be selected on the case-by-case basis.
The dissemination of the ECHOUTCOME results and guidelines will contribute to develop innovative HTA methodologies that promise to bring about more transparency, robustness and consistency to assist meaningful, suitable and equitable health care decision-making, with the aim of optimising patient access to innovative medicines and to maximising health outcomes in Europe.
Project context and objectives:
The ECHOUTCOME consortium is led by Dr Ariel Beresniak (Data Mining International, Switzerland), project leader.
The consortium is composed by 8 partners including academic international experts in Health Economics and Outcome Research (Claude Bernard University, France, Free University of Brussels, Belgium and Bocconi University, Italy), the French Society in Health Economics, the CYKLAD Group (a small and medium-sized enterprise (SME) specialised in advanced mathematics, France), Data Mining International (a SME specialised in modelling, Switzerland) and Lyon Ingenierie Project (a SME specialised in European project management, France). This consortium has been created to leverage an extensive collaboration in outcome research which has generated state-of-the-art publications in the field of outcome research methodology. The general objective of the ECHOUTCOME consortium was to study the ability of European health systems to handle Health priorities thru efficient HTA organisations. In this context and during the 30 last years, most of European decision makers used to manage health system by financial regulation, without taking into account equity, quality of care or patient accessibility. Considering the growing population expectation for a better accessibility of innovative technologies, decision makers have now to take into account not only the cost, but also concepts such as health benefits, quality of life and quality of care. Then the robustness of healthcare outcomes is critical to accurately assess technologies and compare European health systems performances.
However, there are a number of possible healthcare outcomes, from the simplest one (i.e. morbidity criteria) to complex ones such as QALY or disability adjusted life years (DALY). Since the choice of the outcomes can directly impact decisions to be made, there is a strong expectation from European HTA agencies to get scientific guidance about the properties and limitations of health outcomes.
Relationship between quality of care with costs, efficiency and accessibility are classically investigated by cost-benefit studies. The 'cost-benefits' terminology is often use as a generic term to describe analyses linking costs and outcomes such as cost-effectiveness analyses (relation between costs and effectiveness expressed in clinical outcome), cost-utility analyses (relation between costs and quality of life expressed in utility values) and 'strict' cost-benefit analyses (relation between costs and health benefit expressed in monetary values).
Because of the complexity of research in health care outcomes and the number of assumptions underlying some approaches such as cost-utility analyses, most of European HTA agencies do not recommend any particular indicator as reference case. However, the National Institute for Heath and Clinical Excellence NICE in the UK has chosen to use the QALY indicator as reference case for cost-utility analyses expressed in costs / QALY. New EU countries tend to be influenced by the NICE approach in the UK in viewing cost-QALY as a useful tool in decision making treatment that is required to yield all these associated benefits.
Various approaches have been proposed to investigate the relationship between costs, quality of care, efficiency and accessibility. Clinical experts are familiar with healthcare outcomes expressed in clinical measures (number of life years saved, remission rate, number of case avoided, etc.). This approach leads to the construction of 'cost-effectiveness analyses' linking costs and outcomes expressed in clinical measures. This approach is robust and clinically meaningful to compare various interventions in the frame of one disease. However, cost-effectiveness analyses are not designed to perform any cross-disease comparisons.
It is the reason why some economists have recommended the use of synthetic and standard healthcare outcomes, which allow performing cross-disease comparisons such as the QALY. 'Cost-utility' studies expressed in costs / QALY are often presented in the frame of 'cost-effectiveness' studies but remain quite different because of the use of utility assessment. Many techniques exist to assess utility outcomes (standard gamble, time trade off, visual analogic scale, utility questionnaire, etc). Each technique is based on a set of underlying assumptions, such as those of the multi-attribute utility theory. When, combined with survival indicator, utility outcomes are aggregated in QALY which allow cross disease comparisons. This explained why the QALY approach has been selected as reference case by HTA agencies in the UK, Canada and Australia in charge to recommend whether new treatments should be publicly funded. For example, on the basis of this calculation, recommendations are made about whether treatments are offered by the UK's National Health Service (NHS). If the incremental cost per QALY (cost of one additional year in perfect health) is below GBP 30 000 then the treatment is usually recommended to be accessible to patients. Whilst Germany has explicitly rejected it, some European countries are currently considering replicating the NICE model. If equity and ethical issues of the QALY approach are well known, very few studies have tested the methodological robustness of this indicator. The issue of their validity and relevance have raised an active international debate, questioning the level of scientific evidence generated in studies using them. It is the reason why some countries do not recommend the use of such indicator in decision making.
1. In USA, health reform legislation prohibited use of cost-per-QALY thresholds.
2. In Germany, the Institute for Quality and Efficiency in Health Care implemented a new system for evaluating the value of medical technologies but rejected the use of QALY for ethical and methodological reasons.
3. Other countries such as France, Spain, and Italy have opted for other approaches. Acknowledging this international controversy, the United Kingdom has proposed that the NICE, which have always recommended the use of cost-per-QALY for economic appraisal of innovative products, will not in the future use such information to make dichotomist yes/no recommendations. A new concept entitled 'Value based pricing' is still in discussion in February 2013 in the UK.
Project results:
QALY has been proposed in the 80s as a standard health outcome measure for conducting cost-utility analyses. This synthetic indicator which combines the quantity of life lived with the quality expressed as a preference score, is currently recommended as reference case by some HTA agencies such as the NICE in the UK. While most of the critics about the QALY approach concern equity and ethical issues, surprisingly, very few studies have tested the basic methodological assumptions supporting the QALY equation so to establish its scientific validity. The results of the ECHOUTCOME project are derived from a large experiment conducted in 1361 healthy subjects from France, United Kingdom, Italy and Belgium. The subjects were asked to express their preferences regarding various hypothetical health states derived from combining different time durations and health status. The S&T results show that the preferences expressed by the respondents were not consistent with the QALY theoretical assumptions, which confirms how the QALY multiplicative model is flawed. The S&T results also explain why the QALY approach can lead to highly divergent costs/QALY estimates and to inconsistent recommendations relevant to providing access to innovative medicines and technologies. The ECHOUTCOME project has confirmed that the preferences suggested by using one utility instrument cannot be reproduced using another (all other parameters being identical). That is, one utility technique instrument may result in bias towards a particular intervention, when another instrument would suggest quite the opposite. This comparison between disease states becomes even more problematic when publishing and utilising QALY league tables. Many authors have urged that considerable caution should be exercised in interpreting such findings because of the non-comparability of methods, the use of inappropriate comparators and the non-generalisability of results.
These experimental results suggest that the hypotheses that found the QALY approach do not correspond to behaviour patterns observed in real populations. It is then fair to question the representativeness of the study population. However, in mathematical logic, a theory is invalid if one single counter example exist. In this case, at least one population has been found not fulfilling with the QALY theory, which confirms that this theory is not validated. The ECHOUTCOME experimentation is the largest ever study specifically dedicated to test the assumptions of the QALY indicator. The results clearly confirm that the assumptions of the QALY multiplicative model are not validated, which explain the inconsistencies of the approach.
European HTA agencies should consider other cost-effectiveness approaches for health care decision-making.
Potential impact:
Using both the descriptive and the experimental approaches, the ECHOUTCOME consortium was able to investigate the robustness of the QALY health indicator recommended by some HTA agencies in Europe, to assess existing cost effectiveness approaches, and to recommend new cost-effectiveness approaches for health decision making.
The consortium delivered new knowledge and new tools in Healthcare Outcome Research, allowing to significantly improve the quality of economic evaluations. ECHOUTCOME specifically focused on identifying existing approaches and recommending relevant and robust new approaches in outcome research. Considering the fact that some Health agencies in Europe such as the NICE in the UK recommend to use cost per QALY as a requirement for population access to innovative health technologies, it was urgent that EU considers the properties and consequences to use one reference case, before spreading or recommending any specific approach in other member states.
This interdisciplinary effort was a unique combination of Outcome Research, Health Economics and mathematics experts, allowing to make major advancements to improve the knowledge in health outcomes and cost-effectiveness for health decision making. The area of the ECHOUTCOME research is of increasing importance in health decision making. The scientific impact is based on the:
- creation of a unique European database of health system organisations,
- generation of new knowledge to better understand properties and limitations of existing healthcare outcome and cost-benefit approaches,
- implementation of specific experimental surveys to test the scientific validity of the QALY indicator,
- recommendations for conducting robust cost-effectiveness approaches.
Surprisingly, the ECHOUTCOME research is the only European research project integrating all these aspects into one consistent research program, which appeared necessary before recommending any particular approaches for health regulation in member states. NICE is probably the most well-known HTA agency in the world, due to a very efficient communication strategy and because the UK is considered as the gatekeeper for access to the market of innovative products in Europe. Then NICE has directly or indirectly been the main reason of the publication of hundreds of cost / QALY studies. By testing the validity of the QALY indicator recommended by NICE, the ECHOUTCOME will have a major economic impact, including the consequences of the choice of one Outcome on the innovation accessibility. The fact that the ECHOUTCOME project has demonstrated that the QALY indicator is not a valid health outcome will be very useful for European public authorities before recommending any particular methodologies.
The ECHOUTCOME research platform will have a major societal impact in the Outcome Research area. It will enhance the competitiveness of each involved research centre, through the synergy between complementary skills and disciplines. We believe that the scientific assessment study of the QALY indicator is going to be considered as a reference case in HTA. In addition, the ECHOUTCOME deliverable proposed alternative robust approaches which could be used to express the added value of new treatments, and then their acceptability. Therefore the ECHOUTCOME project will, though indirectly, contribute to the equity and accessibility of innovation.
In this perspective, it appears obvious that the insights gained via this consortium will be valuable to the Biopharma industry devoted to the development of new treatments.
Thus, the ECHOUTCOME project is contributing to community social objectives by focusing to the properties of Healthcare Outcomes, the acceptability of potential health interventions, and the performance of Health.
Dissemination activities and exploitation of results
Given the nature of the issue addressed by the project, raising stakeholders awareness and understanding is a fundamental ambition of this project. Because of the current international debate about the use of the QALY outcome in HTA, ECHOUTCOME has generated a huge interest by European stakeholders. A dedicated dissemination plan aimed to address the broader research community, European stakeholders including HTA agencies and health Industry that will profit from the project.
Dissemination activities included:
(1) Scientific communications in international congresses: International Society of PharmacoEconomics and Outcome Research (ISPOR) 2010 and 2012 Latin Association for the Advancement of Health Systems (ALASS 2011).
(2) Dedicated presentations to stakeholders: EUCOMED, EFPIA.
(3) Dedicated presentations to other European projects: HEALTH-NCP-NET, EUROSEARCH, FIT FOR HEALTH, EUROREACH.
(4) Announcement of the project initiation to European media.
(5) Organisation of the special conference about QALY controversy.
(6) Announcement of key results to European media.
(7) Distribution of guidelines and recommendations for conducting cost-effectiveness studies to European stakeholders.
(8) Preparation of scientific manuscripts.
(9) Project website.
Audience monitoring of ECHOUTCOME project website in January 2013, showing visit picks at the date of the final conference presenting the key project results is available in the attachment.
Every dissemination activities and publications in the project acknowledged the EU's Seventh Framework Programme (FP7) funding.
Finally, the exploitation of the results will be made during the 3 next years in conjunction with the continuation of dissemination activities at different levels (within and outside the project). This includes contacts and coordination/networking with HTA agencies, scientific organisations, stakeholders, national programs, industrial actors. Importantly is that the ECHOUTCOME key results presented during the final conference organised on 25 January in Brussels have been referenced by numerous general, economic and scientific media including high impact journals such as Nature medicine and The Economist (in press).
List of websites: http://www.echoutcome.eu
