Final Report Summary - ECHO (European Collaboration for Healthcare Optimization)
Executive Summary:
Europe’s health systems are struggling with providing accessible, equitable high-standard of care while restraining the costs of health services. Underperforming care and overexposure to health services are therefore important challenges to meet. In order to make the right decisions policy makers, health managers and professionals need to count on the right information.
The basis for providing optimum health services lays onto in depth monitoring and analysis of the equitable access, quality and efficiency patterns in our health systems from a multilevel perspective. Thus, the key task facing the interdisciplinary team of doctors, statisticians, economists and information technology officers enrolled in the ECHO project is to refine a comprehensive set of performance indicators to enable the identification of possible deficits (unwarranted variations in health care performance) in the health systems, and to provide evidence to inform policy and management decisions.
The ECHO project, co-financed by the European Commission, is designed to create a “knowledge” system. At the core of this knowledge system is the development and implementation of a common data warehouse and a series of on-line tools allowing users to explore equitable access, quality and efficiency of healthcare at the hospital, healthcare area, regional and country level. Noticeably, the ECHO tool, an analytical device that will be accessible to pre-registered research scientists and policy makers via website. This online analysis tool is scheduled to be operational for EU stakeholders during the 2nd half of 2014. Finally, methods underpinning ECHO findings and developments will be also available on-line.
ECHO has focused on developing a robust methodology to compare performance across healthcare systems – 1) building valid crosswalks across coding languages in up to 50 performance indicators; 2) creating benchmarks to allow sound international comparison; 3) building maps where national vectors have been gathered for depicting the maps of each country taking into account the meaningful areas relevant for each country. ECHO performance Atlases used this methodology to report unwarranted differences in health systems performance across ECHO countries. Atlas reports on cardiovascular care, lower value care and potentially avoidable admissions have been yielded. These reports feature international and in-country variations in performance, its evolution over-time, the effect of socioeconomic gradient, and provide some policy messages derived from the results.
However, ECHO is a starting community. The strength of ECHO as a routine tool for EU decision-makers will become a reality if ECHO is able to provide: 1) the most up-to-date information (ECHO pilot was based on 2002 to 2009 info); 2) a larger case-mix of countries (namely, healthcare systems) enabling more reliable benchmarking; and 3) a strong critical mass supporting a sustainable and growing research infrastructure that regularly yields robust evidence.
The above raises a set of unique challenges. It is overcoming the above challenges that ECHO is concentrated on. To know more, visit the project website at www.echo-health.eu
Project Context and Objectives:
The context
In the last two or three decades research on health systems and policy research has provided numerous evidence on healthcare performance that could be translated into policy messages such as: a) the place where a person lives determines the exposure to high value or low value care; b) more care is not always better; c) healthcare organizations produced uneven health outcomes, once differences in patients are ruled out; d) learning cascades are determinant in attaining good health outcomes but also, might hold substandard performance; e) access to effective ambulatory care improve health outcomes; f) money does not always become value; g) access to effective and safe healthcare may be determined by socioeconomic status; h) high-quality information may reduce allocative inefficiency; i) populations are facing high opportunity costs associated to the exposure to low value care.; or j) continuity of care is critical in attaining better outcomes in chronic patients .
The European Commission, throughout EU research programmes, has funded many research projects meant to shed light on either rethinking performance or studying how systems perform. [http://www.healthdatanavigator.eu/international-home/eu-and-international-projects]
However, even though some National Agencies have been either producers or early adopters of some of those research findings, the uptake of evidence is still scarce and unequal across EU Member States.
Many reasons have been suggested: 1) lack of consensus on what health systems performance is; 2) the inherent difficulties on implementing findings that are usually coming from different contexts; 3) the lack of trust on data; 4) the absence of a common agenda between research and policy-making; 5) the scarcity of research infrastructures that allow international comparison; 6) the need of comparable performance indicators and common benchmarks for comparison; 7) the secular inability of researchers to improve public reporting; or 8) the limited and uneven critical mass on health services and policy research in Europe as compared to USA, for example.
The objectives
The ECHO project, has addressed some of those hindrances.
ECHO (2010-2014), a collaborative project funded by the EC 7th Framework program, was conceived as a demonstration project on healthcare performance assessment, set about the task of bringing together patient-level routinely collected data from Austria, Denmark, England, Portugal, Slovenia and Spain.
From the point of view of the healthcare performance assessment debate, ECHO expanded the usual approach on international comparison based on average figures (i.e. scarcely informative), based on measures uncertainly comparable (i.e. dubious acceptance), and unable to provide reliable benchmarks (i.e. with limited capacity for levering any policy decisions). Conversely, ECHO has been able to analyse performance variation within and across countries at different decision-making levels (i.e. able to inform policy-decision), using comparable measures (i.e. increasing reliance on data), and developing accurate benchmarks (i.e. enabling decision making).
Some instruments were critical in attaining those goals:
1) The construction of a single accessible knowledge infrastructure where multiple datasets from five ECHO countries (up 30 information sources) were harmonized, stored and made accessible for research.
2) The development of around 50 healthcare performance indicators comparable across 5 countries, entailing: the production of a common framework for healthcare performance assessment, a conceptual agreement on the definition for each one of those, the elaboration of a map of codes matching the definition, and the face- and empirical validations of those definitions within each country.
3) The use of tailored benchmarks built on data pooling and using sound and robust statistical techniques. Thus, allowing in-country, cross-country and “aspirational” benchmarking.
4) The inclusion of local stakeholders in the face- and empirical validation process of the performance indicators, and in the dissemination of the in-country performance results.
Project Results:
As outlined hereafter, key ECHO Scientific and Technology contributions involve a number of research areas, from architectural framework for the ECHO Data Warehouse (DWH) to software unique developments for the ECHO Tool. Here, main Scientific and technological outputs are briefly described.
Measuring the performance of European health systems in a comparable and quantifiable way represents a major challenge. Reliability in the ECHO project is being ensured through a set of cutting-edge methodological developments laying the technical foundations of the project:
1. The ECHO DWH has been successfully consolidated (4th version) containing information from the six countries involved in ECHO, following the foreseen logic data model.
2. That version has been exposed to a large and systematic quality checking in order to test its internal coherence, reliability and accuracy; therefore, the logic model and the consistency of the core variables and ECHO indicators over time, and per country have been explored. The results have been collated in the ECHO Information System Quality Report.
3. With the goal of depicting boundaries (natural catchment areas or administratively defined areas) national vectors have been gathered for depicting the maps of each country taking into account the meaningful areas for each country. A Spanish-Danish pilot study was developed to test how size heterogeneity and flows between areas might influence estimates. Based on the empirical analyses, ECHO has decided to add a new map in Portugal, England and Denmark, base upon the actual flows of population seeking care (hospital catchment areas).
4. From a pool of performance indicators that had proven valid for performance assessment purposes at multinational level using administrative data– the ECHO consortium chose around 50 to be analyzed within the ECHO project.
5. Those indicators were defined using national coding systems taking as a reference ICD 9th CM, developing crosswalks across classifications. In order to evaluate the validity of those definitions, face validation and empirical validation were performed. Fact sheets for each indicator and country were built for that purpose. Feedback from country experts was adopted and final definitions implemented before analysis.
6. A handbook including all methodological insight provided by ECHO has been drafted, and will be available on-line.
7. Preliminary results from several of these indicators have been shown in several events; some of them have been already used to inspire policies (SANCO, OECD), some of them have been released in international scientific conferences (iHEA, EUPHA, Wennberg International Collaborative).
8. Additionally, 10 ECHO Local Dissemination Groups, aimed both at validating the ECHO outlets and to discuss uptake and use of project output by different stakeholders, have been organized with a turnout of 160 stakeholders from 5 EU countries.
9. Atlas reports on coronary revascularization, lower value care and potentially avoidable admissions have been drafted. These reports feature international and in-country variations in performance, its evolution over-time and the effect of socioeconomic gradient, and provide some policy messages derived from the results.
10. A set of web-based analytical tools has been developed allowing the replication of new analyses, as well as the report of the main results.
ECHO DWH
ECHO DWH was designed as a relational database for the storage of information of hospitalization episodes, obtained at discharge. Each registry at the DWH, up to 200 millions, corresponds to a single episode, in which all the attributes (variables) of interest for potential analyses are recorded.
ECHO Episodes contribution by country and year (millions)
Output files development
All the ECHO DWH developments are meant to obtain 4 output files (see figure): geographic indicators file, hospital indicators file, risk-adjusters file, DRG file . While the two first are collecting the ECHO performance indicators, the latter are auxiliary files used in risk adjustment modelling when estimating case-fatality rates, adverse events, or resources consumption.
Collection of national databases & data quality checking
The ECHO-DWH is fed with official statistics, national census and nation-wide administrative registries, collected by top-level government institutions, like Ministries of Health. The ECHO project works primarily with sensible data. Clear transfer responsibilities have been contractually set up with partners, making it clear who was responsible before, during transportation, reception and use of the individual data. To this end, a number of data transfer & data use agreements stating use limitations and/or protection requirements were signed.
ECHO Information system quality report
The consolidated version of the ECHO DWH was exposed to a quality analysis, essentially meant to assess internal coherence (a contrast of identity rules, referential integrity, cardinality and preservation of inheritance principles), reliability (a measure of the internal consistency of core elements across countries, and over years) and accuracy (an assessment of potential misclassification biases). The results of this quality analysis have been included in the ECHO Information System Quality Report.
Face- and empirical validation
Face- and empirical validation had as main output the selection of a set of performance indicators sound enough to be used in the performance analysis, in country and cross-country.
The choice of indicators was made on the basis of a valid and reliable comparability across countries and relevance to health care decision makers. At the end of this process a set of around 50 healthcare performance indicators are considered reliable for that purpose.
The validation process consisted of a set of tasks, starting in the previous reporting period with the development of a common framework for healthcare performance assessment, a conceptual agreement on the indicators that match the framework, and the elaboration of definition for each one of those indicators, as well as the map of codes matching the definition. Then, stemming from ICD9th-CM specifications for each ECHO indicator, crosswalks were translated into the different coding classifications in place. ECHO countries used two different systems of diagnosis coding (ICD and ICD10) and four different systems of procedure coding. For each of the indicators, crosswalks from ICD9-CM to the other systems were devised and checked (see table).
Codification of diagnoses and procedures in ECHO countries
Country Diagnoses Procedures
Portugal ICD 9th CM ICD-9th CM
Spain
Denmark ICD-10th NOMESCO
England OPCS4
Slovenia ACHI
Those crosswalks were implemented into the DWH (version 1) and raw numbers (cases and population at risk) for each indicator and country were calculated and mapped out. These figures compose the basic material for validation – essentially an in-country prima facie acceptance of the definitions for each indicator and the corresponding results. In country experts and stakeholders were asked to double-check the definitions with local coding practices and, numbers and figures with local information sources, and to point out those oddities that deserved further attention –looking deeper into the codes, crosswalks or the programming processes within the DWH.
Risk Adjustment
To determine appropriate risk-adjustment procedures two subprojects were carried out. First, the UYORK team reviewed (using systematic methods of searching and critical appraisal) the extensive literature comparing risk-adjustment methods and in particular co-morbidity indices. Second, an empirical comparison of the performance of the two leading comorbidity indices (the Charlson/Deyo and Elixhauser comorbidity indices) was carried out using ECHO data from five countries in 2008-2009. Three inpatient groups commonly used in hospital quality comparisons were included: mortality rates following coronary artery bypass graft surgery, acute myocardial infarction and stroke. The two indices were compared with each other and with a simple model including only patient age and sex, in terms of model discrimination, calibration and goodness of fit, with internal and external validation. The Elixhauser Index was found to have better overall predictive ability in terms of discrimination and goodness of fit than the Charlson/Deyo index or an age-sex only model. All models are well calibrated in all conditions. These findings are robust to the choice of country, to pooling all five countries and to internal and external validation. For the purpose of ECHO we judged the Elixhauser index to be a preferable measure.
DRG weighting
DRG weighting is a particular case of risk adjustment specifically used in comparison of technical efficiency. Only discharges from Portugal and Spain were grouped using DRG grouping system (just these two countries coded using ICD9th in both diagnoses and procedures). Given this limitation, technical efficiency analyses have followed an alternative approach allowing the other countries to be included; so, hospital discharges have been adjusted by multiplying each discharge by its relative weight according to the patient classification system in use in each country.
Mapping out intermediate layers
Geographic variation studies aim at eliciting systematic variation, therefore, variation not attributable to chance. A phenomenon that might jeopardize this goal is the extra-variation related to population size heterogeneity across the geographic areas. Getting sounder estimates of variation requires paying attention to the unit of analysis and the variation estimators. The critical issue with units of analysis is their frequently heterogeneous size - the more the heterogeneity the more likely observing extra-variation amenable to population´s size rather than to practice variation. These issues are even more important in international comparisons when putting side by side units with enormous heterogeneity, both in size and event prevalence, within and across countries.
The ECHO approach was to build an intermediate layer to reduce heterogeneity preserving the meaningfulness of the units of analysis for decision-making purposes. The construction of these new units of analysis had to attain two goals: 1) reflecting the actual population exposure to hospital care, and 2) being respectful with the limits of the existing upper administrative level.
Beyond some effect on the local interpretation of variation and rates in the new geographical units as compared to that one in the older areas, building greater units of analysis entails a change in the estimates of variation. It naturally implies less extreme values and lower exposure to random phenomena, and as a consequence more reliable estimates.
Benchmarking in ECHO
An essential element in the ECHO performance measurement framework is the construction of benchmarks.
Given the advantage of the availability of individual-patient data in a single database, and the allocation of each hospital admission into a geographic area (place of residence) or into a hospital of treatment, ECHO is able to construct robust in-country and cross-country benchmarks.
In the case of the geographic analyses, the benchmark is determined by estimating the expected number of cases in a geographic area, either using a population of reference (direct standardization) or the age-sex specific rates in the standard population (indirect standardization). When the interest is in the national benchmark the standard of reference is the national “population”, while the ECHO population is used when the interest is in the international comparison.
In the case of hospital-specific analyses, the expected number of cases is estimated using logit-type multilevel analyses. When the interest is in-country benchmarking the models use all the patients and hospitals in a specific country; in turn, international benchmarks are estimated using the whole sample of patients and hospitals in ECHO.
ECHO Atlases usually report performance indicators with both national and international benchmarks. But, ECHO allows benchmarking between pairs of countries, as well. This allows country A be compared with the best country within the ECHO sample, providing a sort of aspirational reference for each indicator.
Methodology standardisation
The ECHO Handbook on Methods is meant to critically describe those methodological approaches and analytical techniques used in ECHO (see table), throughout either brief reviews or case studies. Thought as an on-line publication, the Handbook contains those methods upon which the “ECHO Atlas reports” have been constructed. The handbook will be progressively nurtured with new entries detailing those methods used to answer those research questions stemming from the interaction between the ECHO consortium and the ECHO research infrastructure.
ECHO methodological approaches
Geographic approach Hospital-specific approach
Research question
Does the place of residence influence the population experience of getting effective and safe care?
Is the exposure to high-quality and safe care dependant on the provider where a patient is assisted?
Main endpoint
Standardised rate or Standardised Utilization Ratio for hospital admissions or procedures
Adjusted risk and Observed to Expected Ratio, analysing events amenable to healthcare quality
Denominator
Population living in a pre-defined geographical area
Patients treated in a hospital
Main audience
Policy - makers Managers
Clinicians
Final Report
Rather than a single report, the ECHO consortium decided in the previous reporting period to produce several Atlas reports, with both international and in-country sections. While the first one is merely a snapshot of the situation of a particular country with regard to the other using the most recent information, the second one includes also in-country trends for each performance indicator, differences across socioeconomic quintiles and policy implications. ECHO delivers three Atlases reports on unwarranted variations in performance: atlas on coronary revascularization, atlas on lower value procedures and atlas on potentially avoidable admissions in chronic conditions.
ECHO Tool
Going beyond the regular dissemination channels, the web-tool allows a straightforward use of sophisticated statistical apparatus; logged stakeholders can replicate or create their own analysis without needing a good command of the underlying statistical techniques.
Summing up, the above streams of work have produced a set of original research results:
• A refined set of accurate performance measures.
• Reliable information about the actual performance of different providers and different health systems.
• Methodological insight to overcome some of the classical barriers for adequate performance measurement.
• A set of web-based analytical tools to replicate methods and analyses on more specific and local problems.
Potential Impact:
IMPACT
ON FUTURE RESEARCH
ECHO was conceived as a demonstration project aimed at testing whether creating a research infrastructure based on routinely collected data from different countries was feasible, whether making those data comparable was reliable, and whether analysing performance across healthcare systems was accurate. ECHO has fairly succeeded in those tasks creating a starting community that could allow further development on the field of health care performance and policy analysis.
Actually, several institutions across Europe have explicitly expressed interest in joining this community and eventually contributing their national datasets to the ECHO knowledge infrastructure. The referred institutions are: INAMI-Ministry of Health in Belgium, STAMPAR-Institute of Public Health in Croatia, IRDES-Institute for research and information in Health Economics in France, SSSUP-Scuola Superiore Sant´Anna di Pisa in Italy, NIVEL-Netherlands Institute for Health Services Research in The Netherlands, University of Debrecen in Hungary and UNIBERN-University of Bern, in Switzerland.
Last but not least, ECHO has been invited to provide its expertise the meetings of the European Commission EGHI Ad hoc Core Working Group on the potential ERIC on health information. This Experts Group on Health Information is exploring avenues for a long-term health information and knowledge system that would provide harmonized and comparable indicators at national and at EU level with the objective of following trends, serving as a foundation for evidence-based policy and being a basis for research projects.
ON POLICY-MAKING
Local Dissemination Groups at ECHO have actually been an experimental ground where analysing the potential impact of ECHO methods and findings. The interaction with 160 different stakeholders (policy-makers, managers, public health officers, data-analysts, patient-representatives, professional organizations, etc.) provided three main messages: a) a project like ECHO is relevant for decision-making at local level; b) the way results are made public is important to foster change, being the Atlas reports and the ECHO tool appropriate mechanisms; and, c) stakeholders need up-to-date information to support their decisions, encouraging ECHO to continue the endeavour.
While LDGs provided insight at national level, ECHO concept and methods have been deemed meaningful by different international institutions –EC, OECD and WHO Observatory of Health Systems and Policies. As a matter of fact, ECHO was invited to debate the potential implications of the project at DG SANCO (Unit on Healthcare systems -October 2012) and presented at DG SANCO Working Group on Patient Safety and Quality in February 2014. ECHO is providing expert advice to the OECD project on Medical Practice Variations and will participate in the biannual meeting of the Healthcare Quality indicators Project (Q2 2014). Finally, ECHO has been invited to attend the Expert Meeting to review the Joint Assessment Framework on Health (JAF Health) meant to screening device to detect possible challenges in Member State's health systems, with a specific focus on access, quality and equity (DG Employment Q2 2014).
Dissemination
Local Dissemination Groups
ECHO presented a two-fold challenge in promoting the project’s findings: Firstly, the data does not speak for itself but requires further questioning and investigation – the project does not provide piece-meal analyses but conversation starters which need to be assessed in the right way by different target groups. Secondly, different target groups (including policy makers, health managers, and medical specialists) have specific perceptions and preferences on how to use data for decision-making. This requires a ‘dialogue model’ to engage with the different target groups and to moderate their dialogue.
In order to support these conversations the European Health Management Association in cooperation with all partners in the ECHO project developed a series of so-called “Local Dissemination Groups”. The approach to the Local Dissemination Groups (LDGs) was to have two meetings in each of the involved countries. Local project partners played an essential role in identifying the right participants for their respective system, and used their networks and clout to get those stakeholders to the meeting. IACS played a key role in preparing material and presenting the projects output.
The aim of the first round of meetings was (1) to validate output of the ECHO project, (2) to learn about how target groups rate the presentation of the project’s output, and (3) to promote the project. Participants were asked to fill in a questionnaire on the correctness and usefulness of the presented data, which both were measured using a Likert-scale; there was also space for additional comments.
The aim of the second round of meetings was (1) to discuss uptake and use of project output by different stakeholders and promoting their cooperation, and (2) to learn about barriers and opportunities, and stakeholders’ willingness to use the output of the ECHO project. During a discussion session under the Chatham House Rule with different stakeholders their views and input were noted.
A number of general learning points from the first round of LDGs illustrate the interaction between target groups and the project consortium, and how the feedback from the target groups was taken into account when presenting the ECHO output. For instance, the use of colours on the map and in the funnel plots was improved as didn’t help people who are colour blind, and overall participants of the LDGs considered the colour scheme confusing (red and green dots; too similar colours e.g. red and pink).
Feedback often reflected challenges within the countries’ health systems. Main topics included the role and trust in quality of data, which indeed was different in each country. Provided causes for variations showed similarities across the countries; lack of use of evidence and guidelines, lack of a leading organisation or institution to drive the agenda and reduce variations in health services delivery, and – on the demand side – supplier induced demand resulting in overexposure.
In total, the LDGs reached over 160 in-country stakeholders. Ideally, this number would have been larger (also here some countries showed a higher interest than others), and it could have included more policy makers.
Attendees at the Spanish LDG
Conferences
Preliminary results have been shown in multiple international scientific conferences such as Health Services Research Europe, iHEA World Congress, Dartmouth Institute for Health Policy and Clinical Practice, Wennberg International Collaborative, European Health Forum, EHMA Annual Conference or EUPHA, where the ECHO final conference took place.
Final Conference
The ECHO project’s final conference took place on 13 November 2013 in Brussels at the SQUARE venue, organised as a EUPHA pre-conference symposium by the European Health Management Association.
During the final conference results have been presented, showing how the involved countries can improve the access, safety and utilisation of their healthcare services. The emphasis was on how unwarranted variation can be tackled, and which areas of care are in need of improvements. In short, the aims of the conference were: To promote the findings of the project around safety, quality, access and efficiency of healthcare delivery; and To discuss how the impact on managerial and policy decisions can be optimised. 50 delegates participated in this free final conference.
Project partners presented the work carried out, and the day concluded with an interesting policy discussion including the European Commission DG SANCO, the OECD, and the European Observatory on how ECHO could inform Health Systems and Policies across Europe.
Supplement
Noticeably, ECHO has contracted with Oxford University Press the publication of a supplement in the European Journal of Public Health, including a number of papers reflecting the work done by ECHO. Several papers are being drafted with the following working titles:
1. Editorial
2. ECHO Overview: methodological challenges in international performance measurement
3. Comparing the performance of the Charlson/Deyo and Elixhauser co-morbidity indices across five European countries and three conditions
4. Making areas comparable for international comparisons: the case of Denmark, Portugal and England
5. Comparing hospital performance within and across countries: an illustrative study of coronary artery bypass graft surgery in England and Spain
6. Trends in socioeconomic inequality in effective procedures - methodological case study of six European countries from 2002-9
7. Poor quality of ambulatory care: case study of the overall rate of potentially avoidable hospitalizations in six chronic conditions
8. Variations in lower-value procedures – case study of C-section in low risk deliveries
9. Variations in hospital efficiency - an approach using SFA
10. Translating ECHO findings into practice: results and learning from LDGs
Exploitation
The ECHO Tool Software, the ECHO DWH and its compound, the ECHO Tool constitute relevant foreground of the ECHO Project.
The ECHO DWH is a database of all partners’ raw data concerning each party’s national healthcare system. Using this ECHO tool software, online logged users accessing through the ECHO webpage http://www.echo-health.eu/ will perform comparisons between and within English, Danish, Portuguese, Slovenian and Spanish regions, countries and hospitals.
Exploitation of the ECHO DWH
The consortium extensively discussed the possibilities, advantages and disadvantages of charging for the delivery of aggregated data.
Possible positive effects of delivering aggregated data to other parties include:
1. Could create some finance to continue ECHO work as well as covering costs
2. Could ensure that ECHO data is used to improve healthcare in countries
3. Could raise the profile of ECHO so potentially attracting more funding/avenues of funding
However, the discussion reflected a range of technical/practical and ethical issues:
1. Requires cooperation of all partner countries for European comparison (all partners need to agree to collect and analyse data on a yearly basis).
2. The ethical clearance for countries may not allow for ‘sale’ of data; at the moment of writing a heated discussion takes place in England about the availability of pseudonymised data to private companies including the pharmaceutical industry
3. The rules on destruction of data by a certain date for some countries means that longitudinal data may not be available decreasing the attractiveness of what could be on offer
4. It is not clear that the costs of collecting/analysing data could be covered by sale (including necessary back-room staff)
The shared view of the consortium was that the offer would be very limited, in particular taking note of the technical and ethical clearances involved.
Exploitation of the ECHO tool software
There is an opportunity to explore commercial exploitation of the methodology and technical infrastructure behind the ECHO Tool.
The ECHO Software is an innovative initiative in healthcare performance because:
a) It enables the analysis of variation at provider level rather than the analysis of average measures.
b) It pools individual patient data from different sources using different codification languages.
c) It enables to analyse performance at levels of disaggregation meaningful from the perspective of the stakeholders –country, region, healthcare area and hospital.
d) It also allows the linkage with other sources containing relevant information from the point of view of stakeholders –socioeconomic status of the areas where the patients live, for example.
e) It is flexible enough to implement both the simplest graphs and the most sophisticated analyses and plots by linking the dataset feeding the tool.
To sum up, the added value of the Tool is on the implementation of ground breaking scientific methodology and IT solutions into the decision-making arena. ECHO software mining technology is capable of pooling thousand of registries from different countries, using different codification languages, making them comparable over time. At the same time, the Tool implements sophisticated mathematical analyses that support the robustness and reliability of the outputs.
List of Websites:
Contact details
Contact coordinating team
Enrique Bernal-Delgado MD PhD MPH MHecon
ECHO project coordinator ebernal.iacs@aragon.es; Phone +34976716792
http://www.echo-health.eu/
Partners
- Aragon Health Research Institute (IACS – IIS Aragon), Spain
- Escola Nacional de Saùde Publica - Universidade Nova de Lisboa (ENSP), Portugal
- Institute of Public Health of the Republic of Slovenia (NIJZ), Slovenia
- University of York - Department of Health Sciences (UYORK), England
- Private University for Health Sciences, Medical Informatics & Technology (UMIT), Austria
- University of Southern Denmark - Institute of Public Health (SAM_SDU), Denmark
- European Health Management Association Ltd (EHMA), Ireland
Collaborator
- Atlas VPM
Europe’s health systems are struggling with providing accessible, equitable high-standard of care while restraining the costs of health services. Underperforming care and overexposure to health services are therefore important challenges to meet. In order to make the right decisions policy makers, health managers and professionals need to count on the right information.
The basis for providing optimum health services lays onto in depth monitoring and analysis of the equitable access, quality and efficiency patterns in our health systems from a multilevel perspective. Thus, the key task facing the interdisciplinary team of doctors, statisticians, economists and information technology officers enrolled in the ECHO project is to refine a comprehensive set of performance indicators to enable the identification of possible deficits (unwarranted variations in health care performance) in the health systems, and to provide evidence to inform policy and management decisions.
The ECHO project, co-financed by the European Commission, is designed to create a “knowledge” system. At the core of this knowledge system is the development and implementation of a common data warehouse and a series of on-line tools allowing users to explore equitable access, quality and efficiency of healthcare at the hospital, healthcare area, regional and country level. Noticeably, the ECHO tool, an analytical device that will be accessible to pre-registered research scientists and policy makers via website. This online analysis tool is scheduled to be operational for EU stakeholders during the 2nd half of 2014. Finally, methods underpinning ECHO findings and developments will be also available on-line.
ECHO has focused on developing a robust methodology to compare performance across healthcare systems – 1) building valid crosswalks across coding languages in up to 50 performance indicators; 2) creating benchmarks to allow sound international comparison; 3) building maps where national vectors have been gathered for depicting the maps of each country taking into account the meaningful areas relevant for each country. ECHO performance Atlases used this methodology to report unwarranted differences in health systems performance across ECHO countries. Atlas reports on cardiovascular care, lower value care and potentially avoidable admissions have been yielded. These reports feature international and in-country variations in performance, its evolution over-time, the effect of socioeconomic gradient, and provide some policy messages derived from the results.
However, ECHO is a starting community. The strength of ECHO as a routine tool for EU decision-makers will become a reality if ECHO is able to provide: 1) the most up-to-date information (ECHO pilot was based on 2002 to 2009 info); 2) a larger case-mix of countries (namely, healthcare systems) enabling more reliable benchmarking; and 3) a strong critical mass supporting a sustainable and growing research infrastructure that regularly yields robust evidence.
The above raises a set of unique challenges. It is overcoming the above challenges that ECHO is concentrated on. To know more, visit the project website at www.echo-health.eu
Project Context and Objectives:
The context
In the last two or three decades research on health systems and policy research has provided numerous evidence on healthcare performance that could be translated into policy messages such as: a) the place where a person lives determines the exposure to high value or low value care; b) more care is not always better; c) healthcare organizations produced uneven health outcomes, once differences in patients are ruled out; d) learning cascades are determinant in attaining good health outcomes but also, might hold substandard performance; e) access to effective ambulatory care improve health outcomes; f) money does not always become value; g) access to effective and safe healthcare may be determined by socioeconomic status; h) high-quality information may reduce allocative inefficiency; i) populations are facing high opportunity costs associated to the exposure to low value care.; or j) continuity of care is critical in attaining better outcomes in chronic patients .
The European Commission, throughout EU research programmes, has funded many research projects meant to shed light on either rethinking performance or studying how systems perform. [http://www.healthdatanavigator.eu/international-home/eu-and-international-projects]
However, even though some National Agencies have been either producers or early adopters of some of those research findings, the uptake of evidence is still scarce and unequal across EU Member States.
Many reasons have been suggested: 1) lack of consensus on what health systems performance is; 2) the inherent difficulties on implementing findings that are usually coming from different contexts; 3) the lack of trust on data; 4) the absence of a common agenda between research and policy-making; 5) the scarcity of research infrastructures that allow international comparison; 6) the need of comparable performance indicators and common benchmarks for comparison; 7) the secular inability of researchers to improve public reporting; or 8) the limited and uneven critical mass on health services and policy research in Europe as compared to USA, for example.
The objectives
The ECHO project, has addressed some of those hindrances.
ECHO (2010-2014), a collaborative project funded by the EC 7th Framework program, was conceived as a demonstration project on healthcare performance assessment, set about the task of bringing together patient-level routinely collected data from Austria, Denmark, England, Portugal, Slovenia and Spain.
From the point of view of the healthcare performance assessment debate, ECHO expanded the usual approach on international comparison based on average figures (i.e. scarcely informative), based on measures uncertainly comparable (i.e. dubious acceptance), and unable to provide reliable benchmarks (i.e. with limited capacity for levering any policy decisions). Conversely, ECHO has been able to analyse performance variation within and across countries at different decision-making levels (i.e. able to inform policy-decision), using comparable measures (i.e. increasing reliance on data), and developing accurate benchmarks (i.e. enabling decision making).
Some instruments were critical in attaining those goals:
1) The construction of a single accessible knowledge infrastructure where multiple datasets from five ECHO countries (up 30 information sources) were harmonized, stored and made accessible for research.
2) The development of around 50 healthcare performance indicators comparable across 5 countries, entailing: the production of a common framework for healthcare performance assessment, a conceptual agreement on the definition for each one of those, the elaboration of a map of codes matching the definition, and the face- and empirical validations of those definitions within each country.
3) The use of tailored benchmarks built on data pooling and using sound and robust statistical techniques. Thus, allowing in-country, cross-country and “aspirational” benchmarking.
4) The inclusion of local stakeholders in the face- and empirical validation process of the performance indicators, and in the dissemination of the in-country performance results.
Project Results:
As outlined hereafter, key ECHO Scientific and Technology contributions involve a number of research areas, from architectural framework for the ECHO Data Warehouse (DWH) to software unique developments for the ECHO Tool. Here, main Scientific and technological outputs are briefly described.
Measuring the performance of European health systems in a comparable and quantifiable way represents a major challenge. Reliability in the ECHO project is being ensured through a set of cutting-edge methodological developments laying the technical foundations of the project:
1. The ECHO DWH has been successfully consolidated (4th version) containing information from the six countries involved in ECHO, following the foreseen logic data model.
2. That version has been exposed to a large and systematic quality checking in order to test its internal coherence, reliability and accuracy; therefore, the logic model and the consistency of the core variables and ECHO indicators over time, and per country have been explored. The results have been collated in the ECHO Information System Quality Report.
3. With the goal of depicting boundaries (natural catchment areas or administratively defined areas) national vectors have been gathered for depicting the maps of each country taking into account the meaningful areas for each country. A Spanish-Danish pilot study was developed to test how size heterogeneity and flows between areas might influence estimates. Based on the empirical analyses, ECHO has decided to add a new map in Portugal, England and Denmark, base upon the actual flows of population seeking care (hospital catchment areas).
4. From a pool of performance indicators that had proven valid for performance assessment purposes at multinational level using administrative data– the ECHO consortium chose around 50 to be analyzed within the ECHO project.
5. Those indicators were defined using national coding systems taking as a reference ICD 9th CM, developing crosswalks across classifications. In order to evaluate the validity of those definitions, face validation and empirical validation were performed. Fact sheets for each indicator and country were built for that purpose. Feedback from country experts was adopted and final definitions implemented before analysis.
6. A handbook including all methodological insight provided by ECHO has been drafted, and will be available on-line.
7. Preliminary results from several of these indicators have been shown in several events; some of them have been already used to inspire policies (SANCO, OECD), some of them have been released in international scientific conferences (iHEA, EUPHA, Wennberg International Collaborative).
8. Additionally, 10 ECHO Local Dissemination Groups, aimed both at validating the ECHO outlets and to discuss uptake and use of project output by different stakeholders, have been organized with a turnout of 160 stakeholders from 5 EU countries.
9. Atlas reports on coronary revascularization, lower value care and potentially avoidable admissions have been drafted. These reports feature international and in-country variations in performance, its evolution over-time and the effect of socioeconomic gradient, and provide some policy messages derived from the results.
10. A set of web-based analytical tools has been developed allowing the replication of new analyses, as well as the report of the main results.
ECHO DWH
ECHO DWH was designed as a relational database for the storage of information of hospitalization episodes, obtained at discharge. Each registry at the DWH, up to 200 millions, corresponds to a single episode, in which all the attributes (variables) of interest for potential analyses are recorded.
ECHO Episodes contribution by country and year (millions)
Output files development
All the ECHO DWH developments are meant to obtain 4 output files (see figure): geographic indicators file, hospital indicators file, risk-adjusters file, DRG file . While the two first are collecting the ECHO performance indicators, the latter are auxiliary files used in risk adjustment modelling when estimating case-fatality rates, adverse events, or resources consumption.
Collection of national databases & data quality checking
The ECHO-DWH is fed with official statistics, national census and nation-wide administrative registries, collected by top-level government institutions, like Ministries of Health. The ECHO project works primarily with sensible data. Clear transfer responsibilities have been contractually set up with partners, making it clear who was responsible before, during transportation, reception and use of the individual data. To this end, a number of data transfer & data use agreements stating use limitations and/or protection requirements were signed.
ECHO Information system quality report
The consolidated version of the ECHO DWH was exposed to a quality analysis, essentially meant to assess internal coherence (a contrast of identity rules, referential integrity, cardinality and preservation of inheritance principles), reliability (a measure of the internal consistency of core elements across countries, and over years) and accuracy (an assessment of potential misclassification biases). The results of this quality analysis have been included in the ECHO Information System Quality Report.
Face- and empirical validation
Face- and empirical validation had as main output the selection of a set of performance indicators sound enough to be used in the performance analysis, in country and cross-country.
The choice of indicators was made on the basis of a valid and reliable comparability across countries and relevance to health care decision makers. At the end of this process a set of around 50 healthcare performance indicators are considered reliable for that purpose.
The validation process consisted of a set of tasks, starting in the previous reporting period with the development of a common framework for healthcare performance assessment, a conceptual agreement on the indicators that match the framework, and the elaboration of definition for each one of those indicators, as well as the map of codes matching the definition. Then, stemming from ICD9th-CM specifications for each ECHO indicator, crosswalks were translated into the different coding classifications in place. ECHO countries used two different systems of diagnosis coding (ICD and ICD10) and four different systems of procedure coding. For each of the indicators, crosswalks from ICD9-CM to the other systems were devised and checked (see table).
Codification of diagnoses and procedures in ECHO countries
Country Diagnoses Procedures
Portugal ICD 9th CM ICD-9th CM
Spain
Denmark ICD-10th NOMESCO
England OPCS4
Slovenia ACHI
Those crosswalks were implemented into the DWH (version 1) and raw numbers (cases and population at risk) for each indicator and country were calculated and mapped out. These figures compose the basic material for validation – essentially an in-country prima facie acceptance of the definitions for each indicator and the corresponding results. In country experts and stakeholders were asked to double-check the definitions with local coding practices and, numbers and figures with local information sources, and to point out those oddities that deserved further attention –looking deeper into the codes, crosswalks or the programming processes within the DWH.
Risk Adjustment
To determine appropriate risk-adjustment procedures two subprojects were carried out. First, the UYORK team reviewed (using systematic methods of searching and critical appraisal) the extensive literature comparing risk-adjustment methods and in particular co-morbidity indices. Second, an empirical comparison of the performance of the two leading comorbidity indices (the Charlson/Deyo and Elixhauser comorbidity indices) was carried out using ECHO data from five countries in 2008-2009. Three inpatient groups commonly used in hospital quality comparisons were included: mortality rates following coronary artery bypass graft surgery, acute myocardial infarction and stroke. The two indices were compared with each other and with a simple model including only patient age and sex, in terms of model discrimination, calibration and goodness of fit, with internal and external validation. The Elixhauser Index was found to have better overall predictive ability in terms of discrimination and goodness of fit than the Charlson/Deyo index or an age-sex only model. All models are well calibrated in all conditions. These findings are robust to the choice of country, to pooling all five countries and to internal and external validation. For the purpose of ECHO we judged the Elixhauser index to be a preferable measure.
DRG weighting
DRG weighting is a particular case of risk adjustment specifically used in comparison of technical efficiency. Only discharges from Portugal and Spain were grouped using DRG grouping system (just these two countries coded using ICD9th in both diagnoses and procedures). Given this limitation, technical efficiency analyses have followed an alternative approach allowing the other countries to be included; so, hospital discharges have been adjusted by multiplying each discharge by its relative weight according to the patient classification system in use in each country.
Mapping out intermediate layers
Geographic variation studies aim at eliciting systematic variation, therefore, variation not attributable to chance. A phenomenon that might jeopardize this goal is the extra-variation related to population size heterogeneity across the geographic areas. Getting sounder estimates of variation requires paying attention to the unit of analysis and the variation estimators. The critical issue with units of analysis is their frequently heterogeneous size - the more the heterogeneity the more likely observing extra-variation amenable to population´s size rather than to practice variation. These issues are even more important in international comparisons when putting side by side units with enormous heterogeneity, both in size and event prevalence, within and across countries.
The ECHO approach was to build an intermediate layer to reduce heterogeneity preserving the meaningfulness of the units of analysis for decision-making purposes. The construction of these new units of analysis had to attain two goals: 1) reflecting the actual population exposure to hospital care, and 2) being respectful with the limits of the existing upper administrative level.
Beyond some effect on the local interpretation of variation and rates in the new geographical units as compared to that one in the older areas, building greater units of analysis entails a change in the estimates of variation. It naturally implies less extreme values and lower exposure to random phenomena, and as a consequence more reliable estimates.
Benchmarking in ECHO
An essential element in the ECHO performance measurement framework is the construction of benchmarks.
Given the advantage of the availability of individual-patient data in a single database, and the allocation of each hospital admission into a geographic area (place of residence) or into a hospital of treatment, ECHO is able to construct robust in-country and cross-country benchmarks.
In the case of the geographic analyses, the benchmark is determined by estimating the expected number of cases in a geographic area, either using a population of reference (direct standardization) or the age-sex specific rates in the standard population (indirect standardization). When the interest is in the national benchmark the standard of reference is the national “population”, while the ECHO population is used when the interest is in the international comparison.
In the case of hospital-specific analyses, the expected number of cases is estimated using logit-type multilevel analyses. When the interest is in-country benchmarking the models use all the patients and hospitals in a specific country; in turn, international benchmarks are estimated using the whole sample of patients and hospitals in ECHO.
ECHO Atlases usually report performance indicators with both national and international benchmarks. But, ECHO allows benchmarking between pairs of countries, as well. This allows country A be compared with the best country within the ECHO sample, providing a sort of aspirational reference for each indicator.
Methodology standardisation
The ECHO Handbook on Methods is meant to critically describe those methodological approaches and analytical techniques used in ECHO (see table), throughout either brief reviews or case studies. Thought as an on-line publication, the Handbook contains those methods upon which the “ECHO Atlas reports” have been constructed. The handbook will be progressively nurtured with new entries detailing those methods used to answer those research questions stemming from the interaction between the ECHO consortium and the ECHO research infrastructure.
ECHO methodological approaches
Geographic approach Hospital-specific approach
Research question
Does the place of residence influence the population experience of getting effective and safe care?
Is the exposure to high-quality and safe care dependant on the provider where a patient is assisted?
Main endpoint
Standardised rate or Standardised Utilization Ratio for hospital admissions or procedures
Adjusted risk and Observed to Expected Ratio, analysing events amenable to healthcare quality
Denominator
Population living in a pre-defined geographical area
Patients treated in a hospital
Main audience
Policy - makers Managers
Clinicians
Final Report
Rather than a single report, the ECHO consortium decided in the previous reporting period to produce several Atlas reports, with both international and in-country sections. While the first one is merely a snapshot of the situation of a particular country with regard to the other using the most recent information, the second one includes also in-country trends for each performance indicator, differences across socioeconomic quintiles and policy implications. ECHO delivers three Atlases reports on unwarranted variations in performance: atlas on coronary revascularization, atlas on lower value procedures and atlas on potentially avoidable admissions in chronic conditions.
ECHO Tool
Going beyond the regular dissemination channels, the web-tool allows a straightforward use of sophisticated statistical apparatus; logged stakeholders can replicate or create their own analysis without needing a good command of the underlying statistical techniques.
Summing up, the above streams of work have produced a set of original research results:
• A refined set of accurate performance measures.
• Reliable information about the actual performance of different providers and different health systems.
• Methodological insight to overcome some of the classical barriers for adequate performance measurement.
• A set of web-based analytical tools to replicate methods and analyses on more specific and local problems.
Potential Impact:
IMPACT
ON FUTURE RESEARCH
ECHO was conceived as a demonstration project aimed at testing whether creating a research infrastructure based on routinely collected data from different countries was feasible, whether making those data comparable was reliable, and whether analysing performance across healthcare systems was accurate. ECHO has fairly succeeded in those tasks creating a starting community that could allow further development on the field of health care performance and policy analysis.
Actually, several institutions across Europe have explicitly expressed interest in joining this community and eventually contributing their national datasets to the ECHO knowledge infrastructure. The referred institutions are: INAMI-Ministry of Health in Belgium, STAMPAR-Institute of Public Health in Croatia, IRDES-Institute for research and information in Health Economics in France, SSSUP-Scuola Superiore Sant´Anna di Pisa in Italy, NIVEL-Netherlands Institute for Health Services Research in The Netherlands, University of Debrecen in Hungary and UNIBERN-University of Bern, in Switzerland.
Last but not least, ECHO has been invited to provide its expertise the meetings of the European Commission EGHI Ad hoc Core Working Group on the potential ERIC on health information. This Experts Group on Health Information is exploring avenues for a long-term health information and knowledge system that would provide harmonized and comparable indicators at national and at EU level with the objective of following trends, serving as a foundation for evidence-based policy and being a basis for research projects.
ON POLICY-MAKING
Local Dissemination Groups at ECHO have actually been an experimental ground where analysing the potential impact of ECHO methods and findings. The interaction with 160 different stakeholders (policy-makers, managers, public health officers, data-analysts, patient-representatives, professional organizations, etc.) provided three main messages: a) a project like ECHO is relevant for decision-making at local level; b) the way results are made public is important to foster change, being the Atlas reports and the ECHO tool appropriate mechanisms; and, c) stakeholders need up-to-date information to support their decisions, encouraging ECHO to continue the endeavour.
While LDGs provided insight at national level, ECHO concept and methods have been deemed meaningful by different international institutions –EC, OECD and WHO Observatory of Health Systems and Policies. As a matter of fact, ECHO was invited to debate the potential implications of the project at DG SANCO (Unit on Healthcare systems -October 2012) and presented at DG SANCO Working Group on Patient Safety and Quality in February 2014. ECHO is providing expert advice to the OECD project on Medical Practice Variations and will participate in the biannual meeting of the Healthcare Quality indicators Project (Q2 2014). Finally, ECHO has been invited to attend the Expert Meeting to review the Joint Assessment Framework on Health (JAF Health) meant to screening device to detect possible challenges in Member State's health systems, with a specific focus on access, quality and equity (DG Employment Q2 2014).
Dissemination
Local Dissemination Groups
ECHO presented a two-fold challenge in promoting the project’s findings: Firstly, the data does not speak for itself but requires further questioning and investigation – the project does not provide piece-meal analyses but conversation starters which need to be assessed in the right way by different target groups. Secondly, different target groups (including policy makers, health managers, and medical specialists) have specific perceptions and preferences on how to use data for decision-making. This requires a ‘dialogue model’ to engage with the different target groups and to moderate their dialogue.
In order to support these conversations the European Health Management Association in cooperation with all partners in the ECHO project developed a series of so-called “Local Dissemination Groups”. The approach to the Local Dissemination Groups (LDGs) was to have two meetings in each of the involved countries. Local project partners played an essential role in identifying the right participants for their respective system, and used their networks and clout to get those stakeholders to the meeting. IACS played a key role in preparing material and presenting the projects output.
The aim of the first round of meetings was (1) to validate output of the ECHO project, (2) to learn about how target groups rate the presentation of the project’s output, and (3) to promote the project. Participants were asked to fill in a questionnaire on the correctness and usefulness of the presented data, which both were measured using a Likert-scale; there was also space for additional comments.
The aim of the second round of meetings was (1) to discuss uptake and use of project output by different stakeholders and promoting their cooperation, and (2) to learn about barriers and opportunities, and stakeholders’ willingness to use the output of the ECHO project. During a discussion session under the Chatham House Rule with different stakeholders their views and input were noted.
A number of general learning points from the first round of LDGs illustrate the interaction between target groups and the project consortium, and how the feedback from the target groups was taken into account when presenting the ECHO output. For instance, the use of colours on the map and in the funnel plots was improved as didn’t help people who are colour blind, and overall participants of the LDGs considered the colour scheme confusing (red and green dots; too similar colours e.g. red and pink).
Feedback often reflected challenges within the countries’ health systems. Main topics included the role and trust in quality of data, which indeed was different in each country. Provided causes for variations showed similarities across the countries; lack of use of evidence and guidelines, lack of a leading organisation or institution to drive the agenda and reduce variations in health services delivery, and – on the demand side – supplier induced demand resulting in overexposure.
In total, the LDGs reached over 160 in-country stakeholders. Ideally, this number would have been larger (also here some countries showed a higher interest than others), and it could have included more policy makers.
Attendees at the Spanish LDG
Conferences
Preliminary results have been shown in multiple international scientific conferences such as Health Services Research Europe, iHEA World Congress, Dartmouth Institute for Health Policy and Clinical Practice, Wennberg International Collaborative, European Health Forum, EHMA Annual Conference or EUPHA, where the ECHO final conference took place.
Final Conference
The ECHO project’s final conference took place on 13 November 2013 in Brussels at the SQUARE venue, organised as a EUPHA pre-conference symposium by the European Health Management Association.
During the final conference results have been presented, showing how the involved countries can improve the access, safety and utilisation of their healthcare services. The emphasis was on how unwarranted variation can be tackled, and which areas of care are in need of improvements. In short, the aims of the conference were: To promote the findings of the project around safety, quality, access and efficiency of healthcare delivery; and To discuss how the impact on managerial and policy decisions can be optimised. 50 delegates participated in this free final conference.
Project partners presented the work carried out, and the day concluded with an interesting policy discussion including the European Commission DG SANCO, the OECD, and the European Observatory on how ECHO could inform Health Systems and Policies across Europe.
Supplement
Noticeably, ECHO has contracted with Oxford University Press the publication of a supplement in the European Journal of Public Health, including a number of papers reflecting the work done by ECHO. Several papers are being drafted with the following working titles:
1. Editorial
2. ECHO Overview: methodological challenges in international performance measurement
3. Comparing the performance of the Charlson/Deyo and Elixhauser co-morbidity indices across five European countries and three conditions
4. Making areas comparable for international comparisons: the case of Denmark, Portugal and England
5. Comparing hospital performance within and across countries: an illustrative study of coronary artery bypass graft surgery in England and Spain
6. Trends in socioeconomic inequality in effective procedures - methodological case study of six European countries from 2002-9
7. Poor quality of ambulatory care: case study of the overall rate of potentially avoidable hospitalizations in six chronic conditions
8. Variations in lower-value procedures – case study of C-section in low risk deliveries
9. Variations in hospital efficiency - an approach using SFA
10. Translating ECHO findings into practice: results and learning from LDGs
Exploitation
The ECHO Tool Software, the ECHO DWH and its compound, the ECHO Tool constitute relevant foreground of the ECHO Project.
The ECHO DWH is a database of all partners’ raw data concerning each party’s national healthcare system. Using this ECHO tool software, online logged users accessing through the ECHO webpage http://www.echo-health.eu/ will perform comparisons between and within English, Danish, Portuguese, Slovenian and Spanish regions, countries and hospitals.
Exploitation of the ECHO DWH
The consortium extensively discussed the possibilities, advantages and disadvantages of charging for the delivery of aggregated data.
Possible positive effects of delivering aggregated data to other parties include:
1. Could create some finance to continue ECHO work as well as covering costs
2. Could ensure that ECHO data is used to improve healthcare in countries
3. Could raise the profile of ECHO so potentially attracting more funding/avenues of funding
However, the discussion reflected a range of technical/practical and ethical issues:
1. Requires cooperation of all partner countries for European comparison (all partners need to agree to collect and analyse data on a yearly basis).
2. The ethical clearance for countries may not allow for ‘sale’ of data; at the moment of writing a heated discussion takes place in England about the availability of pseudonymised data to private companies including the pharmaceutical industry
3. The rules on destruction of data by a certain date for some countries means that longitudinal data may not be available decreasing the attractiveness of what could be on offer
4. It is not clear that the costs of collecting/analysing data could be covered by sale (including necessary back-room staff)
The shared view of the consortium was that the offer would be very limited, in particular taking note of the technical and ethical clearances involved.
Exploitation of the ECHO tool software
There is an opportunity to explore commercial exploitation of the methodology and technical infrastructure behind the ECHO Tool.
The ECHO Software is an innovative initiative in healthcare performance because:
a) It enables the analysis of variation at provider level rather than the analysis of average measures.
b) It pools individual patient data from different sources using different codification languages.
c) It enables to analyse performance at levels of disaggregation meaningful from the perspective of the stakeholders –country, region, healthcare area and hospital.
d) It also allows the linkage with other sources containing relevant information from the point of view of stakeholders –socioeconomic status of the areas where the patients live, for example.
e) It is flexible enough to implement both the simplest graphs and the most sophisticated analyses and plots by linking the dataset feeding the tool.
To sum up, the added value of the Tool is on the implementation of ground breaking scientific methodology and IT solutions into the decision-making arena. ECHO software mining technology is capable of pooling thousand of registries from different countries, using different codification languages, making them comparable over time. At the same time, the Tool implements sophisticated mathematical analyses that support the robustness and reliability of the outputs.
List of Websites:
Contact details
Contact coordinating team
Enrique Bernal-Delgado MD PhD MPH MHecon
ECHO project coordinator ebernal.iacs@aragon.es; Phone +34976716792
http://www.echo-health.eu/
Partners
- Aragon Health Research Institute (IACS – IIS Aragon), Spain
- Escola Nacional de Saùde Publica - Universidade Nova de Lisboa (ENSP), Portugal
- Institute of Public Health of the Republic of Slovenia (NIJZ), Slovenia
- University of York - Department of Health Sciences (UYORK), England
- Private University for Health Sciences, Medical Informatics & Technology (UMIT), Austria
- University of Southern Denmark - Institute of Public Health (SAM_SDU), Denmark
- European Health Management Association Ltd (EHMA), Ireland
Collaborator
- Atlas VPM
