Final Report Summary - QUASER (Quality and safety in European Union hospitals: A research-based guide for implmeenting best practice and a framework for assessing performance)
Executive Summary:
This translational multilevel study was designed to investigate organisational and cultural factors affecting hospital quality improvement initiatives and produce and disseminate a guide for hospitals to develop and implement organisational-wide quality and safety improvement strategies, and a guide for payers to assess the appropriateness of a hospital’s quality improvement strategy. The study was conducted in five partner countries: England, the Netherlands, Norway, Portugal and Sweden. A group of stakeholders (hospital managers, payers and patients) from across Europe participated in translational workshops to give feedback on the design and potential use of the guides, and assist with dissemination.
Early work to identify the requirements for the guides showed that guides are not used in the same way or to the same extent across Europe, and proposed that guiding as a social process is a useful perspective. The Organising for Quality framework of six quality improvement challenges was used as the conceptual framework, expanded to eight following scoping reviews of the literature. These eight were; structural, educational, cultural, political, emotional, physical and technological, leadership and external demands. Macro-level data were collected for each country using a template developed for the study. A documented selection framework was developed and used to recruit two hospitals in each country; one high performing and one developing hospital. Using standard frameworks, interviews and observations were conducted at the meso-(organisational) level in two hospitals in each country, and micro-(service) level interviews and observations were conducted in one hospital in each country. In addition, two existing quality improvement initiatives were identified in each hospital as tracer projects to study quality improvement in action. Fieldwork took place over 14 months. A common data analysis framework was used. Data from each country were analysed using a cross case analysis guided by our research questions.
Overall, our findings showed that, despite different national and local contexts, there were common themes in how the case study hospitals approached quality improvement. First, key drivers were governance, compliance and accountability rather than learning and cultural change. The focus was on tools and data rather than on changing behaviours and cultures. In many hospitals quality improvement was not central to hospital priorities and financial considerations often took precedence over quality improvement. There was generally a ‘project-by- project’ approach rather than system-wide initiatives to improve quality, and very little focus on patient experience or patient and public involvement. Within our case study hospitals, there were ‘pockets’ where strategies were enacted which contradicted these common features, for example, relating to the cultural and educational challenges. Those hospitals which had an established track record of quality improvement had a long-term commitment to quality and stable leadership. In these hospitals, leaders embedded quality in the culture, aligned quality and cost reduction goals and motivated and energised staff.
From this analysis the implications for the guides were identified, including content, examples of quality improvement work, and the key organisational and cultural factors that facilitate quality improvement. The guides were designed using an iterative process, incorporating our findings about the use of guides in different countries, how best to develop and disseminate them, user feedback from the stakeholder workshops, and the extensive empirical data collected in hospitals. The eight challenges were used as a structure to guide users through the process of identifying strengths and weaknesses, considering strategies and examples from our fieldwork, and developing their own strategies and action plan for implementation. There is a ninth challenge for payers; how can a payer organisation facilitate quality improvement in hospitals. The guides are designed to facilitate dialogue and, as such, support a social process for quality improvement between hospital managers and clinical teams and between payers and hospitals. The guides have been disseminated extensively across Europe and are available for download from the project website www.ucl.ac.uk/dahr/quaser
Project Context and Objectives:
Although quality improvement has been high on policy agendas for a decade or more there is frustration about continuing quality and safety problems and the slow progress in achieving change (Spencer & Walshe, 2009). Additional challenges facing the European Union (EU) include the free movement of patients across borders and the importance of ensuring that they receive high quality care wherever it is delivered (Directive 2011). Healthcare systems are also under economic pressure and face more challenges as the population ages and the incidence of chronic long term conditions increases. Understanding how to improve the quality of care in hospitals is therefore an important aim.
The Quality and Safety in European Union Hospitals (QUASER) study sought to identify the organisational and cultural factors that impede or facilitate quality improvement and to develop guides that explicitly address these factors. There is a good understanding and knowledge of the types of quality improvement tools and techniques (see for example the review undertaken by Boaden et al 2008) but less understanding of the factors that increase the effectiveness of these. Studies on healthcare quality increasingly point to understanding organisational issues in health service delivery as central to explaining variations in care and making progress towards sustained improvement (Shipton et al, 2008). As elaborated by others (e.g. West 2000, Firth-Cozens 2001, Catio et al, 2005) this perspective includes recognising the multiple levels of the healthcare system. High-level influences such as policy, payment rules, regulation and accreditation are strongly mediated by dynamics and responses not only at the levels of hospitals, but also the smaller units of healthcare providers they support in delivering services to the level that matters most patients.
We defined organisational and cultural factors broadly as all the features of a hospital that might impact upon clinical effectiveness, patient safety and/or patient experience. These include, for example, leadership styles and systems, management structures, roles, relationships, mindsets and behaviours of staff, sophistication and use of available information systems, extent and success of implementation of formal approaches to quality improvement, the levels and nature of staff engagement, and the levels and nature of patient involvement. A rigorous, if relatively small, body of research does exist in the health services literature which specifically attempts to unravel this ‘black box’ of organisation at the hospital level and its impact on the quality of care (see Lukas et al, 2007, for development of an empirically-based model for ‘moving organisations from short-term, isolated performance improvements to sustained, reliable, organisation-wide, and evidence-based improvements in patient care’). This work has focused on identifying hospital predictors of successful implementation of quality improvement, typically using multivariate statistical methods and quasi-experimental data (for example, Weiner et al, 2006), and has highlighted a number of factors that appear to be associated with successfully implementing change in hospitals (see Bevan et al, 2008), for a recent review of high performing healthcare organisations). The factors that predict successful quality improvement implementation include leadership support (Shipton et al, 2008), particular dimensions of organisational culture and climate (Singer et al, 2007; Hartmann et al, 2008), and team-based structures and composition (Fay, Borrill, Amir, Haward, & West, 2006; West et al, 2003). Unfortunately, such research approaches have been less adept at shedding light on why these factors relate to one another, how they are related, and how in practice hospitals go about influencing and setting these ‘key success factors’ in motion. Finding the answers to such practical questions lie at the heart of this proposed QUASER study and our proposal enhances and extends existing studies in three ways.
The QUASER project had four key characteristics. First, it was a multi-level study; incorporating data from the macro-(social and political context), meso-(organisational) and micro-(clinical frontline) levels, and the interactions between these levels was a key aspect of the data analysis. This is particularly important for developing guides in Europe, which includes countries with diverse political and social histories and healthcare systems with different structures and processes. Second, the study used a definition of quality comprising three components; clinical effectiveness, patient safety and patient experience, which enabled us to consider how these different components of quality are conceptualised, understood, and prioritised and how they might be differently influenced by organisational and cultural factors. Third, the study viewed quality not just as a technical process, but as a human, social and organisational accomplishment. This allowed us to focus on understanding practices and processes and ask how quality improvement was negotiated, facilitated or impeded by these factors, and to design a guide to specifically support these accomplishments. Finally, the study was translational. It aimed to translate research findings into practical outputs that would assist in solving the difficult challenges of quality improvement and improve the healthcare received by patients in Europe.
The study extended recent research undertaken by members of the consortium that has addressed these questions. Bate, Mendel and Robert (2008) undertook a three-year international study that was explicitly designed to help practitioners and researchers understand the factors and processes that enable hospitals in the US and Europe to achieve, and sustain, high quality services for their patients. Based on in-depth, multi-level case-studies of seven leading hospitals (including hospitals from the UK and the Netherlands), this research found that high-performing hospitals were able to achieve, and then sustain, high levels of quality because they recognised and had been extremely successful in addressing six common challenges. This was the conceptual framework for QUASER, leading us to use a combination of induction (data-driven generalisation) and deduction (theory-driven exploration of hypotheses) (Langley, 1999) as our approach to analysis.
The QUASER study was carried out in five partner countries: England, the Netherlands, Norway, Portugal and Sweden. The objectives were to:
• Review research evidence for links between organisational and cultural characteristics of hospitals and quality of care, and quantitative and qualitative indicators of quality to inform data collection and hospital selection
• Develop a framework for hospital selection and use it to select hospitals
• Review and map the use of guides in the five countries, including how they are developed, designed and disseminated in practice to inform the design of guides
• Conduct multi-level case studies of quality improvement processes in ten hospitals (two in each country at different stages of the quality journey), including a macro-level study of the social political context and structure of the healthcare systems, a meso-level study of organisational characteristics and quality improvement, and in one hospital in each country a micro-level study of quality improvement processes at the frontline
• Carry out a cross-case analysis to identify the important organisational and cultural characteristics associated with effective quality improvement and interactions between macro-meso-and micro-levels, and to identify implications for the guides, including design and content
• Drawing on theory, empirical research and user input, design the QUASER Hospital Guide for guiding hospitals on implementing effective organisation-wide quality improvement programmes and the QUASER Guide for Payers to assess the effectiveness of a hospital’s quality improvement strategy
• Translate research findings into practice by involving users throughout the study and disseminating the outputs widely.
1.2.1 Methods
The methods are set out in the published QUASER study protocol (Robert et al, 2011). Hospitals were selected purposively using a documented selection framework (one ‘high-performing’, one ‘developing’). See the following section on WP2 for more information.
Macro-level data were collected from documentary sources using a framework of seven dimensions relating to the structure of the healthcare system, funding and access, regulatory approach to quality, accreditation and monitoring, information availability, resources available for quality and patients’ rights. For each factor there were a number of specific questions to answer. The output was a detailed narrative report for each country. These data were then aggregated and compared across countries and a typology of healthcare systems and their approaches to quality improvement was developed.
Meso-level interview data were collected using a standardised data collection protocol. Interview questions were based on the organising for quality framework and aimed to identify how hospitals were responding to the 8 quality improvement challenges, the influences of the macro-, meso- and micro-level on quality improvement, and the interactions between these levels. Participants were senior hospital leaders, middle managers, clinicians, and administrators. Observational data were collected during meetings where quality improvement was being planned or discussed using detailed field notes. In addition, two existing quality improvement initiatives that were about to start were identified in each hospital (including healthcare associated infection in all) as tracer projects to understand how quality improvement worked in action in the hospitals. There were 389 interviews, 803 hours of observation and 207 meetings observed in total. Table 1 shows which micro-systems and which tracer projects were studied in each hospital and Table 2 summarises the fieldwork undertaken in each country. See later sections on WP4 and WP4a-e.
Standard frameworks for collecting, analysing and reporting data were used in all countries. All data were analysed according to a standard coding framework with the ability to allow new themes emerging from the data to be identified. This approach was a combination of deduction using the theoretical framework and induction to identify insights emerging from the data. Five country specific case study reports were produced reporting the findings in relation to the research questions. A cross-case analysis was then conducted to synthesise the results and identify content for the guides. More information about this process can be found in the section on WP5. The process used to design the guides and the translational stakeholder workshops are described in sections on WP6 and WP7 respectively.
Project Results:
1.1.1 WP2: Scoping review and selection of case study hospitals
Work Package Leader: Professor Charles Vincent
In this section we describe the results of a scoping review of the literature and the development of the hospital selection framework. A scoping review was undertaken of the organisational and cultural factors relevant for quality improvement in hospitals (WP2). This identified organisational and cultural factors that previous studies have linked to quality of hospital care, and analysed how these factors are reflected in the ‘Organising for Quality’ framework developed by Bate, Mendel and Robert (2008). As QUASER explicitly sought to extend and apply this framework to the study of hospitals in five EU countries, the findings of the literature scoping were used to ‘play' this framework against the most critical healthcare literature. As a result of this scoping review, the 'Organising for Quality' framework of six challenges was extended to include two further challenges: (1) the leadership challenge i.e. 'how is quality improvement led in this hospital?'); (2) Managing the external environment i.e. how is the external environment in relation to QI managed? Later analysis of fieldwork data confirmed the importance of these challenges.
Selecting the hospitals to be involved in this research is described here, including the complexities of the process. For the purposes of the research, the intention was to select two case study sites that appeared from the available indicators to be at contrasting stages of development to each other. It is important to emphasise that this is not a comment on the overall performance or quality of care in these hospitals since they may be performing well on other indicators, for example indicators not publicly available or not collated nationally. The findings from this part of the research have been published (Burnett et al., 2013).
After consideration by the research partners it was agreed that a simple, sensible and easily communicable hospital selection process should be developed that would be applicable in each country. A list of 10 potential process and outcome measures to be used in the selection process was proposed, based on data publicly available in England (the country leading this work package). These included infection rates; surgical site infection rates; a composite mortality score; emergency readmission rates; hip fractures treated within 48 hours; and 24 hour scan rates for strokes. Research teams were asked:
• Are these indicators available in your country?
• Are the data available to your research team?
• Are they available for every hospital or only a selection?
• What type of data is collected?
• How robust are the data?
• What other indicators are available that you would propose?
From the responses it was clear that only three outcome indicators were available in all countries but it was not clear that these were directly comparable due to differences in definitions. These were: infection rates; a range of condition specific mortality rates; and caesarean section rates. Only one of the proposed process indicators was available in all countries – hip fractures treated in a set time. As a result, it was agreed that each country would select different process measures but all would include infection rates and two condition specific standardised mortality rates. It was also clear from the responses that the regulation and accreditation of hospitals differed between countries to the extent that this information was not comparable and could not be used for selection.
Given the different indicators available in each country and the lack of a common indicator set, the following pragmatic selection process was agreed:
1. Four process measures would be selected that are collected nationally in the partner country that: are considered to be robust in terms of the method of data collection; span a range of hospital activities (rather than all four being related to one specialty); and have been collected and are available for the last three years
2. Using the most robust outcome data available, which have been collected and are available for the last three years, a group of hospitals would be selected that fit both of the following criteria, and are placed within the top 20% of hospitals when ranked against each other: (a) have had low infection rates for the last three years, using data for example for MRSA bacteraemia, C.difficile or Surgical Site Infection rates; and (b) have had low standardised mortality rates for two specific conditions for the last three years, for example stroke, coronary artery bypass graft (CABG), Abdominal Aortic Aneurysm (AAA).
3. From the hospitals performing well in (2) only those that also have had good results over time for the selected process measures chosen in (1) would be selected.
Using the same indicators, steps 2&3 were then repeated but selecting hospitals whose performance when ranked against others was in the lower 40%. Following this, other information sources in the country were reviewed (accreditation systems/awards etc.) to confirm these hospitals are seen as high performing or average/developing. Additional factors were also considered at this stage including the size and type of hospital to include. It was agreed that for research purposes: (a) there should be a mix of general medicine and general surgical services, admitting both emergency and planned cases; (b) there should be a balance of teaching and non-teaching hospitals; and (c) the hospitals must provide maternity services (to allow for this micro-system to be part of the research).
The following information about the shortlisted hospitals in each country was then requested: number of beds; number of staff; types of services provided; type of population served (city, town or rural). In requesting this information it became clear that there are differences in the information available about hospitals in each country, for example information on staff numbers is not always available and bed numbers may depend on the organisation of the hospitals, for example into groups.
From all the available data and using a systematic documented approach two hospitals were selected for the research in each country. The following summarises our findings across the five partner countries, with regard to data collection for assessing quality in hospitals. Only three of the proposed outcome indicators were available in all countries but it was not clear that these were not directly comparable due to differences in definitions. The three were: MRSA or C. difficile infection rates; condition specific mortality rates; and caesarean section rates. None of the proposed process indicators were available in all countries, and hospital accreditation varied from a mandatory system in England to no formal system in Sweden.
• Different quality indicator data are collected in each country
• Different definitions of the same indicators are used in different countries
• Differences exist in mandatory versus voluntary data collection
• Different types of organisations oversee data collection – some governmental, some independent
• Different levels of aggregation of data – country, region, hospital
• Different ownership and access to data – some fully public, others owned and accessed through individual hospitals
• Different registries for national data kept by medical societies in each country – cardiology, hip replacements etc.
• Differences in hospital accreditation in each country, some with no system, others with compulsory schemes for all hospitals
1.1.2 WP3: Guiding Quality Work in European Hospitals
Work Package Leader: Professor Roland Bal
Using literature reviews, analysis of documents and semi-structured interviews, this WP compiled a report on the current quality improvement guides, dissemination strategies, levels of knowledge, design requirements and dissemination strategies for each of the five partner QUASER countries. This provides the context within which the outputs of the QUASER project – the QUASER Hospital Guide and the QUASER Guide for Payers – would be introduced.
The study found that quality improvement guides are not the only relevant element for guiding quality improvement work in European hospitals. Rather, the research revealed that the notion of a guide is a way of thinking about the spread/dissemination of quality work that is not common to all the countries. The empirical analysis shows that the overall approaches to health care governance and the respective approaches to quality improvement differ vastly amongst (and even within) countries. Further, where guides are used, they are used in different ways, i.e. supporting ‘bottom-up’ work versus a 'top –down’ implementation of best practice. Therefore, the research agenda was broadened and instead focused on the notion of guidance, that is, methods and processes used to help health care organisations disseminate and implement quality improvement. Guidance can happen in many forms at various levels. It may include performance monitoring and subsequent development of spread and implementation activities, it may include training modules and improvement methodologies (e.g. Breakthrough Collaboratives or Plan Do Study Act), and it might focus on the importance of patient-centered improvement and development of implementation guides (guidance is also different from guidelines that are focused on methods used to attain substantive goals, and therefore aim to improve the quality of care).
It is therefore important to pay attention to both the material scopes of guides and processes of guidance in order to understand quality improvement work in the QUASER countries. Although many differences were found amongst the countries, a categorisation of countries according to their use of guides was possible. The typology deals with the ways in which the healthcare systems in the respective countries are organised, focusing on the main steering mechanisms. These steering mechanisms are related to the ways in which guidance of quality work is performed within respective countries. A distinction is made between three forms of steering: ‘Top-down’/hierarchical, ‘Bottom-up’/consensus based and Market based.
It is proposed to consider this typology as a tool to further explore the challenges that the local, national contexts pose to the QUASER project, where the aim is to provide guidance that is useable and understandable for users in the different countries.
Although guides have a different role to play in the distinguished types, all of them do use guides to some degree. Only in one type, the hierarchical one, do guides play a central role. The criteria of ‘good guides’ coming from this study are: the use of a simple design that allows for quick reference, flexibility and adaptability, the emotional engagement of audiences, the involvement of ‘insiders’ in the development of guides, and the careful balancing of different forms of knowing (evidence-based, experiential, tacit). To a certain extent, it can be argued, these principles can also form the basis for processes of guiding quality work in the QUASER countries.
1.1.3 WP4: Overview of hospital case studies
Project Director Professor Naomi Fulop
In this section, we summarise the findings from the country report from each of the QUASER participants: the Netherlands (WP4a); Sweden (WP4b); the United Kingdom (relating to England) (WP4c); Portugal (WP4d) and Norway (WP4e). These reports, written to a common agreed template, present an analysis of the fieldwork conducted in two hospitals (one high-performing (a) and one ‘developing’ (b)) in each of the five participating countries during the period (March 2011-April 2012). The criteria for selection of these hospitals are set out in the section on WP2. The fieldwork was informed by the results of a stakeholder workshop held in the Netherlands on 5th April, 2011(for further information see section 1.3.11). The reports incorporate an overview of the macro-level (national) healthcare context in each of the participating countries. These reports were the first stage in the analysis of the fieldwork; and provided the data for the cross-case comparative analysis conducted in work package 5 (see section 1.3.9).
Summary of presentation of findings
Each of the following country reports presented a summary of the following:
• the macro-level of healthcare in each country and identifies how macro-level factors influence the organisational management of quality in hospitals. Each country report provides a macro-level summary pertaining to 7 specified domains: healthcare context; funding and access; regulatory framework for quality; accreditation and monitoring; information availability; resources for quality; and patients’ rights.
• the meso-level relationships between the quality improvement programmes in place in each of the two hospitals (one high-performing (a) and one ‘developing’ (b)) studied in each country, the processes by which quality and safety are managed and organised, the cultural characteristics and indicators of quality of care.
• analysis in the high-performing hospital in each country of how quality and safety are organised in two clinical micro-systems. The reports identify how meso-level influences interact with those occurring at the micro-system level to influence quality, safety and the patient experience. Maternity care is one of the micro-systems selected in each country (with the exception of the Netherlands) as it is usually high on the agenda of quality improvement efforts and therefore data rich and will permit cross-case analysis. The other clinical micro-system was selected to contrast with maternity services in order to and allow a study of the differing effect of meso-level influences at the micro-level.
• explores through a longitudinal ‘tracer’ case study the process of implementing a quality improvement project in each of the 10 hospitals. In one hospital in each country the ‘tracer’ project relates to a healthcare acquired infection (HCAI) to permit cross-case analysis (Table 1 summarises which micro-systems and tracer QI projects were studied).
The country reports are structured using our preliminary theoretical framework of 8 quality improvement challenges, and our focus on multiple levels and their interactions, thus exploring the following questions in turn:
• How is QI structured, planned and co-ordinated? How is quality ‘built into’ hospitals?
• How are the politics of change negotiated?
• How are shared understandings & commitment to quality built?
• How do staff learn about quality and quality improvement?
• How are individual and collective enthusiasm for quality and quality improvement engendered and supported?
• How is the physical, informational, social and technological infrastructure used to support quality and quality improvement?
• What are the respective roles of the macro-, meso-and micro-system levels in terms of (a) the successful implementation and spread of quality improvement, and (b) sustained quality?
• How is QI led in the hospital?
• How is the external environment in relation to QI managed?
1.1.4 WP4a: Netherlands Case Study
Work Package Leader Professor Roland Bal
a. Summary of Macro-Level Context
The Quality of Care Act obliges hospitals to have a quality management system that provides (at least) safe, effective, and patient-centred care. The introduction of the Health Insurance Act in 2006 transformed Dutch healthcare into a market-based system. The idea is that the various parties in healthcare regulate healthcare through competition, building on transparency as an organising principle. The healthcare Inspectorate supervises compliance with the different healthcare laws. Healthcare Insurers negotiate contracts with hospitals based on quality and price, and some of them set their own quality agendas and norms (volume, efficiency, etc.) in order to buy quality of care for their insured. In addition, other parties such as professional bodies, patient organisations, scientific bodies, the Ministry of Health and the media play an important role in the quality of care debate. We can, therefore, conclude that there is a strong focus on quality and safety improvement pressures from outside hospitals by means of competition, supervision, and accreditation. We conclude that this leads to the marketisation of QI work in hospitals. The debate on the development of a national quality institute and on concentration of hospital care continues. Cost reduction has been more prominent on the quality agenda due to the economic crises in Europe.
b. Summary of meso- & micro-level work in hospitals
How quality is conceptualised:
The conceptualisation of quality varies between system levels in the Netherlands and differs between professional groups. The concept of quality covers all three aspects of quality (clinical effectiveness, patient-centeredness and safety) in both hospitals, but due to the national programme, the quality effort focusses on patient safety. Also, other more conditional quality aspects are mentioned, like timely access, availability of expertise, communication between departments and organisations.
How QI is structured, planned and co-ordinated and built into the hospital (structural):
Our research highlights that ‘fire fighting’ is an important strategy to deal with the increasingly fragmented external demands in the Dutch healthcare system. In this context, Hospital A is currently developing structures to co-ordinate and prioritise the increasing demands of external stakeholders. Hospital B, however, is struggling and invests in many QI processes at the same time. In both hospitals, QI is the responsibility of management, and a department specialised in QI work is available to support management and professionals. Also, quality is a topic on the agenda of almost every meeting in the hospital. Project-based work is the main format to work on QI themes in the Netherlands. Yet, healthcare professionals often feel overwhelmed by the multiplicity of projects and other QI responsibilities, and therefore lack the motivation to invest in projects, notwithstanding their possible intrinsic motivations. Moreover, projects generate challenges related to QI such as project continuation and internalising expertise.
How are the politics of change negotiated? (Political)
Formally, healthcare professionals (especially physicians) are considered to be in the lead of QI, while the upper management sets the policy framework. Yet, the increasing and diverging national QI demands have led to various non-aligned agendas and fragmentation. In this context, Hospital A is currently experimenting with systematic ways to prioritise and co-ordinate external demands in close alignment with physicians. It has devised formalised structures to prevent fragmentation and employs a demand monitor to co-ordinate audits and visitations internally, and prioritise the rising demands of external stakeholders. Hospital B is trying to do everything at once, resulting in too many activities and no overview. Our study highlights that informal negotiations matter greatly in making QI work. People are adapting and tinkering with formal QI agendas in all layers of the organisation to make QI possible in the first place. Quality is many peoples’ responsibility, is negotiated in character and always requires compromise.
How shared understandings & commitment to quality are built (cultural):
Our research shows formal leadership can be successful when leaders broker and thus translate (‘top-down’) quality demands into relevant agendas. Formal and informal strategies of brokering are distributed across various layers of the hospitals. Brokers are usually people with a good understanding of the hospital organisation, patient processes and have a strong (informal) network. They are often seen as trustworthy professional experts on QI topics and have informal communication skills. In Hospital A, (mainly invisible forms of) brokering are used for strategic management and priority setting. In Hospital B, invisible brokering is not central; instead, management focuses on hierarchical, formal management.
How staff learns about quality and quality improvement (educational):
Both hospitals invest in QI-related education. Several educational sessions on patient safety methods and management skills are available. Also, both hospitals strive to become learning organisations that move towards double-loop learning. For example, both organisations run culture improvement projects, where ideas such as blame-free learning from mistakes are central. Dutch hospitals are currently challenged to construct appropriate reflexive spaces for organisational learning about safety and quality.
How individual and collective enthusiasm for quality and quality improvement are engendered and supported (emotional):
We found that both hospitals are aware that making staff enthusiastic is a crucial component of QI and that lack of emotional support can hamper improvements substantially. Several activities to emotionally engage professionals are used: stage crisis, celebrate events and accomplished goals, patient narratives, idea jars, ‘games’ to support quality work (e.g. blue hand brigade, who is the mole).
How physical, informational and technological infrastructure is used to support quality and quality improvement (physical and technological):
Both Hospital A and B invest much effort in transparency and have set up quality information systems, such as a database for local protocols, an indicator-based monitoring system, and all kinds of systems for reporting, administrating and analysing adverse events and complaints. Both hospitals also invest many resources on collecting, monitoring and analysing performance measurement indicators. Respondents from both hospitals feel that the focus on planning and controlling QI takes away time and resources from actual QI work.
What are the respective roles of the macro-, meso- and micro-level systems in terms of successful implementation and spread of quality improvement, and sustained quality?
The Dutch regulated market based healthcare system, is heavily influenced by different stakeholders, with a variety of QI responsibilities and demands. Hospitals are, therefore, influenced by the macro-context (see above). The different stakeholders on the macro-level set incentives and provide guidance for hospitals to work on QI. We have seen this sets things in motion on the meso-level, but fragmentation of QI demands can also influence QI in a negative way. On the meso-level, macro-level demands need to be translated to the local hospital level and professionals on the micro-level need to be enabled to work on QI since in the end this is where QI needs to be implemented. We conclude that on all levels it is important to create reflective spaces in order to successfully implement QI.
How is QI led in the hospital?
Dutch hospitals separate QI work from quality control and this split is embedded in the organisational structure. Healthcare professionals are considered to lead QI while the upper management sets the policy framework. In both hospitals a diverse range of individuals are key players for QI and networks of topic experts, nurses and doctors with special expertise and interest take the lead in particular QI issues. Individual and network-based leadership is crucial.
How is the external environment in relation to QI managed?
Fragmentation of the macro-context providing multiple pressures (funding, regulations for patient choice and quality and safety requirements and compliance with professional standards) appear to influence a projectification of hospital QI, leading to multiple QI initiatives. Hospital B appeared unable to prioritise multiple QI initiatives and developed an integrated and hospital-wide strategy for QI. In contrast, Hospital A developed an innovative hospital-based ICT system, which assembled external QI demands and translated these into specific tasks for healthcare professionals. Normative pressures for compliance with professional standards appear powerful as doctors negotiate their individual contracts and have relative autonomy.
1.1.5 WP4b: Sweden Case Study
Work Package Leader: Professor Boel Andersson-Gare
a. Summary of Macro-Level Context
The Swedish healthcare system is dominantly public regarding financing, delivery and also control of the system. 21 county councils are the main bodies with the authority by law to tax the population and provide all needed health services. All aspects and parts of health services are integrated within the county council´s authority –delivered either by the county councils or by private providers on contract. This model has been fairly successful to reach overarching goals and solve basic conflicts during a long period since the early 1970´s, when different parts that were earlier run by the government were transformed to the county councils (primary care and psychiatry). The strongly integrated model has been especially successful for cost containment. Specific health reforms (initiated nationally), in recent years, have included a reform on access to primary care where the county councils are forced to implement a system where any provider (given certain criteria are met) is entitled to set up a primary care group practice. Patients can freely choose and register among the providers. A considerable increase of private providers (both not for profit and for profit) has been the result.
b. Comparison between Hospitals ‘A’ and ‘B’
How quality is conceptualised:
Hospital A is a publicly owned part of the county council. Quality work is organised through the core values, the Balanced Scorecard and the IT-based Guide that comprises outcome goals. The hospital uses balanced scorecards to plan, monitor and feedback clinical outcomes, both at hospital-levels and at department-levels. Hospital B is a limited company owned by the county council. The board is comprised of five highly-ranked and engaged business people. Quality is one of its central concerns. The management board consists of the CEO, the chief medical officer, the heads of the administrative departments and the heads of the departments. The hospital uses a balanced scorecard to follow-up on the 22 indicators within the contract with the county council. In both hospitals, top management (CEO, care development co-ordinator and chief medical officer) considered quality to include patient safety, clinical effectiveness and patient experience. Every employee is expected to have two jobs: their current care and their improvement work. Both hospital boards have decided on core values that should guide the hospital’s work. Both also define quality in various parameters within the hospital’s balanced scorecard. However, it seemed as if the personnel at Hospital A were more cognizant about the different QI issues than employees at Hospital B. Moreover, the integrated approach to quality is less well-known in Hospital B.
How QI is structured, planned and co-ordinated and built into the hospital (structural):
In both hospitals, there is a small quality department comprising the chief medical officer, the care co-ordinator, the hospital controller and the patient “ombudsman”. At the departments, the department heads are responsible for quality work but are supported by controllers and care developers. This is where improvement work is taking place.
How the politics of change are negotiated (political):
In both hospitals, QI work permeates the clinical activities and QI work is a natural part of the daily clinical work. Change is the norm due to the long history of QI work. Hospital management believes that most employees know the central values and that the hospital focuses on finance and access to health services. The clinical department heads need to participate in setting their goals. The hospital management assigns task forces for specific processes or projects organised as network groups across departments and across professions as means to assure that experiences and interests are aligned. However, it is often difficult to engage doctors in QI because their working days are often divided up between different departments.
How shared understandings & commitment to quality are built (cultural):
In both hospitals, shared understanding for QI issues and means is created through education and through the internal quality awards. Shared understanding and commitment are also created through the QI efforts themselves, within improvement teams and projects such as the outpatient clinics’ project at Hospital B. In Hospital A, some departments let employees rotate between different specialties and positions, as a way to increase understanding of colleagues’ tasks and problems. Similarly, the outpatient clinics’ QI project at Hospital B used collaborative learning through process mapping as a means for staff in different outpatient clinics and different professions to learn about each other’s’ work and the whole patient trajectory.
How staff learn about quality and quality improvement (educational):
Education in both hospitals is used to promote a shared understanding regarding QI and patient safety among the personnel. Hospital A has a well thought-out strategy for spreading knowledge, and educates the staff in QI and patient safety issues. Characteristic for the hospital is a wide range of educational activities supporting staff, formal education as well as informal training and support on-demand from the micro-systems. Hospital B has a less varied range and training is less extensive, where doctors and heads of departments are less knowledgeable about QI concepts and means than nurses.
How individual and collective enthusiasm for quality and quality improvement are engendered and supported (emotional):
In both hospitals, management uses different ways to support individual and collective enthusiasm for quality and quality improvement, e.g. quality awards, presenting good examples and interesting projects at conferences, highlighting people that have performed something extraordinary in clinical work.
How physical, informational and technological infrastructure is used to support quality and quality improvement (physical and technological):
Both hospitals use the intranet for publishing data about clinical outcomes, and they use it for training, although Hospital A is more advanced in the former aspect. In both hospitals, management finds it worrisome that reporting the same data in multiple databases is increasingly asked for because the county council (for hospital B), the Swedish Association of Local Authorities and Regions (SALAR) and the government imposes more compulsory reporting schemes for different issues such as pressure ulcers and patient safety culture. At hospital A, the IT-based management system is described as supportive. However, the IT-based patient record system as well as other IT-systems used for reporting patient and quality data is criticised for being too time consuming because it is insufficiently structured and difficult to overview.
What are the respective roles of the macro-, meso- and micro-system levels in terms of successful implementation and spread of quality improvement?
The county council finance care through direct taxes but also receives substantial funding from the state, increasingly through performance-based means such as the Care Guarantee. Moreover, the medical communities, the county council and the state provide knowledge-based governing through a number of means:
1. The “quality registers” that the clinical specialists have developed, report to and discuss trends and causes of variation revealed by the data. The National Board of Health and Welfare are working on improving the standards and also using the registers for recommendations.
2. The “open comparisons” that the Swedish Association of Local Authorities and Regions publishes, where the performances of individual departments can be compared. Data is mainly derived from the quality register described in point 1.
3. The compilation of evidence based care data offered by the Swedish Council of Health Technology Assessment (SBU).
4. The recommendations for best care provided by the National Board of Health and Welfare partly based upon the SBU evaluations partly by assignments to broad expert groups.
5. The recommendations from the specialist councils within the county council, which are an important input to the quality indicator work within the county.
6. There is also the Patient Safety law, the Care Guarantee Act, the Health and Safety Act and the Right to choose care provider Act, with corresponding instructions for reporting incidents and accidents and systematic quality work.
Thus, knowledge based governance is increasingly being integrated into the hospital management and economic governance systems. Common to both hospitals, there have always been many local initiatives, “flowers that bloom”, on the wards and clinical departments, as well as analyses of medical mishaps. This means that the hospital management sits “in between” micro- and macro-initiatives and various governing logics, often organised in parallel and competing with each other. The quality department liaises between the macro-context and the departments, through providing methodological quality support for knowledge improvement and through the follow-up process.
For Hospital A, QI is more an issue of knowledge than a financial one. The county council resources are salient to continued QI. The county-wide meeting is held four times a year including city councils, and hospital care is seen as an important part of this process. Also primary care is invited two times a year (although now less attendant due to privatisation). For hospital B, the county council governs QI efforts through performance-based contracts with care providers, private and public, for acute hospitals through the 22 indicators. However, the hospital management is critical of the use of remuneration for quality indicators. The indicators are defined by county council officials who are seen as not having enough knowledge about medical practice. This is why the quality network (composed of chief medical officers and care developer co-ordinators from the acute hospitals in the county) meet to try to negotiate these indicators, making them drive QI, rather than just being political symbols. There are knowledge-based resources available for hospital B as well but they are not described as that important – knowledge is also found at other hospitals (the improvement schools) or through consultant companies.
How is QI led in the hospital?
Both hospitals have clear QI leadership. However, it seems to be more deeply integrated in hospital A than B both in terms of the formal management structure and through the IT-based management reporting system. Knowledge about QI methods is quite widespread among the employees at hospital B. However, department heads and many doctors in hospital B often lack specific quality improvement training. In Hospital A, all department heads are trained in leadership and QI. In both hospitals, the heads of the departments are responsible for QI at the clinical level. The leadership at the micro-level units is not perceived to be ‘top-down’. It is characterised by a high degree of participation, communication and openness between the leaders and the employees.
How is the external environment in relation to QI managed?
Normative external pressures from professional colleges and associated organisations emerged as a key influence on QI in the hospitals studied. In Sweden, ‘there are few national regulations regarding health care’ and there is no national accreditation system. Health care standards were described as emerging from medical professional groups (physician societies that set informal standards and follow up via ’89 national quality registries. Balancing cost and quality demands were a key focus in both hospitals and instituted into structures that linked cost-efficiency to QI initiatives. Care Guarantees or contractual arrangements between healthcare providers and service users were identified as ways to provide structures for alignment of service user demands and hospital efficiency. Sweden B, uses a Balanced Score Card to follow-up on the 22 indicators within the contract with the county council and Sweden A, has an IT- based management system available on the hospital intranet
1.1.6 WP4c: England Case Study
Work Package Leader Ms. Susan Burnett
a. Summary of macro level context
Health policy has been devolved to the governments in Wales, Scotland and Northern Ireland with the UK government setting policy for England, and retaining authority for UK wide issues such as the regulation of healthcare professionals. Government reforms to the healthcare system over the last 10 years has been towards introducing an improved market system with a wider range of providers and improved commissioning. This has seen the introduction of payment by results (PbR) and a reorganisation of commissioning leading to the present reforms that will place commissioning in the hands of local General Practitioner commissioning groups. In terms of providers, there has been an expansion of independent treatment centres providing elective services and further independence for those acute trusts that reach a certain level of financial and efficiency standards as they become ‘Foundation Trusts’. Government reforms to the healthcare system over the last 10 years has also been towards introducing an improved market system with a wider range of providers and improved commissioning. The healthcare system is funded through direct taxation (a Beveridge System) with anyone resident in the UK being entitled to free healthcare. In the present climate there is major pressure on healthcare finances and all providers are now being required to make major savings in their budgets. At the same time the pressure remains to achieve national targets for access and quality/safety.
b. Comparison between hospitals ‘A’ and ‘B’
How quality is conceptualised:
Hospital A and B both articulated an explicit public narrative that defines quality in-line with government documents encompassing: patient experience; clinical effectiveness and patient safety. Hospital A managers suggested that meeting externally imposed targets had usefully developed understanding of quality and safety by focusing attention on reducing complaints, mortality, and avoidable readmission. This has directed the development of detailed action plans and timelines to address issues. In contrast, Hospital B appeared unable to operationally prioritise quality and safety due to factors such as restructuring and financial problems, and there seemed to be a disconnection between the narrated desire for quality and safety and concrete demonstration of this priority. Hence, Hospital B interviewees reflected on how quality has ‘slipped off the agenda’ in the face of financial pressures.
How QI is structured, planned and co-ordinated and built into the hospital (structural):
Both hospitals were following a process of re-structuring instigated by senior management, and both had experienced recent mergers. In both, responsibilities for quality improvement were referred to by interviewees as ‘dispersed’, for example in Hospital A, the matrons were described as influential in championing care for the elderly and ensuring compliance with standards. Respondents from both hospitals also acknowledged the roles of specialist improvement teams, providing training in the use of QI tools and techniques, and employing their knowledge and capabilities for facilitating improvement. Hospital B respondents generally reported governance structures as having been very bureaucratic, slow and centralised. Members of the senior management team were undertaking a radical overhaul setting out to impose a flatter devolved structure. Interviewees recounted their experience of a rapid turnover in senior management and how prior attempts at restructuring had floundered due to a financial crisis. They appeared cynical about this recent attempt at restructuring and suggested that the current financial crisis may lead to the Chief Executive centralising power, as has occurred in the past. Staff also reported how limitations in financial resources had led to a reactive quality and safety strategy focused on compliance and assurance.
How the politics of change are negotiated (political):
A range of powerful stakeholders appeared to drive quality improvement in both hospitals. In Hospital A, respondents emphasised the importance of internal stakeholders. For example, the Chief Executive and the senior management team involvement in executive ‘walk-arounds’. However, micro-level staff criticise these ‘walk-arounds’ and reflected that they felt scrutinised. Middle managers also mentioned this emphasis on ‘top-down’ leadership of change and how it limits clinical engagement in quality improvement. Despite this, a few entrepreneurial senior clinicians passionate about quality improvement in specific services (e.g. stroke care) had emerged. Hospital B was in the process of reconfiguring its internal management arrangements whilst also consulting the public on a major change to the location and delivery of its services. Tensions were evident throughout the organisation as a result of these ongoing changes, and the politics of change were high on the agenda of the senior management team.
How shared understandings & commitment to quality are built (cultural):
Cultural organisational characteristics, fostering commitment to quality, appeared marginalised in both hospitals by senior leaders prioritising financial stability. Managers in both hospitals report that re-structuring aims to enable a culture supportive of quality and safety and referred to a number of reward schemes for good practice. Hospital A micro-level staff, report supportive, open and trusting relationships between themselves but suggested that this is largely absent in their relationships with meso-level managers. Staff, generally, describe a controlling ‘top-down’ orientated culture and report that a priority for meeting targets overrides a focus on quality and innovation. The CEO in Hospital B considered the restructuring of management arrangements to be crucial for changing the culture by breaking down silos and enabling cross-organisational learning. The reorganisation was seen by staff as a start in developing a shared commitment to quality but emphasised that there was a long way to go.
How staff learn about quality and quality improvement (educational):
Hospital A staff were critical of formal training for QI and related how on-line training is less effective but was becoming the norm. Hospital B staff focused on how financial constraints had impacted on the resourcing of formal training. Spreading of best practice was described in both hospitals, for example, Hospital A staff were proud of their efforts to integrate exemplars of best practice into formal medical training, and identified how they used clinical guidelines to inform the development of improved practices.
How individual and collective enthusiasm for quality and quality improvement are engendered and supported (emotional):
Nursing Directors in both hospitals were seen as energising staff. In Hospital A micro-level staff reported the effectiveness of the Nursing Director ‘putting on a uniform’, and in Hospital B, staff reported the positive impact of the Nursing Director being ‘out on the wards’. Individual senior clinicians were also described as enthusing improvement by championing their ‘pet projects’. In contrast, staff in both hospitals reflected on the de-energising effects of ‘capacity pressures’, restructuring and constraints in financial resources. Staff in both hospitals appeared worried about losing their jobs.
How physical, informational and technological infrastructure is used to support quality and quality improvement (physical and technological):
Geographical dislocation of hospital sites was identified by staff, in both hospitals, as impacting on the spread of best practice. In Hospital B, two sites were a ‘45 minute drive apart’. This limited accessibility and appeared to compound difficulties in developing a central, common identity for the hospital. Use of technology to improve care was perceived by doctors in Hospital A as a necessity, and they highlighted the emphasis in manual maintenance of patient records. Nursing staff, however, reported an aversion to using computers. Hospital B managers recognised the value of information technology for improvement but suggested that the IT infrastructure was poor, and that this limited the real-time availability of data. In describing informational support of quality and safety, micro-level staff in both hospitals also drew attention to how they were overwhelmed by too much data, and how this led to difficulties in using data for improving practice. Both hospitals reported difficulties in clinical coding and failures in recording co-morbidities.
What are the respective roles of the macro-, meso- and micro-level systems in terms of successful implementation and spread of quality improvement, and sustained quality:
Macro-level factors appeared to drive quality improvement in both hospitals. Prescriptive criteria set out for hospital Trusts in England to achieve ‘Foundation Trust’ status also appeared to influence how both hospitals structured clinical governance and improvement. Similarly, restrictions in government funding appeared to limit time, effort and resourcing of improvement in both hospitals. Likewise, displacement of micro-level staff arising from funding restrictions was reported as de-energising staff and causing stress and lowering morale. Consequently, these powerful coercive external forces appear to have filtered down to direct ‘top-down’ changes in both hospitals. In both hospitals, Directors of Nursing, matrons and clinical leaders with responsibilities for improvement appear to have a vital role in energising and brokering improvement work. They were attributed with boundary spanning qualities and capabilities related to high status, reputation and linguistic, cultural, structural and contextual awareness. Some micro-level innovation and sustained improvement work is apparent in both hospitals, and in both cases, led by clinicians passionate about improving their particular speciality.
How is QI led in the hospital?
Leadership approaches differed in relation to which of the other 7 challenges was its focus. Nursing leadership was characterised as predominantly employing an emotional approach, often mobilising change through teams and focussing on the staff and patient experience. Medical leadership was often played out through approaches that tackled the political challenge, using networks, negotiation and influencing, knowing who to talk to and how to engage medical colleagues. In both organisations, structural approaches were favoured by the CEO, senior executive team and those in managerial positions describing structure as necessary in enabling culture change.
How is the external environment in relation to QI managed?
Hospital managers and clinicians emphasised that external pressures, related to external regulation, targets and accreditation supported and often directed health care improvement. In Hospital A, meso-level leaders appear to use regulation to give focus to ‘top-down’ stimulated QI strategy and, in Hospital B, meso-level staff report that an inspection by the regulator has focused attention in Hospital B on addressing failures, and has helped structure QI strategy. Provision of performance data to satisfy regulation and accreditation requirements appeared to drive information generation and overload staff with work. Constraints in funding were described as directing hospital reconfiguration strategies and efficiency-targeted improvement.
1.1.7 WP4d: Portugal Case Study
Work Package Leader: Professor Francisco Nunes
a. Summary of Macro-Level Context
All residents in the country are covered by the NHS, which is universal, comprehensive and almost free at the point of use, according to the Portuguese Constitution. The health system in Portugal is a network of public and private healthcare providers, each connected to the Ministry of Health. The Ministry of Health co-ordinates all healthcare provision of NHS and public health care delivery funding. It is also responsible for the regulation, auditing and inspection of private health services providers, whether they are integrated into the NHS or not. In Portugal, public health hospitals are allocated global budgets based on contracts signed with the Ministry of Health. This budget had been based just on diagnosis-related group (DRG) information, as well as on non-adjusted hospital outpatient volume. The assessment and monitoring of the quality of care is very poor, due to the unavailability of reliable quality information. Quality of care is measured by a set of indicators used by the Ministry of Health. Most of the indicators concentrate on hospital care and on logistical goals. Quality is monitored at hospital level; nevertheless there are National programmes, like Infection Control, that produce guidelines that assist evaluation. Moreover, these indicators are not published and the only accessible data are national data.
b. Comparison between Hospitals ‘A’ and ‘B’
How quality is conceptualised:
Quality is conceptualised in a very different way in both hospitals. At Hospital A (HA), there is evidence related to different components of the quality concept used in the QUASER project, which can be seen in its mission and values, its investment history, and especially the importance of clinical effectiveness, which is consistent with the training nature of the hospital. What is invisible at this hospital is an interconnection and of a common understanding of the different components, which reduces its potential effectiveness, sending out incoherent signals to hospital members. Hospital B (HB) resembles HA in its notion of clinical effectiveness, patient experience and patient safety, as these aspects are also part of the formal mission of the hospital and the development of structural elements. The biggest difference between them is the major efforts made by HB to develop an integrated vision of quality and how to foster it, which is rooted in the accreditation of this hospital, and promoted and supported by a cohesive team of ’activists‘, which is able to garner support from top management and the majority of the key elements at the micro-level
How QI is structured, planned and co-ordinated and built into the hospital (structural):
Both hospitals clearly show their concern with the creation of structure to formally support quality initiatives. However, the hospitals studied show a considerable difference. In HA, two important structural units were placed in clearly superficial positions within the structure (physically in the cluster’s other hospital, led by staff of this other hospital) and the powers given to the infection control committee seem insufficient. HB demonstrates not only more core structural elements, but its members have established work routines, based on the legitimacy they have achieved, which can influence the entire hospital, with clear signs of extensive institutionalisation of certain practices initiated by the structures responsible for QI.
How the politics of change are negotiated (political):
The evidence found in both hospitals is convergent regarding the processes of influence required for QI initiatives to be implemented by front-line professionals. As such, the support of top management is essential for signalling the importance of quality and facilitating professionals’ adherence, especially as most of the structures created to develop quality are advisory in nature. According to the data from both micro-systems, directors of the department and nursing service play a key role in regulating the initiatives coming from front-line professionals, but are also initiators of change and especially influential on the upper echelons to ensure support and resources. On the other hand, evidence was obtained in both hospitals on the importance of style of how professional members of the structures responsible for implementing quality initiatives acted.
How shared understandings & commitment to quality are built (cultural):
QI efforts encounter different cultural bases in both hospitals. In HB, one can see the development of a learning culture that comes from an ongoing accreditation programme, characterised by the increasingly widespread practice of the analysis of critical situations, their resolution and the knowledge sharing. HA has a strong identity rooted in its long history and the role it plays in the NHS. As a top hospital in terms of training, an end-of-line hospital and one that should be able to deal to a wide variety of cases whose complexity or rarity meant that other hospitals were unable to help, clinical effectiveness is core to its identity. This identity, whether at the level of service, or the hospital as a whole, is used as the framework for all quality initiatives. This cultural attribute is responsible for the constant search for knowledge and technological upgrading, but also for the major difference in status between doctors and nurses, groups that differ in how they value the various dimensions of the concept of quality used in the QUASER project: doctors are more concerned with clinical effectiveness and nurses are more focused on patient experience, and to a lesser degree, on patient safety. This professional “specialisation” is not so marked in HB, whose culture takes a more holistic view of health care.
How staff learn about quality and quality improvement (educational):
Being knowledge-based organisations, both hospitals demonstrate widespread practices designed to maximise updated knowledge, both formal and informal, attending or organising seminars, etc. However, in both hospitals, in the various departments, the use of outcome indicators as source for collective learning and the use of systematic methods of analysing problems or errors is not widespread practice, despite the increasing availability of indicators and the opportunity to compare them with other hospitals. In accordance with previously presented data on professional differentiation, the sharing of knowledge among doctors and nurses is a catalyst for development to be encouraged, considering the level of complexity and interdependence required for services provided.
How individual and collective enthusiasm for quality and quality improvement are engendered and supported (emotional):
The main mechanisms for ensuring enthusiasm for quality and improvement are linked in both hospitals to the direct recognition of the involvement of professionals and results obtained. For example, in HA, considering the importance of clinical effectiveness, an important part of the favourable emotions come from the recognition of the resolution of difficult cases, noticed by renowned professionals. In HB, the top leadership and members of the disseminating structures play a key role in maintaining and encouraging positive emotions about quality improvement. In addition, celebrating accreditation, albeit at the first level, was important for increasing collective self-esteem.
How physical, informational and technological infrastructure is used to support quality and quality improvement (physical and technological):
Concern regarding physical aspects and how architecture affects service quality is found in both hospitals, although at HA this has been one of the strategic priorities over the last five years. The effect it has had on co-ordination between departments and infection control is recognised, however, as seen in both hospitals, architectural shortcomings have an influence on professionals’ emotions. Both hospitals have made major investments in technology and this has contributed to improving the quality of service, in all its facets. However, an important aspect to be done is the integration of information systems, as they were developed for different professions or departments. In this field, HA’s experience may be considered important, be it in efforts towards integration or with regard to the involvement of professionals in customising features of the systems and the information produced by them.
What are the respective roles of the macro-, meso- and micro-level systems in terms of successful implementation and spread of quality improvement, and sustained quality?
This sector is strongly regulated, so the effects of external factors like macro-meso-micro-initiatives in QI are crucial to what happens in both hospitals. For example, the specialised colleges of the medical association are particularly important as they impose explicit quality requirements, which, unless adhered to, mean the closing of services and departments. On the other hand, as an expression of organisational, professional or even individual identity, specific individuals also take the initiative of introducing changes in service provision, managing to sell their legitimacy to the upper echelons and sometimes to external bodies. Within this context, we believe that it is an almost impossible task for top management to get a complete picture of what is happening in the hospital in terms of QI (see infection control tracer project).This is the more likely, the larger the hospital and the more comprehensive the concept of quality that we adopt is.
How is QI led in the hospital?
Although there has been no evidence of a clear strategy for developing leadership to stimulate QI efforts, the role of leaders is clear in both of the hospitals and both levels of analysis. In the case of HB, at the meso-level, leadership is playing two key roles: a) legitimiser of the initiatives of the group of 'activists', accepting and supporting the creation of a strategy and integrated structure for quality improvement and celebrating the victories associated with accreditation; b) calibrator of the tension between reducing costs imposed by the macro-level, and maintaining the quality level of services provided by the hospital (micro). While for HA, at the meso-level, we can consider that the two roles referred to in the case of HB are being fulfilled (legitimiser and calibrator). By playing this set of roles they are helping to manage one of the fundamental tensions that characterises the QI process in hospitals: the assimilation and use of new knowledge and using what has already been learned.
How is the external environment in relation to QI managed?
In Portugal, quality was described as not the only concern of top management and intermediate leaders and concerns about cost containment and efficiency were perceived as of extreme importance. Meso-level managers were described as ‘calibrators’ of tension between reducing costs imposed by the macro-level and maintaining the quality of services provided by the hospital. Normative pressures for compliance with professional standards appear to drive hospital QI and were perceived as having positive and negative impacts on QI. For example, external resourcing of QI was resourced by professional organisations. In Portugal B, the nursing association developed programmes geared to improving services. In Portugal A, some units were physically located in another hospital of the cluster, and were led by people belonging to a specific professional group.
1.1.8 WP4e: Norway Case Study
Work Package Leader: Professor Karina Aase
a. Summary of Macro-Level Context
The funding of the Norwegian healthcare services is mainly tax-based, and most services are offered free of charge. A national QI strategy and regional QI strategies are in place within the regional health authorities. The regulation of quality and safety in healthcare is based on an enforced self-regulation regime. No mandatory hospital accreditation system is in place in Norway. The national indicator data are published and available for the last three years. The Norwegian Directorate of Health and the Norwegian Knowledge Centre for the Health Services are the two leading public organisations related to quality improvement. With regard to the supervision of healthcare, the Norwegian Board of Health Supervision is the national regulatory authority. Norway’s Patients Rights Act (1999) covers all inhabitants. The purpose of the act is to ensure that the public has equal access to high quality healthcare services.
b. Comparison between Hospitals ‘A’ and ‘B’
How is quality conceptualised?
The conceptualisation of quality varies between system levels and professional groups, and depends on the type of provided services. Within the meso-level at both hospitals, we find a holistic perspective on quality and safety, where quality is conceptualised as a core of service provision. In considering the three aspects of quality( clinical effectiveness, patient safety, and patient experiences)the main emphasis at both hospitals relate to clinical effectiveness.
How is QI structured, planned and co-ordinated? How is ‘quality’ built into the hospital? (structural):
With regard to structure and size, we studied two hospitals with significant differences. Hospital A is a relatively small hospital in a rural area with a hierarchical organisational structure. Hospital B is a large university hospital located in one of the largest cities in Norway. Hospital B has a flat organisational structure, emphasising clinical empowerment. The formal quality structure at both hospitals is linked to regulatory demands and management lines. Quality is a managerial line responsibility at both hospitals.
How are the politics of change negotiated? (Political):
Large differences in structure, size, and organising for clinical empowerment are evident. Hospital A is small and has a traditional hierarchical organisational structure where the meso-level managers argue in favour of a combination of ‘top-down’ and ‘bottom-up’ leadership approaches. Hospital B, on the other hand, is large and organised as a flat organisation, where senior managers argue in favour of ‘bottom-up’ leadership approaches and a culture of clinical empowerment. If we categorise the hospitals as either ‘top-down’ or ‘bottom-up’ driven, the politics of change at Hospital A are negotiated ‘top-down’; and ‘bottom-up’ at Hospital B.
How are shared understandings and commitment to quality built? (Cultural)
The hospitals differ with regard to cultural characteristics. Taking into consideration that we have not studied the entire hospitals, but have focused on gaining insights into clinical micro-systems, our interpretation is that Hospital A has a more coherent and homogenous culture than Hospital B. Hospital A employs a more systematic ‘top-down’ QI approach and aims to build a culture of QI awareness through its Improvement Program. Both hospitals place a premium on providing high quality care; however, the strong emphasis on clinical effectiveness and the powerful professional groups at Hospital B imply that building a culture at Hospital B is founded on socialisation processes within the professional communities. There is a non-punitive open culture at Hospital A while Hospital B has a climate where speaking-up can be challenging, especially for newcomers and inexperienced personnel.
How do staff learn about quality and quality improvement? (Educational)
Regular educational activities at both hospitals are mainly related to professional development and clinical effectiveness. At the micro-level in both hospitals, we find similarity with regard to the importance of the professional development/educational nurse position. These professionals play a boundary-spanning role between the meso- and the micro-systems in energising QI within procedures and practice. They are dedicated and promote the continuous awareness to quality as part of service provision and development.
How is individual and collective enthusiasm for quality and quality improvement engendered and supported? (Emotional)
Quality champions exist at the executive level within both hospitals. At Hospital A, the quality champions are found within the executive management, including the CEO and key administrative staff. Moreover, Hospital A is characterised by clinical enthusiasts and professional pride; there seems to be a tendency for ‘top-down’ leadership approaches accompanied by empowering professional enthusiasts. At Hospital B, the CEO is also characterised as a quality champion due to the evident emphasis on quality at the strategic level accompanied by high expectations of professionals. Hospital B also has quality champions among clinicians at all levels. The professional enthusiasts play a vital role at this hospital, and the enthusiasm is challenged by the continuous emphasis on balancing the budget imposed on the hospital by both the inner and outer contexts.
How are the physical, informational, and technological infrastructures used to support quality and quality improvement? (Physical and technological)
Accessible, useful, and systematised QI information has been a key for Hospital A’s QI journey. The systematic use of data and managerial reporting on indicators has contributed to top performance on the national quality indicators. Hospital B significantly emphasises the use of data to monitor service performance in relation to clinical effectiveness. This is driven by clinical champions.
What are the respective roles of the macro-, meso- and micro-system levels in terms of (a) the successful implementation and spread of quality improvement, and (b) sustained quality?
National strategies, regulations, campaigns, and targets play an important role at the meso-level. The influence is related to increased QI expectations, targets, and reporting; support and legitimising of QI; and requirements to take action. The data indicate that the role of the macro-system is slightly stronger at Hospital A compared to Hospital B. The role of the meso-level in both hospitals is related to leadership involvement and the energised professional enthusiasm and support at the micro-level. The role of the meso-level in Hospital A is mainly related to efforts to foster a culture of appreciating and using quality information; integrating quality improvement into the ordinary operations of running a hospital; implementing an improvement programme as a strategic decision for QI; and empowering clinicians and heads of departments. The role of the meso-level at Hospital B is to broker the politics of change in order to nurture clinical empowerment and engage the strong professionals within the organisation. The role of the micro-level in implementing, spreading, and sustaining quality somewhat differs between the hospitals. At Hospital A, the micro-level relies on professional communities outside the hospital, keeps updated on professional guidelines, complies with internal guidelines and error management systems, and sustains a QI culture as part of the professional performance accepted in the system. At Hospital B, the micro-system comprises a larger professional community depending on clinical empowerment, research, and the development of new knowledge and skills. Hence, the studied micro-system at Hospital B is expected to play a leading role with regard to clinical effectiveness in a national perspective.
How is QI led in the hospital?
The importance of leadership is highly strongly emphasised at meso- and micro-level in both hospitals, but substantial differences exist regarding how the issue of leadership is enacted. The political leadership is less evident in Hospital A compared to Hospital B. At Hospital A the ‘top-down’ approach gives less emphasis on the skills to broker and manage the politics of engagement in QI. Hospital A has been subject to major organisational changes, but the micro-level attitude is more restricted to comply with strategic decisions, compared to the micro-level in Hospital B. The lack of emphasis on the political leadership is compensated by a strong emphasis on the symbolic, mobilising and strategic leadership at the meso-level in Hospital A, which has resulted in a shared culture of commitment to quality and safety, and QI accountability by line managers at the micro-level. At Hospital B, the politically credible leadership at meso- and micro-level is a key in the politics of engagement of the clinicians. Any kind of improvement effort or change in service provision is depending on clinical engagement, ownership and empowerment.
How is the external environment in relation to QI managed?
Overall, institutional influences appear centralised, for example, hospitals reported how funding of health is mainly set by Regional Health Authorities and reporting of hospital performance appears driven by increased accountability to meeting macro-level targets for quality and safety as well as service user involvement. Both hospitals structured QI to comply with regulatory demands. Funding constraints impacted on both hospitals resulting in restructuring and efficiency focused improvement.
1.1.9 WP5: Cross Case Analysis and Synthesis
Work Package Leader: Professor Glenn Robert
Objectives
The three objectives for work package 5 (WP5) were to:
• Synthesise the results from the hospital case studies and previous work packages
• Identify the implications for the design and content of the guides for hospitals and payers
• Gain stakeholder feedback on the emerging implementation guide
Methods
In order to address these three objectives, we began by examining the five country reports (WP4a-e) to identify similarities and differences between the countries and the 10 hospital case studies in which we had conducted our fieldwork (and between micro-systems within the 5 hospitals).
In WP5, we undertook the following six cross-case analyses in order to inform the elements (content and design) of the QUASER Hospital Guide and the QUASER Guide for Payers. Following discussion amongst the QUASER team, the questions for the analyses were selected to maximise the value of our cross-case learning for the development of the guides. Each of the country partners took responsibility for leading at least one analysis (as indicated in brackets) and their initial findings were extensively reviewed (first by email and then by face-to-face discussions) by all the remaining consortium members in order to ensure that they accurately reflected the data collected:
• How is ‘quality’ conceptualised at the macro-, meso- and micro-levels in healthcare systems in five European countries? (Norway)
• What are the important features of the interactions between the macro- and meso-levels that impact on quality improvement in hospitals in five European countries? (United Kingdom)
• Are there any distinguishing features in the approach to quality of (a) five ‘high-performing’ and (b) five ‘developing’ hospitals in Europe? (Portugal)
• To what extent , and how , do interactions between the macro-, meso- and micro- levels shape the quality of maternity care in hospitals in four European countries? (Sweden)
• What are the characteristics of successful implementations of quality improvement projects in relation to Healthcare Acquired Infection (HCAI) in hospitals in five European countries? (the Netherlands)
• How is quality improvement enacted in hospitals in five European countries? (United Kingdom)
A summary of each of these completed analyses is presented below. Our overall approach to data analysis was to use a preliminary theoretical framework (Miles, 1979) drawn from our literature scoping in WP2 and based on our adapted version of the ‘Organising for Quality’ framework (Bate et al, 2008) rather than a purely grounded theory (Glaser and Strauss, 1967), so that data analysis was a combination of induction (data-driven generalisation) and deduction (theory-driven exploration of hypotheses) (Langley, 1999).
As part of WP5, we held a translational workshop in London in September 2012, to share our emerging findings with the wider stakeholder group. At the workshops our emerging findings were reviewed by hospital leaders, patient representatives and payers for feasibility and practicality of implementation in each participant country. The implications of our research and the prototypes of the guides were reviewed and participants’ feedback was incorporated into the guides. A summary of the implications for the guides from the discussions at the workshop is provided in the findings section below. (Further information about the three stakeholder workshops can be found in WP7 section of this report). Drawing on the findings and lessons from the cross-case analyses and translational workshop, we then identified the important implications for the development of the QUASER Hospital Guide and the QUASER Guide for Payers; these are summarised in the findings section below.
Findings
Synthesis of results from the hospital case studies and previous work packages
Our synthesis of the hospital case studies helped shape our thinking about the development of the QUASER Hospital Guide and the QUASER Guide for Payers. Despite different national and local contexts, there were common themes arising from the synthesis as follows:
• QI work resides largely at the periphery or margins of hospital priorities and routines
• In many of the hospitals there are enclaves of high quality services but these are typically fragmented; multi-level and hospital-wide leadership systems for QI are rare but crucially important
• Hospital approaches to QI remain dominated by a ‘project-by-project’ mentality, rather than large-scale, system-wide approaches; without a strategic, hospital-wide approach that seeks synchronisation and complementarities ‘death-by-project’ is a commonly heard refrain in complex, differentiated and highly institutionalised organisations like hospitals
• The formal, rational ‘science’ of QI (with its focus on systems, tools and data) predominates over the informal, political ‘art’ of QI (with its focus on changing attitudes, behaviours and cultures)
• Governance, compliance and accountability (quality assurance looking outward) are key contemporary drivers rather than learning and cultural change (QI looking inward); aligning these agenda is a crucial task for senior leaders
• There is very limited patient and public involvement in QI (or, more basically, even the use of patient feedback on their experiences)
• QI is largely enacted through professional silos; leaders have a key role to play in brokering connections and relationships both between different professions and levels of a hospital
• There is a wide variation in the nature of managerial and clinical relations in the five EU countries we studied and this has implications for the way in which QI is conceptualised and implemented, and by whom
• Senior hospital leaders cannot rely on only one or a small number of specific approaches to QI; the key lies in the interactions between a range of solutions and the levels within a healthcare system and therefore a core task for senior leaders is one of alignment and entrainment
• The context in which a particular hospital is situated over time has a key bearing on the course of its QI journey; understanding and managing this context is another core task for senior leaders
• Sustained QI is not a discrete event; it is a long process that proceeds gradually over time, often with unexpected twists and turns, requiring both stable and adaptive forms of strategic leadership.
However, within our case study hospitals, there were ‘pockets’ where strategies were enacted which contradicted these common features, for example, relating to the cultural and educational challenges. Those hospitals that had an established track record of quality improvement had a long-term commitment to quality and stable leadership. In these hospitals, leaders embedded quality in the culture, aligned quality and cost reduction goals and motivated and energised staff. These themes provided important contextual information about the milieu into which the QUASER Hospital Guide and QUASER Guide for Payers were to be disseminated and implemented, and are ones that we took into account in our design work in WP6.
Stakeholder feedback on the emerging guides (from 2nd workshop Sept 2012)
Hospital Guide
1. There was strong support for the Guide as a reflective, self-assessment tool focusing on cultural and organisational issues. There was a need for the Guide, and it could serve to provide a focus on quality.
2. Clarify throughout that the tool is reflective and should be used to facilitate reflection and dialogue, not just as a tick box exercise.
3. Make it clear how the Guide fits with other Guides/initiatives/accreditation requirements. What does it add and what value will it bring?
4. The Guide should not only be used by top hospital leaders but has potential for facilitating dialogue between the levels within a hospital – vertically and horizontally. Also between hospitals and payers.
5. The Guide could be used to structure patient inspections
6. It should be dynamic and be capable of being used as a planning tool. For example, it should support recording of the reasons for the self-diagnosis, plans to change the rating and facilitate evaluation of whether what was implemented worked. In this way it could be used to track improvement over time and sustain quality improvement efforts.
7. There is a need to address potential pitfalls of using the Guide e.g. to prevent it being used as a tick box exercise, which was most likely in hospitals and countries with low QI maturity, to ensure that it is used as a dialogue tool to facilitate deep reflection.
8. Need for a diagnostic diagram of the 8 challenges.
9. More description/definition of the ratings is needed.
10. An issue to be decided is whether the challenges can be prioritised or whether they are all equally important and the organisation itself decides the priority. Once decided, this should be clearly described in the introductory section.
11. Consider whether similar challenges can be grouped.
12. The external demand challenge may not be needed and external demands could instead be incorporated into the other challenges. (BUT this may indicate lack of attention to external context and hence the need for this challenge).
13. Ensure that as much information as possible is provided before users complete the diagnosis to ensure accuracy and full understanding. E.g. Examples of good practice.
14. Provide reference material.
15. Build in the interactions between the challenges at stage 3. For example, diagnoses for some challenges may need to be revisited in light of overall results.
16. Provide links to the hospitals where the examples came from in order to understand context.
17. Link to best practice in each country.
18. Use good and not so good examples to illustrate do’s and - do not’s.
19. External context is not helpful at the start. There was a feeling that the external environment was always difficult but QI had to proceed anyway. (This may indicate that more attention is needed to the external environment, so we need to retain this focus).
20. Primary care is part of the external context.
21. QI maturity of the hospital is part of the organisational context.
22. The Guide needs to include guidance on patient rights.
23. There should be guidance on involving patients in QI, including ways to facilitate patient feedback, learn from patients, focus on patient experience, and use patients as a source of information.
24. The Guide should explicitly use language that is focused on patients.
25. Patient engagement could be considered as one of the challenges and needs to be included. This should take into account power differences between patients and professionals and how this differs across the EU.
Guide for Payers
1. There was strong support for the payer guide as a tool to facilitate dialogue between hospitals and payers about quality, although some tension was noted about whether it could be used only for dialogue or to apply sanctions.
2. Some tension was also noted about the role of payers in directing hospitals – some payers may desire to focus on indicators of quality and be less interested in how this is achieved.
3. Make the case for focusing on quality as well as cost. View that payers are too driven by cost.
4. Consider how the guide could encourage transparency (seen as a goal in long-term relationships).
5. Clarify the purpose and use of the guide.
6. Make it clear how the guide fits with other regulatory frameworks. What distinguishes this framework – evidence base should be emphasised?
7. Include guidance about how to involve patients.
8. Clarify how the guide deals with differences between departments within a hospital – not just the whole organisation.
9. Make it flexible so it can be adapted to the external context in each country -i.e. aligning internal systems with external demands will be different in each country.
10. Include guidance for payers to reflect on how their actions impact QI.
11. Consider including guidance about dialogue between payers and the doctors who advise them about which healthcare to procure.
12. Take into account the length of the contract and how this could create sensitivities and work against openness.
13. Include guidance on patient involvement in QI including avoiding tokenistic involvement.
14. Include guidance on how to incentivise QI through additional funding.
15. Involve patients in dialogue between payers and hospitals – i.e. in using the guide.
16. Recognise that patients are the payers.
Discussion
Our cross-case analyses in WP5 had strengths and some limitations. Important strengths were:
• The extensive and rich data collected to an agreed protocol across five EU countries
• The multi-level perspective that allows closer examination of the interactions between macro-, meso- and micro-levels in five different EU healthcare systems, and the implications of these interactions for QI
• The extensive ‘member checking’ of each of the reports in sections 3.1 to 3.6 by the other four partner countries.
Across WP5 as a whole, the results should be interpreted in the light of the following constraints and limitations:
• The process of selecting ‘high-performing’ and ‘developing’ hospitals in each partner country based on a common set of indicators was challenging, as described in the section on WP2
• The conceptualisation of ‘quality’ used in the QUASER project (clinical effectiveness, patient safety and patient experience) was a relatively narrow one, with other important dimensions emerging from our data (for example, efficiency) which were not able to be explored in depth
• Access to potential interviewees and opportunities to conduct non-participant observation sometimes varied between the five countries resulting in differing levels of detail and range of data being available to inform some of the cross-case analyses.
Notwithstanding the constraints outlined above, the findings from the six cross-case analyses in WP5 – informed the contents of the guides. Both guides were broadly structured around the 8 common challenges to implementing effective organisation-wide quality improvement and safety programmes.
1.1.10 WP6: Design of QUASER Hospital Guide and QUASER Guide for Payers
Work Package Leader: Professor Johan Calltorp. Report prepared by Dr. Janet Anderson
The aims of work package 6 were to design the QUASER Hospital Guide and the QUASER Guide for Payers, ensuring they contain:
• Guidance about the elements of an effective quality and safety improvement programme
• Guidance about the process of implementing quality and safety improvement methods
• Information tailored to organisations that are at different stages of the quality journey
• Information about taking into account the cultural context into which the methods will be introduced
• Information about common pitfalls
• Information about management structures to facilitate implementation
• Characteristics of effective organisations
• Links to detailed guidance and materials that can be used by hospitals
The design process started with the cross-case analysis undertaken in Work Package 5 which suggested a framework for the guides based on the 8 quality improvement challenges;
• Structural - how is QI structured, planned and co-ordinated?
• Political - how are the politics of change negotiated?
• Cultural - how are shared understandings & commitment to quality built?
• Educational - how do staff learn about quality and quality improvement?
• Emotional - how are individual and collective enthusiasm for quality and quality improvement engendered and supported?
• Physical and technological - how is the physical, informational, social and technological infrastructure used to support quality and quality improvement?
• Leadership - how is QI led in the hospital?
• External demand - how is the external environment in relation to QI managed?
The design process proceeded in steps which are described below.
1. The structure of the guides was determined. The structure includes the flow of information, how it is logically ordered and presented to users, and the process the user would be guided through, including the outputs at each stage of using the guide. The final agreed structure was based many inputs, including;
a. The eight challenges, which were derived from a combination of theory, scoping reviews of the literature and our extensive fieldwork. For the payer guide, a ninth challenge was added following feedback from the stakeholder workshop – the role of payers in facilitating quality improvement in hospitals through their own actions.
b. Findings of WP3, which suggested that guiding quality improvement work was essentially a social process and should involve more than following a set of instructions.
c. Feedback from our second stakeholder workshop. More detail about this is contained in sections 1.3.9 and 1.3.11 but briefly, we discussed our fieldwork findings and the emerging structure of the guides with stakeholders and sought feedback from them in a series of focus groups. The proposed structure was revised based on the feedback obtained.
The agreed structure was that users of the Hospital guide are navigated through a process of diagnosing the strengths and weaknesses of their response to the challenges and then to prioritise which areas they will focus on. For the chosen challenges, they then diagnose how well they are doing on the strategies for meeting the challenges, leading to a list of prioritised areas for action. They then read examples from the data of how other hospitals have met the challenges and there are a series of prompts to help them to think about whether any of these examples could work in their organisation. Each example links to more than one challenge, highlighting how a strategy might enable multiple goals to be achieved. Finally, they document an agreed quality improvement action plan that specifies who is responsible for which actions, a timeline, and requirements for resources. This plan provides a guide for their quality improvement journey which is then reviewed and revised as necessary over time.
The guide for payers is similarly structured and is designed to be used by payers to assess the quality of a hospital’s quality improvement strategy. Based on feedback from our stakeholder group, a number of changes were implemented:
• Payer organisations across Europe are varied (this was also a finding of WP3) with diverse roles, responsibilities and expectations of their hospitals. It was, therefore, best to design the guide so that it could be used as a dialogical tool between hospitals and payers, or as a tool payers may require their hospitals to use or as the basis of reporting.
• A ninth challenge was added to the payer guide. For the ninth challenge, payers are invited to reflect on their own role in facilitating quality improvement by, for example, providing incentives for quality, co-ordinating their requirements with those of other groups to avoid hospitals having to meet duplicate or conflicting requirements. Using the guide they diagnose their own strengths and weaknesses and devise their own action plan for facilitating quality improvement.
The final structure of the guides is shown in Figure 1 using the educational challenge as an example.
2. The purpose and users of the guides were determined. This was based on input from our findings from WP3, which emphasised the social process of guiding and the theoretical framework we used, which emphasised the organisational and cultural aspects of quality, including such things as building shared meanings and enabling and nurturing initiatives. We discussed emerging ideas with the stakeholder group who confirmed our thinking that the hospital guide should support a facilitative dialogue about quality improvement with the aim of assisting organisations to devise, implement and review their quality improvement strategy. Participants in the process could be executives, managers, clinicians, or teams. The payers guide was also viewed as a tool for dialogue between payers and hospitals or as a way to assess the effectiveness of a hospital’s approach to quality improvement. It was decided that the guides are not intended to be used for the summative assessment of hospital quality, but rather as a guide for hospital leaders for planning and implementing a quality improvement strategy and for payers to assess the adequacy of a hospital’s quality improvement strategy.
3. The structure of the guides (shared by both guides) was populated with content. This was an extended, iterative process with the following steps:
a. The definitions of the challenges were refined and confirmed based on reviewing the cross case analysis report.
b. Descriptions of strategies for each challenge were developed based on high level reading of the case study country reports.
c. Strategies for each challenge were used as a coding framework to extract data related to those strategies from the case study country reports. We searched for examples of the strategies hospitals used to respond to the quality improvement challenges and also searched for strategies that we had not yet captured. During this process the developing content was reviewed and refined, gaps were identified (e.g. strategies that were not supported by the data) and more details were sought from partner countries.
d. The developing examples were edited and reviewed for applicability, generalisability and clarity. The most appropriate examples were chosen for inclusion in the guides. Many examples fitted with more than one challenge and this was highlighted to emphasise how actions could potentially meet a number of different goals. Users would be encouraged to devise strategies that would address more than one challenge.
e. Prompts were added to assist the dialogue. For example, when reading examples of strategies other hospitals have developed, users would be prompted to consider how they currently deal with the issues raised and whether any of the examples presented could be used or adapted to their context.
4. The near-final refined list of examples was then reviewed by researchers in each country to check the validity of the content.
5. Prototypes of the guides were produced.
6. A third translational workshop with stakeholders was held to review the prototypes. In a structured facilitated process, stakeholders gave detailed feedback on all aspects of the prototypes, which was incorporated into the guides. In summary, their recommendations were:
Hospital Guide
The groups gave very specific feedback about all aspects of the guide, including wording of various aspects. Broadly, the implications were:
i. Confirmation of the purposes and approach
ii. Broaden user group, not just senior leaders
iii. Need for clearer definitions of challenges, strategies, terms throughout
iv. Use graphics to illustrate that challenges are all important and are linked
v. Confirmation of value of examples and these should be more central
vi. Develop action plan to document agreed actions and to include time, resources, people
vii. Confirmation of using rating scale to diagnose current strengths and weaknesses
viii. Confirmation of value of spider diagrams to show outcome of diagnosis
Guide for Payers
i. Confirmation of purposes and approach
ii. Acknowledge that it will be used in different ways depending on users and context
iii. Need for clearer definitions throughout especially term ‘payer’
iv. Clarify purpose and how it should be used
v. Clarify action plan and develop it
vi. Confirm use of rating scale to diagnose strengths and weaknesses
vii. Emphasise interaction of challenges
viii. Examples are very valuable
ix. Titles of two guides should be similar so complementarity is clear
7. Content was then revised to include stakeholder feedback, and refined to ensure that it was consistent, met the aims we had for the guides, and was clear and usable. Professional copy-editing services were used at this stage to finalise the text.
8. A professional designer was employed to ensure the guides were appealing and user friendly.
1.1.11 WP7: Translation and dissemination
Work Package Leader: Dr. Janet Anderson
The overall aims of this work package were:
• To develop a dissemination plan and co-ordinate the dissemination activities of the QUASER project
• To co-ordinate a series of translational stakeholder workshops.
A project dissemination plan was written and all dissemination activities, including preparation of academic papers was co-ordinated via this work package. Full details of all dissemination activities can be found in section 1.4 of this report. WP7 co-ordinated three translational workshops held during the course of the QUASER project the first in the Netherlands and the latter two in the UK bringing together hospital leaders, payers and patient representatives from across Europe to gain their feedback and input into the design of the Guides. Our approach to the translational element of the project was to engage with a stakeholder group of hospital managers, payers and patient representatives as soon as possible. Detailed reports of these workshops were written to enable us to capture the recommendations in the ongoing design work.
Stakeholder Workshop 1
We held the first workshop in the Netherlands in April 2011 to discuss the project’s aims and methods and to gain feedback on their needs in relation to quality improvement and to explore the influence of national context and healthcare system structure on quality improvement. Nine European countries were represented at the workshop with an additional participant from Taiwan. They represented a range of senior managers and clinicians and purchasers of healthcare. The countries were as follows:
• Estonia • Denmark • Spain • England • Turkey • Portugal • Norway • Sweden • The Netherlands • Taiwan
From the discussion about national context, it was clear that there were major differences between the countries in the way healthcare is resourced, organised and delivered, and the way that quality is regulated. These differences mean that what is needed to progress quality improvement varies between countries. For example, in some countries quality improvement is at such an early stage of development that it is not on the agenda of healthcare managers but in other countries there are well developed systems of support for quality improvement. Many participants mentioned the fact that cultural differences are also important and will affect how receptive people are to quality improvement guides and what they would like to see in those guides. The participants expressed some scepticism that a single guide would be relevant and applicable across Europe. They discussed various ways to address this problem such as allowing a high degree of customisation of the guide.
Stakeholder workshop 2
A second workshop to review the emerging structure and design of the guides was held in London in September, 2012. The workshop was attended by 23 participants including 6 payer representatives, 7 patient representatives and 10 hospital representatives. They came from 14 European countries:
• Estonia • Denmark • Spain • England • Romania • France • Norway • Finland • Netherlands • Estonia • Lithuania • Italy • Hungary • Belgium
The proposed purpose, structure and users for the guides were reviewed. The participants gave detailed feedback about the approach that the guides should take, the needs of the user groups and how material should be presented. The stakeholders were extremely supportive of the proposed development of guides to support dialogue and for addressing the organisational and cultural factors that are important for ensuring the success of quality improvement initiatives. The outcomes of this workshop informed the cross case analysis undertaken in work package 5 and fed directly into the outcomes of this work. More detail about the feedback given by stakeholders is contained in section 1.3.9.
Stakeholder workshop 3
A final workshop was held to review prototypes of the guides in May 2013 in London. There were 3 payer representatives, 4 patient representatives and 5 hospital representatives from 8 countries.
• England • Denmark • Hungary • Belgium • Lithuania • France • Norway • Netherlands
Stakeholders gave very specific and detailed feedback about the content of the guides and highlighted aspects that were unclear and others that needed to be strengthened. The results of this consultation with stakeholders were fed directly into the design work in work package 6 and more detail about the general advice given by the stakeholders can be found in that section. The stakeholder group was extremely supportive of the project and interested in the outcomes. We have disseminated the guides to them directly and intend to engage with them in any future work to implement the guides.
Potential Impact:
1.4.1 Dissemination
We have actively disseminated our results throughout the project to both practitioner and academic audiences. There have been five papers published in academic peer reviewed journals (and many more are under review and in preparation), 19 invited plenary presentations at high profile conferences and policy fora, and presentations at 23 national and international conferences. Plenary presentations and conference presentations were attended by both practitioner and academic audiences. Dissemination activities focused on clinical and management audiences occurred through 12 seminars and workshops, national workshops in each partner country, an exhibition stand at the conference of the European Health Management Association in 2013 and a prominent article in the Health Services Journal (a major weekly journal read by practitioners and managers) with the QUASER project featured on the front cover of the journal. Dissemination activities are ongoing. In Portugal, a formal presentation of the QUASER project is being planned for October 2013 for national stakeholders (managers of hospitals, State, insurers), to summarise the results and discuss the implications for Portugal national stakeholders. The scientific co-ordinator has been invited to speak to the European Commission expert group on patient safety and quality of healthcare in Brussels on 4 November 2013, and a joint session will be held with the DUQuE project team at the NHS Confederation Hospitals Forum and the European Hospital and Healthcare Federation (HOPE) conference on 28-29 October 2013. The Norwegian team will give a presentation of the Norwegian QUASER results at the National Patient Safety Conference in September 2013.
Free access to the Guides is via the UCL website, where the Guides are available to download. A QUASER section appears on the website of Department of Applied Health, UCL which contains the two QUASER guides, along with project relevant material including newsletters, project leaflets, details of publications and information about the project. The web address has been circulated to interested parties including our stakeholders, Expert Advisory Board members, people on our mailing lists, and other interested parties who have requested the Guides. Information about the guides was circulated by HOPE, EHMA and the NHS Confederation, reaching over 2,000 hospitals in Europe. The QUASER guides can be downloaded from here: www.ucl.ac.uk/dahr/quaser . The following sections contain more information about dissemination in each partner country.
1.4.2 Netherlands
There are several avenues through which the work done has had, and can have, an impact on healthcare in the Netherlands. During the project we were able to have a significant impact on the two hospitals studied in terms of translating quality information between hospital levels. In both hospitals, feedback sessions with senior management and quality staff have been organised, in which we were able to mirror our findings of the ethnographic research (interviewing managers, staff and professionals, patient representatives, as well as outside stakeholders such as insurers and inspectors; observing meetings and clinical work, analysing documents) to the hospital. Those meetings were generally found to be very productive and hospitals stated they learned much about their own quality improvement processes. This is exemplified by the urge of both hospitals to set up a network to further work on quality improvement, using the guide as developed in the QUASER project as a framework.
On the 29 May 2013, we organised a conference, presenting the results of the QUASER project to an audience of about 200 people from the Dutch health sector, including many hospitals, patient and professional organisations, insurers and the healthcare inspectorate. The meeting was organised together with the Dutch team of the DuQue project, thus enabling the presentation of both projects. Apart from presenting general results, specific sessions were organised on e.g. professional-management interactions, patient involvement, knowledge brokering, the use of qualitative and quantitative data, and the role of hospital boards in QI. We received many enthused reactions from the audience, as well as some press overage (both radio, professional and general media; more have shown interest in the QI guide, once it is published). We have made an e-book with the results from the conference, which was published at the end of June, 2013.
1.4.3 England
The QUASER study has generated a lot of interest from the healthcare sector in England. Discussions with managers and clinicians indicate a need to focus on the organisational and cultural aspects of quality improvement and that organisations require assistance with this. There have been many inquiries from hospitals interested in assisting with development of the guides, disseminating information and using the guides. The project has raised awareness of the importance of organisational factors in determining quality improvement and the value in addressing these challenges.
The results of the study have been disseminated widely in England at major patient safety and health services research conferences. Many invited keynote presentations have been given, including at major national conferences for NHS practitioners and academics, including the BMJ Forum on Quality and Safety, the NHS Confederation, the Health Services Research Network Conference. Discussion at these meetings has been enthusiastic, especially among hospital managers who acknowledge that current approaches to quality improvement do not adequately address the organisational and cultural factors that are crucial to the success of any initiatives. The results were directly disseminated to hospital chief executives at a forum organised at the Patient Safety Congress, held in Birmingham, UK, 2013. This included panel discussion and interaction with the audience who were greatly interested in a new approach to quality improvement that addresses the challenges of quality improvement implementation, especially cultural factors.
1.4.4 Sweden
In general terms the QUASER project has raised a lot of attention among many actors in the healthcare system in Sweden; locally, where the research has been performed, regionally with the involvement of people in the wider QUASER reference group, nationally through presentations for leaders and drivers in different bodies and in the scientific community nationally and internationally. Discussions on the conceptualization and deeper understanding of quality improvement as both a social and technical endeavor have been energized by the approach and the findings in the QUASER project. A general curiosity about the Guides and how they can be used in the future has also emerged among leaders and professionals who have been exposed to the idea.
Locally, the participating hospitals have found it valuable to be part of the research. They have expressed it as a “learning journey”, with opportunity for reflection and feed-back, different from when consultants come in and tell how things should be. We believe there will be an ongoing conversation on these issues when the Guides become available. In addition, the planned research project (see below) will be a way to support sustainability in these organisations and further learning together with other hospitals which will be included.
On the national arena, we have had several opportunities to present and discuss the results and processes in the QUASER project. These have been both meetings for practice and meetings for researchers, and we feel there is a growing interest in this field of comparative research in Sweden, which has been scarce before. Especially in today´s situation with many political and structural changes in healthcare, comparative research with an interactive approach can be helpful in future decision making. We have also had the chance to present data for national bodies like the SALAR, which is the interest organisation for county councils and municipalities. They showed a particular interest on the Guides and how these can be helpful when adapted to the Swedish context.
1.4.5 Portugal
We contacted hospitals A and B in order to present the results of our study and also to get feedback from the study results. We found that there was no interest on the part of the Hospital A to meet with the Portugal QUASER team. On contrary, Hospital B showed great interest in meeting. A meeting of about 3 hours was held to present the study, attended by eight people from the hospital including two members of the Hospital Board. After the presentation of the results we obtained a very favorable assessment of the results and also discussed the implications that the results have for the future of the hospital.
The guides were also sent to the department of health quality from the health ministry the Hospital Guide. We will maintain contact with this department in order to clarify and assist them in the implementation of these guidelines among hospitals. In September 2013, there will be a press conference to present the Hospital Guide to the media and we also intend to send the Hospital Guide to all public and private hospital boards.
1.4.6 Norway
There are relatively few multilevel studies related to quality and safety in healthcare in Norway. Hence the QUASER results are of interest to multiple actors in the healthcare system. The qualitative orientation is providing in-depth knowledge on how hospitals organise and establish quality and safety processes and is of high relevance for other hospitals, governmental organisations and research bodies in Norway. The focus on different quality dimensions in the project – clinical effectiveness, patient safety and patient experience – is interesting, especially since the results show a clear need to increase attention to patient experiences in Norway. The results have highlighted the large macro-level attention to patient experiences and the difficulties for hospitals to involve patients and patient experiences in quality improvement processes.
The Norwegian QUASER team has published results from the project since 2011. The Strategic Communication Department at UiS has been involved in spreading the project to the media and in the UiS magazine “UniverS” http://www.uis.no/om-uis/nyheter-og-presserom/magasinet-univers/. The project has been presented at scientific conferences nationally and internationally, in teaching activities at UiS and other universities and university colleges in Norway, in presentations for external bodies such as the Regional Health Authority, and in conferences for managers and professionals such as the World Hospital Congress 2013.
The project has also been presented for the Ministry of Health and Social Services as input to the first Norwegian Report to Parliament on Quality and Safety Healthcare. A spin off of this presentation was a request from the Ministry to the research group to prepare a note on different countries’ systems for investigating and learning from large scale adverse events in healthcare. As a response we involved all QUASER partners and submitted a note (Wiig & Njå, 2012) to the Ministry of Health and Social services with examples of how all QUASER countries and USA handle this, and recommended that Norway in the next period should assess how Norway could take a leading role in establishing a national independent investigation system for healthcare. This was taken into the Report to the Parliament (10, 2012-2013) (“High Quality – Safe Services”) http://www.regjeringen.no/en/dep/hod/documents/regpubl/stmeld/2012-2013/meld-st-10-20122013.html?showdetailedtableofcontents=true&id=709025 .Recently, the Norwegian QUASER team member Siri Wiig has been appointed by the King, in his meeting with the Government, as member of a Norwegian Public Commission: http://www.regjeringen.no/nb/dep/hod/pressesenter/pressemeldinger/2013/utvalg-skal-vurdere-oppfolging-alvorlige.html?id=731236 .The Commission will, over a two year period, as a response to the specific topic in the Report to Parliament, develop a NOU (Norwegian Public Inquiry) suggesting changes in how the society should respond to large-scale adverse events and suspicion of violation of the law. The Norwegian QUASER team has presented the QUASER Hospital Guide to the case study hospitals, the Regional Health Authority, and to the Norwegian Knowledge Centre for the Health Services. We have suggested for the Knowledge Centre to include the guide as part of a new Patient Safety Program that will be developed in Norway when the National Patient Safety Campaign ends in 2013. Both case hospitals in the Norwegian QUASER study are interested in implementing the Guide.
1.4.7 Joint dissemination activities with the Deepening Understanding of Quality Improvement in Europe (DUQuE) project
Meetings have been held with members of the DUQuE project, throughout the course of the QUASER project, to discuss methods and approaches. The projects take different but complementary approaches to quality improvement and so we have presented together to practitioner and academic audiences including:
a. Joint sessions of QUASER and DUQuE at the International Forum for Quality and Safety in Healthcare (workshop in 2012, presentation in 2013).
b. Presentations by QUASER and DUQuE at the DNV European workshop on Risk Management for Patient Safety (2012).
c. Joint plenary session by QUASER and DUQuE at the UK Patient Safety Congress (2013).
d. Joint presentation of QUASER and DUQUE results to approx 200 participants from the Dutch healthcare sector by Roland Bal (Netherlands team, EUR). See section 1.4.2 for more information.
1.4.8 Dissemination beyond partner countries
QUASER researchers have been invited to present the results at fora across Europe including at University College Dublin, Dublin City University, French Ministry of Health, the Polish Association of Trade Unions, Karolinska Institutet, DNV workshop on Risk Management for Patient Safety held in Brussels, Nordic Conference on Research in Patient Safety and Quality, NHS Scotland, and St. Andrew’s University Scotland. The results of the study have also been presented in Canada at the Canadian Association for Health Services and Policy Research and in Australia at the Australian Institute for Health Innovation at the University of New South Wales.
1.4.9 Exploitation
The Quaser Hospital Guide is for senior leaders and managers in hospitals to develop and implement their quality improvement strategy. It is designed as a tool for dialogue between leaders and clinical teams and it supports them in identifying weaknesses in their approach to quality improvement and in developing their strategy. The guide is structured around 8 quality improvement challenges that should be addressed by all hospitals seeking to improve the quality of care; structural, educational, cultural, political, emotional, physical and technological, leadership and external demands. The Quaser Hospital Guide is freely available and can be used by hospitals to improve the quality of the care they provide to patients. Researchers can also use the guide in future projects. Results of the Quaser research identified that although quality improvement guides are useful, there is also a need for quality improvement guidance – an active process of working with hospitals to improve quality. Members of the Quaser team either have received funding, or have submitted funding applications to explore how the Quaser guide can be best implemented. Further details are in the following sections.
The Quaser Guide for Payers is for payer organisations to assess the adequacy of a hospital’s quality improvement strategy. Quality is a key concern of many payer organisations and focusing on how a hospital is improving quality in addition to assessing indicators of quality, will be a positive influence on quality. The guide is designed to assist payers to enter into dialogue with hospitals about their quality improvement strategy. It is structured around 8 quality improvement challenges that should be addressed by all hospitals seeking to improve the quality of care; structural, educational, cultural, political, emotional, physical and technological, leadership and external demands. There is a ninth challenge for payers; how can a payer organisation facilitate quality improvement in hospitals. The Quaser Guide for Payers is freely available and can be used by payers to assess the adequacy of a hospital’s quality improvement strategy. Researchers can also use the guide in future projects. Payer organisations are diverse and have different needs, roles and responsibilities. Further research is necessary to investigate how the guide is used by different types of payer organisations and to investigate how to implement the guide with payers in practice.
In the Netherlands, we are now talking to some 12 hospitals (and more have shown interest) to set up a network of hospitals working on QI, and intend to involve both healthcare insurers and the healthcare inspectorate of the Netherlands, who has already shown interest. Generally, the message of QUASER – that an integrated QI perspective needs to be developed, bringing QI to clinical work floor levels – is taken up very positively. The Dutch members of the QUASER stakeholder group, as well as the participating hospitals, have been and are very instrumental in getting others on board. We are also talking to the healthcare inspectorate to use the QI and payers guides in its newly developed ‘system supervision’ and to one of the big healthcare insurers, to use the payers guide in its audit system.
In England, we have received funding to implement the hospital guide as part of a large scale initiative of applied health research funded by the National Institute of Health in England. This three year project, starting January 2014, will investigate the effectiveness of an organisational development (OD) focus for implementing the guide and evaluate the effect on quality improvement knowledge and activities. Currently, 7 hospitals have signed up to participate in the study and will pay directly for the OD intervention. This indicates there is a need for the guide and a real willingness on the part of hospital leaders to invest in quality improvement activities that address their needs. It also confirms the value of our conceptualisation of quality improvement as a social process as we will be working directly with hospitals to facilitate the development of their quality improvement strategy.
In Sweden, a research application on the issue “Integrating knowledge by an interactive guide for quality improvement in health care” has been sent in to a Swedish research foundation (FAS). It is a first step application aiming to prepare and plan for introducing the guide at a selection of hospitals in Sweden in order to study how it affects quality improvement work in the health care system and primarily at the hospital and clinical level. A multiple case study with a cross-case analysis involving four hospitals and eight clinical departments in Sweden will be planned during the preliminary study. The selection of cases, criteria and methods will be made so that comparisons will be feasible between the four countries Sweden, England, the Netherlands and Norway where national guide implementation studies also are planned. An interactive research approach with methodological triangulation will be used providing continuous feedback, knowledge integration and dissemination to the health care practitioners.
The Norwegian QUASER team has been involved in applications for funding to the EU (2012) and to the Norwegian Research Council (NRC) (2013) on implementing the QUASER Hospital Guide into Norwegian hospitals and nursing homes. Collaborating partners in the NRC application were other universities and university colleges, hospitals, nursing homes, and municipalities in Norway. The application got a good review, but was unsuccessful. The application will be further developed and submitted to the Regional Health Authority in September 2013.
1.5 References
Bate SP, Mendel P and Robert G. (2008) Organising for Quality, The improvement journeys of leading hospitals in Europe and the United States. Oxford; Radcliffe Publishing
Bevan H, Ham C, & Plsek P. (2008). The next leg of the journey: how do we make ‘High Quality Care for All’ a reality? NHS Institute for Innovation & Improvement; Warwick
Boaden R, Harvey G, Moxham C, & Proudlove N. (2008). Quality improvement: Theory and practice in healthcare. NHS Institute for Innovation and Improvement.
Burnett S, Renz A, Wiig S, Fernandes A, Weggelaar AM, Calltorp J, Anderson JE, Robert G, Vincent C, Fulop N. (2013). Prospects for comparing European hospitals in terms of quality and safety: lessons from a comparative study in five countries. Int J Qual Health Care. 25:1-7.
Catio M, Albolini D, & Vannucci A. (2005). An organizational multi-level model for the analysis of failures in medicine – The case of fortuitous connection. International Conference on Healthcare Ergonomics and Patient Safety, Florence Italy. Available at: http://www.salute.toscana.it/sst/grc/pubblicazioni/organizational_multi-level_model.pdf
Directive 2011/24/EU of the European Parliament and of the Council of 9 March 2011 on the application of patients’ rights in cross-border healthcare. 2011.
Fay D, Borrill C, Amir Z, Haward R, West MA (2006). Getting the most out of multidisciplinary teams: A multi-sample study of team innovation in health care. J Occup Organ Psychol. 79:553–567.
Firth-Cozens J. (2001). Cultures for improving patient safety through learning: The role of teamwork. Quality and Safety in Health Care. 13 (Suppl 2): ii3-ii9.
Glaser B, and Strauss, A. (1967). The discovery of grounded theory. Chicago: Aldine Langley
Hartmann CW, Rosen AK, Meterko M et al. (2008) An overview of patient safety climate in the VA. Health Services Research, 43(4): 12631284.
http://eur-lex.europa.eu/LexUriServ/LexUriServ.do?uri=OJ:L:2011:088:0045:0065:EN:PDF
Langley A. (1999) Strategies for theorising from process data. Academy of Management Review. 24(4): 691710.
Lukas CvD, Holmes SK, Cohen AB, et al. (2007) Transformational change in healthcare systems: an organisational model. Health Care Management Review. 32(4): 309320.
Miles M.D. (1979). Qualitative data as an attractive nuisance – The problem of analysis. Administrative Science Quarterly 24: 590601.
Robert G, Anderson J, BurnettS, Aase K, Andersson-Gare B, Bal R, Calltorp J, Nunes F, Weggelaar A, Vincent C, Fulop N, and the QUASER Team (2011). A longitudinal, multi-level comparative study of quality and safety in European hospitals: the QUASER study protocol. BMC Health Services Research.11: 285. doi:10.1186/1472-6963-11-285
Shipton H, Armstrong C, West M, & Dawson J. (2008). Theimpactof leadership and quality climate on hospital performance. Internation journal for quality inhealth care. 20: 439-45.
Singer S, Meterko M, Baker, L et al. (2007) Workforce perceptions of hospital safety culture: development and validation of the patient safety climate in health care organisations survey. Health Services Research. 42(5): 19992021.
Spencer E, Walshe K. (2009). National quality improvement policies and strategies in European healthcare systems. Qual Saf Health Care. 18(Suppl I):i22–7.
WeinerB. J, Alexander J. A., Baker L. C., Shortell S. M., & Becker, M. (2006). Quality improvement implementation and hospital performance on patient safety indicators. Medical Care Research and Review,63(1): 2957.
West E. (2000). Organisational sources of safety and danger: Sociological contributions to the study of adverse events. Quality in Health Care. 9: 1120–1126.
West Michael A, et al.(2003). Leadership clarity and team innovation in health care. The Leadership Quarterly. 14.4:393–410.
Wiig S, Storm M, Aase K, Gjestsen M.T Solheim M, Harthug S, Robert G, & Fulop, N. Investigating the use of patient involvement and patient experience in quality improvement in Norway: Rhetoric or reality? Accepted for publication in BMC Health Services Research
List of Websites:
www.ucl.ac.uk/dahr/quaser
