Periodic Reporting for period 4 - REFORMED (Reforming Schools Globally: A Multi-Scalar Analysis of Autonomy and Accountability Policies in the Education Sector) Reporting period: 2021-01-01 to 2022-06-30 Summary of the context and overall objectives of the project Since the turn of the century, educational reforms place a much larger emphasis on performance goals, outcomes and metrics than ever before. To address competitive pressures, governments increasingly resort to data-intensive policy instruments, standards and systems of incentives in the governance of education. This new governance logic, which we call school autonomy with accountability (SAWA), has become central in defining the goals and rationales of contemporary educational reform. Within the SAWA model, schools are conceived as autonomous managerial units that enter into contractual relationships with public authorities and are encouraged to engage in continuous improvement cycles. This governance approach is politically convenient since it allows governments to steer schools at a distance and places greater responsibility for system quality at the school level. Nonetheless, the dominance of SAWA policies in education reform processes is striking for two main reasons. First, counter-intuitively, countries with very different administrative traditions, reform capacity and educational institutions are embracing SAWA policies to enhance system performance; and secondly, countries are adopting SAWA policies despite there is only weak evidence on their benefits on learning outcomes and increasing evidence on their side effects, such as curriculum narrowing, excessively test-oriented teaching, and student tracking and discrimination. Against this background, the REFORMED project scrutinises how and why SAWA policies are being adopted and re-formulated by policy actors operating at multiple scales, and examines the institutional frameworks and enactment processes behind SAWA’s differential effects in different school contexts. Specifically, the main project objectives are to: O1. Analyse the international spread of SAWA reforms.O2. Understand how institutional and political factors intervene in the selection and retention of SAWA policies in different countries. O3. Analyse how teachers and principals engage with SAWA policies, from a comparative perspective.O4. Analyse the impact of SAWA policies in educational and organisational practices in different school contexts. O5. Explore how the institutional design of SAWA has the potential to activate/inhibit a series of side effects. Work performed from the beginning of the project to the end of the period covered by the report and main results achieved so far The work has been organised in two main research strands (RS). In RS1, which focuses on the global dissemination of SAWA (O1 and O2), we have conducted international policy convergence analysis and identified SAWA reform paths by conducting a systematic literature review and interviews with key informants in Norway, The Netherlands, Italy, Spain and Chile. We have also constructed an original dataset on the adoption and characteristics of national large-scales assessments. Our findings challenge the idea that the global spread of SAWA instruments is making educational systems evolve more alike. Even when similar school governance discourses and instruments are being adopted internationally, the rationales behind the selection of these instruments, their final configuration and uses, and the administrative capacity to implement and retain them are manifestly different. In the context of RS2, which focuses on SAWA effects in schools (O3-O5), we have designed a novel survey instrument to measure how teachers and school principals perceive, interpret and respond to the SAWA mandate. The survey includes so-called survey experiments, which allow to capture under what conditions side effects emerge in SAWA environments, and overcome endogeneity and social desirability biases in data gathering. Survey findings have been complemented with interviews with school actors in a representative sample of schools. Our findings show that schools respond to SAWA regulations differently, and that the presence of undesired effects, decoupling and dilution strategies is more persistent in schools serving socially disadvantaged populations.If not implemented with sufficient support, SAWA policies trigger differentiated practices that contribute to reinforce school segmentation—creating very disparate and context-sensitive educational experiences and practices, and ultimately exacerbating the social stigmatisation of disadvantaged schools. The two described strands are intimately interconnected. Reformed findings emphasise the interconnectedness between institutions, politics and policy in school governance reforms. We show that the 'epistemic fit' between new data-intensive policy instruments, on the one hand, and previous institutions on the other, is key to understanding how school actors receive and enact new policy mandates. Overall, country- and school-specific processes of policy negotiation and contestation result in idiosyncratic practices and outcomes. Progress beyond the state of the art and expected potential impact (including the socio-economic impact and the wider societal implications of the project so far) In both research and policy circles, high-stakes accountability is increasingly perceived as a generator of side effects. In contrast, low-stakes accountability is conceived of as a source of formative assessment, educational improvement and, accordingly, as a more desirable policy option. Nonetheless, our research shows that the reactivity of school actors to accountability does not only depend on the formal consequences of external evaluations, but on the very fact of school actors being externally surveilled. Through a survey experiment, we have been able to analyse the effects of different types of incentives (individual or collective, symbolic or material) on the likelihood of teachers trying to inflate assessment results. Counterintuitively, we show that the symbolic consequences of accountability predispose teachers to over-react, even more frequently than material consequences. Teachers’ professional reputation concerns, and the fact that national assessments, independently of the stakes, work as a form of commensuration are some of the social mechanisms explaining this counterintuitive finding. A second survey experiment has captured teachers’ preferences regarding school governance policy in three different countries. The experiment shows that, in all country contexts, material incentives, in the form of individual salary bonus, are not a main driver of teachers’ motivation, and that teachers prefer their work to be assessed through methods of a qualitative and procedural nature (such as classroom observations), as well as work environments with clear objectives and cooperative relationships. Our research findings invite policy-makers to rethink accountability policy, and to give greater consideration to the locus and space of accountability. New forms of network and incumbent accountability, grounded in local education spaces that transcend individual schools, could contribute to generate organisational improvement, quality assurance and policy learning dynamics among schools, without necessarily generating competition and segmentation within educational systems. Finally, the REFORMED Survey delivers large-scale datasets that can be used by scholars to explore and analyse the enactment of school autonomy and accountability policies in different contexts, by taking into account context-, school- and individual-level relevant variables that have been overlooked in previous research. The conception and development of the REFORMED questionnaires, as well as the sampling strategy, have involved a number of steps aiming at comparability across cases and at obtaining data of a very high standard. project logo
