Periodic Reporting for period 3 - METRO (International Organisations and the Rise of a Global Metrological Field)
Période du rapport: 2021-05-01 au 2022-10-31
The project ‘International Organisations and the Rise of a Global Metrological Field’, or in short METRO, cast light on the processes of the making of quantification by large International Organisations. This is important, as it is only through understanding how experts and other actors make decisions and choices on what and how social issues are measured, that we can subsequently make sense of how certain answers to policy problems have been reached, as well as -to an extent- understand how governments set policy priorities.
Secondly, METRO examines what impact measurement practices have on the governing of International Organisations (IOs) themselves. Over the last 20 years, large IOs have become dominant in developing the technocratic knowledge to produce such metrics. What are the consequences of these IOs working closely together? How do they influence one another whilst maintaining their unique contribution and identity? METRO suggests that these questions are critical if we are to understand the role and impact of international organisations in the governing of public policy arenas, such as education, sustainability and health.
METRO was completed at the end of 2022. Some of the project’s major findings reveal that IOs are increasingly interdependent and that this has important implications on the ways that they work with one another and in the field. The project found that the values of participation and democracy have taken a central positioning in the governance of major transnational policy arenas and this has had implications for quantification and the balancing of the objectivity of numbers with their relevance and use in national contexts. The project produced significant findings in relation to reflexivity, future-making, economisation, the role of data visualisations, the production of indicator ‘markets’ and even the influence of ‘bad’ numbers (ie. provisional or placeholder numbers) in the making of global public policy.
METRO has made a significant contribution to the empirical and theoretical analysis of the role of quantification and IOs. Most notably, the project revealed and analysed International organisations’ measurement practices through the concept of ‘epistemic infrastructures’ – interconnected structures of socio-material data and diverse actors that together have had significant effects on the changing paradigm of governing public policy globally. The project’s influence and contributions are evidenced through its publications, as well as the extensive presentation of the project findings to a large number of conferences, seminars and workshops in Europe and beyond.
Secondly, METRO examines what impact measurement practices have on the governing of International Organisations (IOs) themselves. Over the last 20 years, large IOs have become dominant in developing the technocratic knowledge to produce such metrics. What are the consequences of these IOs working closely together? How do they influence one another whilst maintaining their unique contribution and identity? METRO suggests that these questions are critical if we are to understand the role and impact of international organisations in the governing of public policy arenas, such as education, sustainability and health.
METRO was completed at the end of 2022. Some of the project’s major findings reveal that IOs are increasingly interdependent and that this has important implications on the ways that they work with one another and in the field. The project found that the values of participation and democracy have taken a central positioning in the governance of major transnational policy arenas and this has had implications for quantification and the balancing of the objectivity of numbers with their relevance and use in national contexts. The project produced significant findings in relation to reflexivity, future-making, economisation, the role of data visualisations, the production of indicator ‘markets’ and even the influence of ‘bad’ numbers (ie. provisional or placeholder numbers) in the making of global public policy.
METRO has made a significant contribution to the empirical and theoretical analysis of the role of quantification and IOs. Most notably, the project revealed and analysed International organisations’ measurement practices through the concept of ‘epistemic infrastructures’ – interconnected structures of socio-material data and diverse actors that together have had significant effects on the changing paradigm of governing public policy globally. The project’s influence and contributions are evidenced through its publications, as well as the extensive presentation of the project findings to a large number of conferences, seminars and workshops in Europe and beyond.
The project began its work by conducting a comprehensive literature review , performed through a variety of disciplinary lenses such as sociology, science and technology studies and critical accountancy.
In parallel, one of the first steps of our work was to construct a full and detailed research design for each case study before commencing fieldwork. This work led to the conclusion that some adjustment was necessary, in order to yield insightful and original data. Therefore, although the cases on childhood poverty (case 2) and European education governance (case 3) remained unchanged, we decided to examine the production of the production of data for Sustainable Development Goal 4 (case 1). Secondly, although one of the four original cases was intending to examine the development of sustainable development indicators (case 4), we found that a focus on the issue of statistical capacity building within the SDG agenda would have to become our focal point of analysis.
During the first two years of the project, the team focused primarily on cases 1 and 2, and developed work on cases 3 and 4. In terms of childhood poverty, the team drew preliminary findings in relation to a. the production of numbers as a mechanism of coordination and consensus-making; b. the use of numbers as action and rhetorical frames; c. the use of numbers in the making of new policy instruments; d. the different ‘cultures of objectivity’ and different approaches to navigating quantification, depending on the organisation; and last but not least, e. the use of numbers as an advocacy tool, used to close controversies and harmonise complexity and diversity.
In terms of case study 1, on the SDG4, some of the project findings showed that a. metrics are produced by primarily experts with a disciplinary background in economics; b. numbers often mask controversies amongst actors (experts but also national representatives) over the usefulness, robustness and relevance of indicators chosen; c. there is a large amount of emphasis on statistical capacity building, the construction of a discourse around data as a public good and the democratisation of data; d. International Organisations’ most important function is not to produce new data but to construct consensus, mobilise, persuade and take countries on board; and finally e. the role of the data visualisation as a medium of communication, of story-telling and of the construction of numbers as ‘world-making’ for all users rather than just elite actors.
Case study 3 analysed how quantification, and specifically processes of data collection and measurement practices, have shaped the field of higher education in Europe. We found that there has been significant expansion and dominance of quality assurance processes and agencies, whose efforts focus primarily in collecting evidence, establish protocols of quality and thus have significant effects on the regulation and governance of the higher education policy space in Europe. Our analysis of this expanding and ever-evolving large field of actors – with some more and some less established ones- coincided with the introduction of new policy agendas in the field of European education and most notably, the introduction of the European Education Area. In parallel to these agendas, we see increasing focus on the collection of data on higher education across Europe and renewed efforts to strengthen their influence in increasing mobility.
Lastly, case study 4 focused on statistical capacity development and the work of key actors within the SDG space to establish improved statistical practices in countries of the Global South. The case examined the history and current developments of the movement for strengthening statistical capacity, and identified tensions between donor demands for development data and the nationally-owned approach that many declarations since the late 1990s called for.
In parallel, one of the first steps of our work was to construct a full and detailed research design for each case study before commencing fieldwork. This work led to the conclusion that some adjustment was necessary, in order to yield insightful and original data. Therefore, although the cases on childhood poverty (case 2) and European education governance (case 3) remained unchanged, we decided to examine the production of the production of data for Sustainable Development Goal 4 (case 1). Secondly, although one of the four original cases was intending to examine the development of sustainable development indicators (case 4), we found that a focus on the issue of statistical capacity building within the SDG agenda would have to become our focal point of analysis.
During the first two years of the project, the team focused primarily on cases 1 and 2, and developed work on cases 3 and 4. In terms of childhood poverty, the team drew preliminary findings in relation to a. the production of numbers as a mechanism of coordination and consensus-making; b. the use of numbers as action and rhetorical frames; c. the use of numbers in the making of new policy instruments; d. the different ‘cultures of objectivity’ and different approaches to navigating quantification, depending on the organisation; and last but not least, e. the use of numbers as an advocacy tool, used to close controversies and harmonise complexity and diversity.
In terms of case study 1, on the SDG4, some of the project findings showed that a. metrics are produced by primarily experts with a disciplinary background in economics; b. numbers often mask controversies amongst actors (experts but also national representatives) over the usefulness, robustness and relevance of indicators chosen; c. there is a large amount of emphasis on statistical capacity building, the construction of a discourse around data as a public good and the democratisation of data; d. International Organisations’ most important function is not to produce new data but to construct consensus, mobilise, persuade and take countries on board; and finally e. the role of the data visualisation as a medium of communication, of story-telling and of the construction of numbers as ‘world-making’ for all users rather than just elite actors.
Case study 3 analysed how quantification, and specifically processes of data collection and measurement practices, have shaped the field of higher education in Europe. We found that there has been significant expansion and dominance of quality assurance processes and agencies, whose efforts focus primarily in collecting evidence, establish protocols of quality and thus have significant effects on the regulation and governance of the higher education policy space in Europe. Our analysis of this expanding and ever-evolving large field of actors – with some more and some less established ones- coincided with the introduction of new policy agendas in the field of European education and most notably, the introduction of the European Education Area. In parallel to these agendas, we see increasing focus on the collection of data on higher education across Europe and renewed efforts to strengthen their influence in increasing mobility.
Lastly, case study 4 focused on statistical capacity development and the work of key actors within the SDG space to establish improved statistical practices in countries of the Global South. The case examined the history and current developments of the movement for strengthening statistical capacity, and identified tensions between donor demands for development data and the nationally-owned approach that many declarations since the late 1990s called for.
METRO produced exciting data on the production of quantification for global governance. Building on Gibbons et al’s (1994) and Nowotny et al.’s (2001) influential work on the shift from the production of hierarchical and disciplinary knowledge to a more horizontal, distributed and applied one, METRO identified further changes to the sociology of knowledge as a result of global quantification. This is what METRO calls ‘Mode 3’ knowledge; it relates to the ideas of advocacy, consensus-building, and the democratisation of data. This has been a bold theoretical proposition, but worthy of analytical articulation and peer-review given the clear new directions that our findings point towards. The project produced new theoretical analyses in relation to actor-hood; the affordances of technology; data visualisations; epistemic infrastructures; expert reflexivity; and quantification as utopia-making.