Periodic Reporting for period 4 - FICKLEFORMS (Fickle Formulas. The Political Economy of Macroeconomic Measurement)
Période du rapport: 2020-03-01 au 2021-06-30
Macroeconomic indicators are integral to economic governance. Yet their air of objectivity notwithstanding, it is far from self-evident how for example growth or inflation should be defined and measured. At the same time, our measurement choices have deeply distributive consequences, producing winners and losers, and they will shape our future, for example when GDP figures hide the cost of environmental degradation. So why do we measure our economies the way we do?
FICKLEFORMS has tackled this question through six sub-projects. The first has systematically compared the evolution of indicators in central OECD countries. The second has analysed the timing and content of statistical harmonization efforts through the United Nations, the IMF and the World Bank. The third has constructed a new database of “measures of measures” to quantitatively explore data defects and their consequences for scholarly analyses. The final three sub-projects have reached beyond the OECD and studied the politics of macroeconomic measurement in China, Brazil and South Africa.
FICKLEFORMS has tackled this question through six sub-projects. The first has systematically compared the evolution of indicators in central OECD countries. The second has analysed the timing and content of statistical harmonization efforts through the United Nations, the IMF and the World Bank. The third has constructed a new database of “measures of measures” to quantitatively explore data defects and their consequences for scholarly analyses. The final three sub-projects have reached beyond the OECD and studied the politics of macroeconomic measurement in China, Brazil and South Africa.
The OECD cross-country comparison has been conducted by Daniel Mügge, the principal investigator. This subproject leveraged differences across the countries studied to understand better which factors have been decisive in shaping local measurement choices. For example, why do some countries include real estate prices in their inflation figures while others do not? Why do some jurisdictions count people as unemployed when they work less than 12 hours per week, while in other countries, one hour or more is enough for the “employed"-status? Like the other qualitative subprojects, this project has leveraged a mix of data sources: official reports, secondary – often rather technical – literature, and most importantly personal interviews, prominently including national statisticians in the countries studied, but also representatives of international organizations with which these have interacted, for example from the International Monetary Fund, the World Bank or the World Trade Organization, and civil society stakeholders.
PhD candidate Daniel DeRock’s research has focused on the work that international organizations such as the World Bank and the International Monetary Fund do to promote the dissemination of statistical standards around the world. These dissemination activities are not politically neutral. Often, DeRock has found, the standards in question are ill-fitting for developing countries. Hence, he has asked why the standards are promoted there nevertheless, who does so in particular, with what motivation, and what the response from the target countries is.
At the same time, we have dug into a number of specific country case studies. PhD candidate Joan van Heijster has ventured to China to understand better how this country, still rule by the Chinese Communist Party, has embraced an essentially capitalist economic measure since the late 1980s. PhD candidate Roberto Aragao has done the same for a whole range of economic measures in Brazil, post docs Juliette Alenda-Demoutiez and Francisca Grommé have investigated unemployment in South Africa and inflation measures in the Dutch Caribbean, respectively.
The quantitative subproject in FICKLEFORMS, focusing on trade data, has taken a completely different approach. Every international transaction is recorded twice – once by the sending country, and once by the receiving country. In theory, these two values should match. In practice, we find significant and persistent differences, indicating that such international economic data is a lot less reliable than the official figures suggest.
Using such mirror statistics, we have built an alternative dataset and replicated major academic studies. They show that academic insights are indeed sensitive to measurement error, even if to different degrees. But also after thorough analysis, it is not possible to specify the sources of data defects. I have therefore devised a range of strategies researchers should employ to improve the robustness of inferences they draw from large economic datasets.
PhD candidate Daniel DeRock’s research has focused on the work that international organizations such as the World Bank and the International Monetary Fund do to promote the dissemination of statistical standards around the world. These dissemination activities are not politically neutral. Often, DeRock has found, the standards in question are ill-fitting for developing countries. Hence, he has asked why the standards are promoted there nevertheless, who does so in particular, with what motivation, and what the response from the target countries is.
At the same time, we have dug into a number of specific country case studies. PhD candidate Joan van Heijster has ventured to China to understand better how this country, still rule by the Chinese Communist Party, has embraced an essentially capitalist economic measure since the late 1980s. PhD candidate Roberto Aragao has done the same for a whole range of economic measures in Brazil, post docs Juliette Alenda-Demoutiez and Francisca Grommé have investigated unemployment in South Africa and inflation measures in the Dutch Caribbean, respectively.
The quantitative subproject in FICKLEFORMS, focusing on trade data, has taken a completely different approach. Every international transaction is recorded twice – once by the sending country, and once by the receiving country. In theory, these two values should match. In practice, we find significant and persistent differences, indicating that such international economic data is a lot less reliable than the official figures suggest.
Using such mirror statistics, we have built an alternative dataset and replicated major academic studies. They show that academic insights are indeed sensitive to measurement error, even if to different degrees. But also after thorough analysis, it is not possible to specify the sources of data defects. I have therefore devised a range of strategies researchers should employ to improve the robustness of inferences they draw from large economic datasets.
Our main ambition has been to understand why we measure the economy the way we do, particularly in countries beyond the Global North.
Beyond the truly case-specific insights, a broader pattern emerges when it comes to the embrace of international standards outside the Global North. Consonant with the political baggage that macroeconomic measurement approaches always have, the motivations to embrace or resist international standards, too, are politically driven – and not just attempts to measure the economy “better”. For example, the embrace of GDP in China has been an important political symbol of China's 1980s turn to a more market-oriented economic organization. Statistics South Africa has embraced international unemployment measures not because they suit the country particularly well, but because such global best practice standards were necessary to insulate the organization from charges of political bias. In smaller countries in the Global South, standard adoption has been promoted by the material incentives international organizations offered – even when standard implementation was superficial rather than genuine. Our case studies show how not just the design and the consequences of macroeconomic measures are political, but also the way and degree to which they are embraced around the world.
More generally, many macroeconomic measures face serious and growing measurement problems in times of globalization and digitization. And they are frequently poorly aligned with societies’ normative priorities and people's lived realities, for example when they ignore environmental destruction or informal labour markets. The ensuing question has been: if macroeconomic statistics suffer such defects, why is so little done to mend them? We focused on both technical measurement problems (the "concept-measurement gaps") and lacking citizen identification with official figures (the "measurement-experience gap").
Initially, we considered a range of answers to this question, including the power and material interests of status-quo champions, and ignorance and tunnel vision among the statisticians setting the standards. Our research has found neither intuition to withstand scrutiny. Statistical standard setting is largely in the hands of experts. And these clearly understood, and often lamented, the limitations of present-day statistics.
Instead, we found that statistics’ need for measurement reliability is increasingly at odds with blurring economic boundaries – between countries, between work and non-work, employment and non-employment, for-pay and for-free services, and so on. The 21st century economy is harder to pin down in neat statistics than that of the preceding century.
At the same time, citizen expectations of statistics – how "hard” and reliable they expect statistics to be – has increased. That instils a conservative bias into measurement approaches. Statisticians are unwilling to sacrifice the reliability of their measures to increase validity.
In addition, to make figures comparable internationally, governments have agreed to follow similar measurement routines. Consensus on measurement reforms is often elusive, however, and updates to for example the System of National Accounts typically take decades. Inevitably, statistical practices lag socio-economic transformations.
Beyond the truly case-specific insights, a broader pattern emerges when it comes to the embrace of international standards outside the Global North. Consonant with the political baggage that macroeconomic measurement approaches always have, the motivations to embrace or resist international standards, too, are politically driven – and not just attempts to measure the economy “better”. For example, the embrace of GDP in China has been an important political symbol of China's 1980s turn to a more market-oriented economic organization. Statistics South Africa has embraced international unemployment measures not because they suit the country particularly well, but because such global best practice standards were necessary to insulate the organization from charges of political bias. In smaller countries in the Global South, standard adoption has been promoted by the material incentives international organizations offered – even when standard implementation was superficial rather than genuine. Our case studies show how not just the design and the consequences of macroeconomic measures are political, but also the way and degree to which they are embraced around the world.
More generally, many macroeconomic measures face serious and growing measurement problems in times of globalization and digitization. And they are frequently poorly aligned with societies’ normative priorities and people's lived realities, for example when they ignore environmental destruction or informal labour markets. The ensuing question has been: if macroeconomic statistics suffer such defects, why is so little done to mend them? We focused on both technical measurement problems (the "concept-measurement gaps") and lacking citizen identification with official figures (the "measurement-experience gap").
Initially, we considered a range of answers to this question, including the power and material interests of status-quo champions, and ignorance and tunnel vision among the statisticians setting the standards. Our research has found neither intuition to withstand scrutiny. Statistical standard setting is largely in the hands of experts. And these clearly understood, and often lamented, the limitations of present-day statistics.
Instead, we found that statistics’ need for measurement reliability is increasingly at odds with blurring economic boundaries – between countries, between work and non-work, employment and non-employment, for-pay and for-free services, and so on. The 21st century economy is harder to pin down in neat statistics than that of the preceding century.
At the same time, citizen expectations of statistics – how "hard” and reliable they expect statistics to be – has increased. That instils a conservative bias into measurement approaches. Statisticians are unwilling to sacrifice the reliability of their measures to increase validity.
In addition, to make figures comparable internationally, governments have agreed to follow similar measurement routines. Consensus on measurement reforms is often elusive, however, and updates to for example the System of National Accounts typically take decades. Inevitably, statistical practices lag socio-economic transformations.