Periodic Reporting for period 1 - MLending (An ML Approach to IFIs: Determinants of IMF Lending)
Okres sprawozdawczy: 2020-12-01 do 2022-11-30
This MSCA IF project “An ML Approach to IFIs: Determinants of IMF Lending” focuses on a controversial aspect of the International Monetary Fund’s (IMF) primary mission, i.e. lending to its members in need of finance. The research concentrated on the following questions: What factors influence the terms of an IMF program? And how do those factors play into shaping the design of the programs? In order to answer these questions, this research employed mixed methods including machine learning (ML), natural language processing (NLP), statistical analysis and process tracing. According to the project findings, the IMF’s Executive Board (EB) sentiment has a strong impact on the number of conditions that the IMF attaches to its loans. A significant relation was particularly observed in the external and financial sector. Likewise, GDP growth has a significant negative relation with the number of conditions in the external and financial sector. GDP growth also has a strong positive impact on the loan amount, implying that when a borrowing country is in a better economic phase, it can secure a larger loan. Although political fragmentation does not have a significant relation with loan size, it is found to have a positive correlation with the total number of conditions when interacted with EB sentiment. The analysis of the EB meeting minutes provides evidence for antinomic delegation by the EB, which is a path-dependent causal mechanism observed to influence the negotiations of a program with insistence on stricter conditionality by the IMF staff acting precautious in response to criticisms by the EB regarding the design of a previous program. The main findings of the analysis on the case studies, namely Romania and Greece, point towards the influence of economic bureaucracy on all sides of the negotiations on the program design. The overarching objective of this project is to create a comprehensive novel methodology and framework for understanding IMF program design, shedding light on the processes leading to variation in IMF lending. Through creating this framework, this project provides an indispensable and extensible tool for international political economy researchers, policymakers, and IMF staff to model the program design and implementation process with high predictive power of the outcomes.
The interdisciplinary nature of this project required it to be broken down into 6 work packages. WP1 entailed improvement of the theory and hypotheses to be tested, which guided the rest of the WPs. The goal of WP2 was data collection. Through web scraping of the IMF’s online archives, close to 40 thousand pages of EB meeting minutes were gathered. For the statistical analyses involving all IMF programs implemented between 1978-2014, data for numerous independent variables were collected. WP2 also involved data collection for case studies: Secondary data, through archival research of Romanian and Greek daily newspapers as well as newspapers and magazines with a special emphasis on business and economic news, and the IMF’s Independent Evaluation Office reports, were collected. The purpose of WP3 was NLP tool development. In this work package, an NLP tool for automatic extraction of EB views on IMF program design was developed, which is able to process the EB meeting minutes data collected in WP2. The output of the tool is the values of the specific EB-related variable defined in the theoretical model for the different IMF programs analyzed. The NLP tool was designed and implemented specific to the EB meeting minute domain. The model trained with the IMF dataset achieved above 95% accuracy, while the model that improves a popular publicly available BERT model without using any domain data achieved above 87% accuracy, which achieved the promise of the project. The tool was then applied to a different dataset to be able to test its power, which got published in the journal Sustainability. WP4 focused on statistical analyses. The degree of influence of EB’s sentiment on IMF loan amount and number of conditions was analyzed. The results indicate that the EB’s positive sentiments decreases the number of conditions, particularly in the external and financial sector. The intention of WP5 was ML model selection and development. In this work package, ML algorithms to most accurately model the IMF program design process based on the data collected in WP2 were selected. The selected models were implemented in software using an ML framework, and the accuracy of the models were evaluated with held-out test data for a subset of the IMF programs. The objective of WP6 was the analysis of all models and results. In order to compare the results of the regression models with those of the ML models created for estimating (a) the loan size (b) the total number of conditions (c) the number of condition categories, the estimates for each data point based on the obtained regression equations were categorized using the same cutoff values utilized in the ML models. Using a binary categorization scheme, it was observed that the regression models achieved around 72% accuracy in detecting the loan size, 20% accuracy in detecting the total number of conditions, and 55% accuracy in detecting the number of condition categories. The highest accuracies achieved by the developed ML models using the same independent variables are as follows: 85% for loan size, 60% for total number of conditionalities, and 63% for the number of conditionality categories. These results support the hypothesis of the research that it is possible to achieve significantly higher accuracy in program design estimation by utilizing state-of-the-art ML algorithms as compared to statistical regression models. A comprehensive report on all findings of the research, demonstrating the complementary aspects of the methods employed, has been accepted for presentation at the APSA Annual Meeting 2023.
This is the first study on IMF lending that jointly considers the impact of all actors & factors determining the terms of an IMF program. The employment of ML & NLP techniques in the research are innovative aspects of the project, which have created a new avenue for IMF and International Organizations studies, relieving researchers from the burden of manual analysis and enabling informed decision making for policymakers with the ability to grasp the full picture of program design. The project has also made a significant contribution to computational social sciences (CSS) through developing techniques not explored in the context of IMF studies before, which do the groundwork for future advanced studies.
With the start of the project, the MA-Computational Social Science Lab (MA-CSSL) was established at Koc University (KU), with the purpose of conducting cutting-edge research on the applications of computational methods to social science questions. MA-CSSL has had a growing number of members from multiple disciplines, including political science, economics, and computer science among others, with 2 postdocs, 3 practitioners, 5 PhD students, 6 MA students and over 80 undergraduate students from various institutions. Both economics and computer science are men-dominated fields; MA-CSSL has set gender equality as a priority and accordingly ensured to engage women students in its projects and encourage involvement of women in the field. More than half of the researchers at MA-CSSL are women and they come from diverse disciplines.
With the start of the project, the MA-Computational Social Science Lab (MA-CSSL) was established at Koc University (KU), with the purpose of conducting cutting-edge research on the applications of computational methods to social science questions. MA-CSSL has had a growing number of members from multiple disciplines, including political science, economics, and computer science among others, with 2 postdocs, 3 practitioners, 5 PhD students, 6 MA students and over 80 undergraduate students from various institutions. Both economics and computer science are men-dominated fields; MA-CSSL has set gender equality as a priority and accordingly ensured to engage women students in its projects and encourage involvement of women in the field. More than half of the researchers at MA-CSSL are women and they come from diverse disciplines.