Final Report Summary - LAB2DATA (Large-scale Agent-Based models to the DATA. Structural estimation for improved policy making.) The objective of LAB2DATA was to advance the state of the art in estimating agent-based (AB) models. AB modelling is a technique primarily concerned with the analysis of local interaction between heterogeneous agents endowed with limited information, in detailed-rich institutional environments. AB models are sometimes considered as a candidate to replace or at least complement dynamic stochastic general equilibrium (DSGE) as the standard tool for macroeconomic analysis. However, a common critique addressed towards AB models is that they often remain at a theoretical level, and lack a sound empirical grounding. When present, this is often limited to some ad-hoc calibration of the relevant parameters. Estimation however is crucial for the empirical validation of a model, for comparing a model with other available models, and for policy analysis. Also, estimation (as opposed to calibration) involves the attainment of clearly specified scientific standards in the way models are confronted with the data, and this sort of “certification” is highly needed to gain confidence and ultimately support from the wider scientific community and the policy circles in the use of a new tool. The advantage of AB modelling over more conventional techniques ultimately lies in the increased flexibility in model specification, but this flexibility comes at the price of making estimation more difficult. Paradoxically, AB models can provide a more realistic description of how real economies actually work, but this description resists fine tuning to real data. However, fine-tuning is important as small differences in the values of the parameters might be amplified by the nonlinear feedbacks in the system and result in large differences in the effects of the policies under examination. Moreover, AB models are often conceived to investigate what happens “out-of-equilibrium”, during adjustment phases: systemic disruptions, regime shifts, rare events are usual objects of analysis. This complicates their analysis and estimation. A first goal of the project was to investigate in what respect estimation of AB models poses new challenges. A new formal characterisation of AB models as recursive systems has been proposed, together with a new classification of model behaviour which distinguishes between ergodic and non-ergodic behaviour, both of which can display transient or absorbing statistical equilibria (i.e. stationary states). It has been shown that meaningful estimation can be performed only in those equilibria, and that consistency of the estimates can be obtained only in the ergodic case. By comparing the structure of AB models with that of DSGE models, it has been made clear that the main difficulty in estimating AB models stems from their being potentially more complex, from a computational perspective. Not only there are more agents (equations) to be simulated in AB models, but the lack of equilibrium constraints on individual behaviour (which typically originate, in DSGE models, from the Rational Expectations assumption) requires the simulation of all individual trajectories along the entire adjustment phase to the statistical equilibrium, if any. This means that their properties (for instance, their ergodic or non-ergodic behaviour) have to be understood and estimation has to be performed by means of computationally intensive simulations.A second goal of the project was to explore the use of Simulated Minimum Distance techniques for estimating AB models. Examples of application of Simulated Minimum Distance (SMD) techniques have been provided both for ergodic and non-ergodic models, and both in transient and in absorbing equilibria. It has been shown that the use of nonlinear summary statistics, as in the Method of Simulated Moments (a standard SMD technique), leads to a small sample bias, which can be to some extent controlled and reduced by appropriately transforming (i.e. linearising) those statistics. An interpretation of non-ergodicity as a source of additional uncertainty about the true value of the parameters has been advanced. This uncertainty can be quantified by applying the same SMD techniques as in the ergodic case, but to multiple instances of the model obtained using different seeds for the pseudo-random number generators. This produces distributions of estimates (one for each seed), and the amplitude of these distributions measures the uncertainty coming from the non-ergodic nature of the model. The cost is a further increase in computing time, given that the whole estimation process has to be repeated for each different seed employed in the simulations.A third goal of the project was to explore the use of Bayesian estimation methods. The advantages of Bayesian methods with respect to the frequentist and the calibration approaches are twofold: (i) they do not require to pre-select moments (as in the Method of Simulated Moments) or an auxiliary model (Indirect Inference), or other metrics to evaluate the distance between the real and the simulated time series (as in the calibration approach), and (ii) they allow to incorporate prior information, leading to a proper statistical treatment of the uncertainty of our knowledge, and how it is updated given the available observations. Moreover, with respect to methods that require the use of summary statistics (as MSM), the Bayesian approach fully exploits the informational content of the data, hence achieving, at least asymptotically, greater efficiency. The main disadvantage is increased computational costs. To save on these costs, in addition to using efficient sampling schemes such as Markov Chain Monte Carlo methods in models with large parameters’ space, several approximations can be introduced, whose appropriateness should be evaluated on a case-by-case basis. These approximations might also involve giving up (i), and resorting again to make inference based on the informational content of (generally insufficient) summary statistics, an appropriate choice of which can also result in more robust estimation. In particular, given the computational costs of estimating the likelihood function, for instance by kernel density estimation, alternative approaches such as normal approximation of the likelihood and Approximate Bayesian Computing (ABC) have been tested. These different Bayesian techniques have been first applied to simple AB models with a limited number of parameters, and then tested on a medium-scale AB macroeconomic model with 12 parameters (9 of which were estimated). Moreover, a state-of-the-art large-scale AB macroeconomic model, with individual workers, firms and banks interacting to generate endogenous business cycles and the possibility of systemic crises, has also been implemented, to be used for a follow-up project focused on the application of advanced estimation techniques from the machine learning literature (e.g. Bayesian Monte Carlo, Bayesian-Hermite Quadrature). The simulation platform used for implementing the macro-model has been explicitly developed within the Marie Curie fellowship. It is a general-purpose open source platform for discrete-event simulations, including both AB and microsimulation (MS) modelling. A dedicated web site has been set up (www.jas-mine.net) completed with software documentation, tutorials, sample models and a download page where all the simulation libraries and the source code can be accessed. The target of this dissemination activity is composed by undergraduate and postgraduate students, as well as practitioners in the field of AB and MS modelling (in 2016 alone, the simulation libraries have been downloaded more than 400 times). Prospectively, as new tools for interactive scenario creation and analysis will be integrated in the platform, the target will also include policy makers, policy analysists and the general public. Specific courses have been given on how to use the platform, as well as presentations at key microsimulation conferences.The platform has also been used to develop rich hybrid AB-MS models. Historically, AB and MS models have followed different trajectories, with AB models focusing more on theoretical issues and MS models being more data-oriented, often featuring processes modelled as probabilistic regressions. In general, AB models are structural models with a primary concern on understanding, while microsimulations are reduced-form models geared towards forecasting. As data becomes more readily available and technology becomes increasingly sophisticated at handling such data, there has been an inevitable trend of convergence between AB and MS modelling styles, with AB models evolving to be more empirical in nature and MS models integrating interactions and feedback effects. The JAS-mine platform offers tools used in both approaches, thus helping the process of convergence. A new release of the platform, scheduled at the end of the year, will feature the integration of Bayesian estimation libraries, with Markov Chain Monte Carlo sampling schemes.
