Skip to main content

New Methods and Applications for Forecast Evaluation

Periodic Report Summary 2 - FORECASTING (New Methods and Applications for Forecast Evaluation)

Forecasting is a fundamental tool in Economics, Statistics, Business and other sciences. Judging whether forecasts are good and robust is of great importance since forecasts are used every day to guide policymakers' and practitioners' decisions. With the help of the grant, Barbara Rossi is developing several new methodologies to improve forecast evaluation and estimation. The new methodologies resolve several important issues that researchers encounter in practice.

A first issue is how to assess whether forecasts are optimal in the presence of instabilities. Optimality is an important property of models’ forecasts: if forecasts are not optimal, then the forecasting model can be improved. Existing methods to assess forecast optimality are not robust to the presence of instabilities, which are widespread in the data. Barbara Rossi developed a new methodology to test forecast optimality that is robust to the presence of instabilities, published in: B. Rossi and T. Sekhposyan (2016), “Forecast Rationality Tests in the Presence of Instabilities, With Applications to Federal Reserve and Survey Forecasts,” Journal of Applied Econometrics 31(3), pp. 507-532.

A second problem faced by forecasters in practice is to evaluate density forecasts. Density forecasts are important tools for policymakers since they quantify uncertainty around forecasts. However, existing methodologies focus on a null hypothesis that is not necessarily the one of interest to the forecaster. Barbara Rossi developed tests to evaluate forecast densities that address forecasters’ needs in: B. Rossi and T. Sekhposyan (2016). Alternative Tests for Correct Specification of Conditional Forecast Densities. Barcelona GSE Working Paper No. 758).

Another important question is how researchers can improve models that do not forecast well. In her work (published as: M. Carrasco and B. Rossi (2016), “In-sample Inference and Forecasting in Misspecified Factor Models”, Journal of Business and Economic Statistics 3, pp. 331-338), Barbara Rossi designs methodologies to improve the forecasting performance of possibly misspecified factor models. The methodologies exploit information from a large number of predictors and are robust to misspecification due to the absence of a factor structure. The robustness is obtained by developing data-driven selection methodologies based on a new cross-validation procedure. Furthermore, model misspecification is widespread, still economists are often left wondering exactly which parts of their models are misspecified. Barbara Rossi developed an empirical framework that estimates time-varying margins to assess where misspecification is located, and how important it is (C.H. Kuo A. Inoue and B. Rossi (2017), “Identifying the Sources of Model Mis-specification,” BGSE Working Paper).