The importance of evidence-based policy making is widely recognized amongst policy makers worldwide. A vast amount of empirical work in economics aims to achieve this goal, but the current literature of econometrics provides rather limited recommendations for estimating a welfare maximizing policy. The main goal of this project is to develop statistically optimal and practically attractive ways of learning welfare maximising policies from the data, which can inform the design of public policies and decision making by policy makers.
The project made contributions to the following three aspects of evidence-based policy design:
(1) Estimation of causal effects and optimal policies: what estimation procedure should be used to estimate optimal policies?
We proposed empirical welfare maximization (EWM) methods for estimating an optimal targeting policy that prescribes who should get what policy. We clarified that the EWM-based optimal targeting policy shows its statistical optimality in terms of welfare performances of the estimated policy. The framework allowed a large class of social welfare criterion, and indicated how to make use of machine learning algorithms for the design of treatment allocation policies. We also developed inference methods for the welfare attained by an estimated policy. The main contributions appear in the following papers:
- "Who Should Be Treated? Empirical Welfare Maximization Methods for Treatment Choice," with Aleksey Tetenov, (2018), Econometrica
- "Equality-minded Treatment Choice," with Aleksey Tetenov, (2021), Journal of Business and Economic Statistics
- "Inference on Winners," with Isaiah Andrews and Adam McCloskey, (2024). Quarterly Journal of Economics
(2) Decision making: what is the optimal decision-making procedure given uncertainty about the welfare impact of the policies?
We developed a novel framework for binary policy choice based on nonlinear welfare regret criterion, and showed that Bayes and minimax optimal policy choices are randomized decisions. We also studied treatment choice in a meta-analysis context, and clarifies a set of conditions under which computation of the minimax regret decision can be simplified.
For general cases where the data and assumptions cannot pin down a unique point estimate for causal impact of policy (set-identified causal effect), we studied inference for it from the robust Bayes viewpoint. This approach can mitigate a common concern of prior dependence in Bayesian inference and at the same time, we showed it achieves desired frequentist performances asymptotically. The main contributions appear in the following papers.
- "Evidence Aggregation for Treatment Choice," with Takuya Ishihara, (2021), arXiv.
- "Treatment Choice with Nonlinear Regret," with Sokbae Lee and Chen Qiu (2023), arXiv
- "Robust Bayesian Inference for Set-Identified Models," with Raffaella Giacomini, (2021), Econometrica
(3) Applications to reality: Can the developed methods apply to the real-world?
To apply the methods developed in this project, we focused on targeted rebate policy in electricity market.
We ran a field experiment in Japan to learn the heterogeneous household demand response of electricity consumption to electricity pricing. Using this experimental data, we applied the methods developed in (1) to estimate optimal targeting of rebate program of electricity conservation, and showed that the targeted assignments of the rebates can significantly reduce the dead-weight loss in electricity market. The completed manuscript is "Choosing Who Chooses: Selection-Driven Targeting in Energy Rebate Programs" by Ida, Ishihara, Ito, Kitagawa, Sakaguchi, and Sasaki (2023, NBER working paper), which is currently revise and resubmit at Econometrica.
