Final Report Summary - ADVANCED-EVAL (Development and application of advanced quantitative methods to ex-ante and ex-post evaluations of rural development programmes in the EU) ADVANCED-EVAL project aimed to develop innovative quantitative methods to improve evaluation techniques applied in rural development programmes (RDPs) across the European Union. The project team applied interdisciplinary approaches to model rural development (RD). ADVANCED-EVAL provided a set of applicable tools for efficient policies evaluation and also resulted in important research findings, theory developments and policy-relevant knowledge acquisition. Evaluation of RDPs in terms of effectiveness and efficiency was estimated necessary by the policy makers in order to determine whether the provided support brought out the expected results. A key issue for this evaluation was the determination of a counterfactual state, which referred to the conditions that would be observed in case RD was not applied. Given that most up-to-date evaluations were based on qualitative approaches, the observed results were often non-representative of the actual conditions. Thus, the primary ADVANCED-EVAL objective was to further develop quantitative tools to enable a more accurate assessment of the microeconomic and macroeconomic effects of RDPs. The tools were subsequently applied and tested against other widespread methods. The research regarding microeconomic impacts examined, using specific policy indicators, the way various microeconomic and general equilibrium effects of RD could be calculated using advanced evaluation methodologies. Sensitivity tests were performed to analyse the stability of the obtained results and to define a minimal, yet optimal, set of conditional variables to be included in the propensity scores estimation. The developed methodologies were implemented and evaluated in various cases, including currently applied RDPs in different countries. Macroeconomic effects were also examined, in order to be able to calculate an overall net effect of the policy measure investigated. A multidimensional rural development index (RDI) was constructed as a proxy indicator describing the level of regional development and the quality of life in individual rural regions. The RDI was then applied, together with other relevant indicators, to estimate the net effect of a given RDP. The composite RDI embraced all important RD domains, such as economic output, investment, employment, poverty, education, health, housing conditions, environment, urbanisation and land use and aggregated them into a one-dimensional indicator using objective and statistically verifiable weights. The developed macroeconomic approach was then employed to evaluate effects in three cases: on regions which participated in a given programme, on an average region randomly selected which could either be or not be a participant and on the regions which did not participate in the programme. RDI use allowed for the analysis of various RD domains, at very regional levels, even at village scale. In addition, the selected approach enabled the technical separation of quality of life from migration. Social relations and networks of such relations were recognised, by both political and academic levels at an international scale, as having significant impacts on the behaviour of individual agents at the micro level and, accordingly, on aggregate outcome at the macro level. However, methodology and tools to analyse these relations were not fully developed. ADVANCED-EVAL aimed to develop and apply such approaches for quantitative network analyses in RD. The undertaken research focused on the impact of networks structure on knowledge spillovers and regional economic growth, on belief formation and economic performance and on firm's transaction costs. Two alternative statistical approaches were developed and empirically applied. The project clearly proved that social networks were a key factor determining RD. Ex ante evaluation of policies to be implemented was an additional tool that contributed to effective management planning, since it helped to avoid the expense of implementing ineffective programmes and provided an idea of what impacts to expect after RDPs completion. In addition, it allowed for an appraisal of the consistency of the invested resources with the anticipated results. ADVANCED-EVAL developed and applied adequate methodologies and tools. The selected approach allowed for simultaneous derivation of evaluation indicators at both macroeconomics and microeconomics level and its sequential structure allowed for temporal analysis of policy impacts. However, the model was time consuming and data intensive to calibrate, while the simulated results had to be interpreted with caution. Generalised matching methods were also utilised for ex ante evaluations, which though delivered less good simulations compared to similar techniques applied for ex post assessments. Nevertheless, if properly applied, these techniques could still serve as useful evaluation tools. The quantitative methods and techniques which were developed as part of the ADVANCED-EVAL project were compared to the qualitative approaches which were commonly used for RDPs evaluations, in order to assess and explain the differences. It appeared that quantitative approaches were advantageous tools which defined the magnitude and direction of effects; however, no method was infallible. The project clearly showed that the combination of qualitative and quantitative methods in a triangulate fashion was the optimal approach to ensure high quality of RDPs' impacts evaluations in the future. Given that all methods had advantages and disadvantages, a way forward could be to develop rankings for the alternative methods selection under specific circumstances so as to optimise the various analyses' benefits. Related documents Final Report - ADVANCED-EVAL (Development and application of advanced quantitative methods to ex-ante and ex-post evaluations of rural development programmes in the EU)
