EURAB proposes broad approach to FP6 impact assessment
Two alternative ways in which to assess the success of European research policy and its accompanying framework programmes were proposed by the European Research Advisory Board (EURAB) at its final meeting. The alternatives involve an assessment of goal attainment, and assessing the evidence base for research policy-making. A working group within the European Commission's Research DG is currently working on a strategy and procedure for the performance of scientific and technological project reviews and ex post impact assessments. Within this context, EURAB was asked to provide recommendations on ex-post impact assessments. The imminent end of EURAB's current term led to the Board deciding to publish a brief opinion now, and to recommend that the topic of impact assessments be considered afresh early in the lifecycle of the new EURAB. EURAB's paper begins by recognising the 'social and political pressures for accountability and value for money in the field of research spending', and then welcomes increased efforts to satisfy these demands. The paper therefore recommends an increase in the funds allocated to ex-post assessment, but cautions against channelling this money into the assessment of individual research projects. Instead, 'Given the intrinsic uncertainties associated with the practice of scientific research and the recognised difficulties associated with the attribution of long-term socio-economic impacts to particular research projects, EURAB suggests two alternative emphases, both worthy of increased support.' The first is an increased emphasis on the assessment of goal attainment and efficiency of implementation at project and programme levels. Within such an emphasis, particular attention should be paid to the diversity of projects funded by the EU's framework programmes, and their diverging goals. Different evaluation approaches and techniques would need to be implemented as appropriate. EURAB states that: 'customised studies focusing on goal attainment and efficiency of implementation would yield valuable lessons for the future conduct of the framework programmes and would have a greater impact on overall programme and project performance than futile efforts to link individual projects with long-term socio-economic impacts.' EURAB's second proposal is a move away from the evaluation of projects and programmes towards an appraisal of the processes leading up to the setting of research policy and spending. EURAB would welcome, for example, more information on the evidence base that led to the establishment of the research investment target at 3% of GDP, as set in 2002. The Board concedes that such an exercise would fall outside the spheres of programme evaluation and ex-post evaluation, but emphasises that: 'Any increase in funds for impact assessment would be better spent on efforts to improve the evidence base for research policy-making in general rather than on more limited ex-post project impact assessments.'