Auditors question Commission's evaluation of research funding programmes
Auditors have taken the European Commission to task over how it has evaluated its technology research funding for the past 12 years. The European Court of Auditors questioned how the Commission has assessed the success of European research and technological development (RTD) funding programmes since 1995. Currently, Commission funding for technology research is running at €7.2 billion per year, compared with €2.7 billion per year under the Fourth Framework Programme (FP4). The auditors would have liked to have seen more explicit underlying intervention logic, or description of causal links, between framework programme activities and expected outcomes. 'Poorly defined programme objectives and weak performance measurement undermined effective monitoring and evaluation,' said the Court of Auditors in a statement. 'The absence of a comprehensive evaluation strategy, agreed between the Research Directorates-General implementing the framework programmes, resulted in inconsistent approaches between the different Commission services,' states the report. The Court of Auditors, a panel of financial experts that examines the management and overall efficiency of the EU's budget and bureaucracy, also drew attention to the Commission's plans for assessing its research programmes during the current phase of funding, which will last until 2013. Not many changes have been made. 'Evaluation of the framework programmes was decentralised, the existing coordination mechanisms among the Directorates-General implementing the framework programmes were not effective, and the Commission's central services, in particular [the] Directorate-General [that controls] budget, had no enforcement role,' said the auditors. 'Inadequate methodological guidance was provided, evaluators found difficulties in gathering relevant data, and there were no evaluation studies that addressed the longer term outcomes and impacts of the framework programmes.' As a result of these weaknesses, the Commission evaluations of the schemes were of 'limited use to policy-makers', say the auditors. They found that ultimately, not enough was known about the achievement of programme objectives and the results of the last three framework programmes. The Court's report went on to recommend that framework programme intervention logic should be made explicit through the legislation underlying Commission technology funding. 'Underlying assumptions should be explained, the link between scientific and socio-economic objectives clarified and appropriate performance indicators developed,' say the auditors. They also call for the development of a comprehensive evaluation strategy for the Seventh Framework Programme (FP7).