Community programme (EEC) in the field of strategic analysis, forecasting and evaluation in matters of research and technology (MONITOR) - Activities in support of the evaluation of R&D programmes (SPEAR) -, 1989-1993

Part of the second Framework programme for research and technological development (1987-1991) under subactivity 8.3: "Forecasting and assessment and other back-up measures, including statistics", SPEAR is one of the three components of the MONITOR programme, along with SAST (Strategic Analyses in Science and Technology) and FAST III (Forecasting and Assessment in Science and Technology). It continues and sustains the activities which have been carried out by the Commission in the area of R&D evaluation since 1979.

In particular the programme is called upon to:
- Improve methods for evaluations of Community R&D programmes within the context of, and drawing as appropriate on, the experience of related national and international R&D programmes;
- Stimulate research into the methodology of evaluation and its use in Member States;
- Develop quantitative indicators that can describe the quality and utility of research and its contribution to the Community's economic and social development;
- Prepare guidelines (no later than June 1993) for the carrying out of Community R&D programme evaluations in light of the European experience;
- Prepare guidelines for evaluating the quality of management using a set of related criteria (utilization of appropriations, allocation of contracts, meeting of deadlines, etc.).

With regards to methodology, one of the most important studies undertaken by SPEAR is the development of an economic model and evaluation technique to allow the Commission to determine the economic impact of R&D programmes. A second area of study involves the determination of the industrial relevance of R&D programmes.
To improve the quality, utility and methodological bases of the Commission's independent external evaluations of R&D, to define an evaluation procedure applicable to the broad range of Community R&D activities, and to analyse and improve the effectiveness and impact of these activities.
Two main activities:

- Preparation, in conformity with the Community plan of action relating to the evaluation of Community research and development activities for the years 1987-1991, of four to five horizontal evaluations, within the context of, and drawing as appropriate on, the experience of national or international research programmes, in order to analyse their impact and identify means to improve their effectiveness at Community level. This covers analyses of methods of support and of management of national and Community research;

- Research actions into methodologies of evaluation of R&D programmes in order to increase their utility and make them more credible to their users.
The Commission, assisted by an Advisory Committee composed of the representatives of the Member States and chaired by the representative of the Commission, is responsible for implementing the programme.

SPEAR activities are undertaken or managed by the Commission's evaluation staff who also work as executive secretaries for the independent external evaluations of specific research programmes ("vertical" evaluations). This dual role ensures that the results of the work of SPEAR will be used immediately to help these individual evaluations, as appropriate. It also enables the work to be tailored to the continually-evolving needs of the programme evaluations.

The activities are implemented through the involvement of research centres or research teams from Community countries specializing in the evaluation of R&D programmes, and by organizing workshops, seminars, etc. In addition, provision is made for the secondment of visiting scientists from national institutions and governments to take part in various activities and for the dissemination of knowledge and results obtained through research publications, policy notes, the organization of national "MONITOR days" and other publicity activities.

SPEAR activities are also implemented by means of contracts for studies and services to be carried out on behalf of the Commission. As a general rule, where appropriate, contracts are awarded on the basis of calls for proposals or for tenders (restricted or public) published in the Official Journal of the European Communities. The Community's financial contribution may cover up to 100% of the expenditure involved. The contracts entered into by the Commission regulate the rights and obligations of each party, including the methods of disseminating, protecting and exploiting the research results.

The activities are carried out in liaison with a "12 + 1" network consisting of the Commission services, the competent units in the field within each Member State, as well as experts. The objective of this network is the exchange of information on the work currently underway in Europe relevant to evaluation, the improvement of evaluation methodology, and the systematic diffusion of SPEAR's results.

The Commission is authorized to negotiate agreements with non-Member States and international organizations, with a view to associating them wholly or partly with the programme. This applies in particular to those countries participating in European cooperation in the field of scientific and technical research (COST) and to those which have concluded framework agreements for scientific and technical cooperation with the Community. Where such framework agreements exist, organizations and enterprises established in those countries may, on the basis of mutual advantage, become partners in a project undertaken within the programme.

During the third year of implementation, the Commission is required to review the programme and to submit a report to the European Parliament and the Council. It must also submit to these same bodies a final evaluation of the results achieved when the programme is completed.