Community Research and Development Information Service - CORDIS

FP7

IMPACT-EV Report Summary

Project ID: 613202
Funded under: FP7-SSH
Country: Spain

Periodic Report Summary 2 - IMPACT-EV (Evaluating the impact and outcomes of European SSH research)

Project Context and Objectives:
The importance of outcomes and impacts of SSH research has been highlighted in past and current editions of the European Framework Programme. Nonetheless, these impacts have been critically contested and lack of adequate tools for comprehensively assessing impact, as well as the perception of poor impact of EU funded SSH research have been an object of concern for the European Commission. Major European research funding organizations are discussing the impact of social research, beyond the scientific impact, while some sectors question the need for specific research in the social sciences.

IMPACT-EV responds to this scientific and socio-political need. The main objective is to develop a permanent system of selection, monitoring, evaluation and comparison of the impact and outcomes from European SSH research, taking into account the latest quantitative and qualitative evaluation tools, identifying new ways of implementing them and exploring new standards and indicators that complement existing impact assessment processes. Specifically, IMPACT-EV addresses the following sub-objectives:
1) To review the existing scientific knowledge concerning research evaluation tools and the changing structure of scientific, policy and social impact of SSH research as well as in relation to the development of the ERA.

2) To compare European SSH national research evaluation systems, as well as other relevant systems developed in leading countries and organizations in order to identify the systems which better contribute to the EU2020 and all EU main targets. This comparative analysis will focus on how these systems measure scientific impact and assess social and political impact.
3) To conduct an impact assessment of FP6 (last call) and FP7 SSH research in the three highlighted impact areas (scientific, policy and social impact) and to develop a map of SSH research impact in the EU, in which Success Stories are identified as well as those that have not achieved the expected success.
4) To analyse strategies that promote the achievement of high quality outcomes and impact in terms of the scientific production accomplished by SSH research as shown in the quality of publications, training of young researchers, forms of interdisciplinarity and transdisciplinarity and the constitution of European scientific excellence networks, among others.
5) To explore in which ways the impact of SSH research on European, national, regional and local policies can be assessed, and therefore upgraded.
6) To analyze the social impact of SSH research and the factors that have contributed to obtain or not this impact in order to create indicators to identify and evaluate the social impact of the SSH research ex-ante and ex-post. Especially relevant for this elaboration are the results obtained from the analysis of the Top Success Stories in social impact from SSH research.
7) To examine a) in which ways SSH research projects have contributed to develop the European Research Area in SSH and b) in which ways the various ERA initiatives at EU level have had an impact on the design and development of SSH research.
8) To integrate the knowledge produced in IMPACT-EV and develop a system of selecting, monitoring and evaluating SSH research which includes comprehensive indicators of scientific impact and the procedures to identify Success Stories on policy and social impact.
9) To explore and propose, based on previous experience, potential indicators of policy and social impact of SSH research projects, which can complement existing qualitative assessment methods. The focus will be on the projects which totally or partially address at least one of the EU main targets.
10) To define appropriate management procedures which secure the scientific and technical dimension of the project’s implementation.

11) To disseminate the project results and outputs among the scientific community, policymakers and stakeholders in order to increase its effectiveness and maximize its impacts.

Project Results:
The main results during the first and second reporting periods have been as follows:

- A mapping of what is already known about SSH research evaluation, by the means of an exhaustive review of international scientific literature on research evaluation procedures and the changing structure of scientific, social and political impacts of SSH research.

- A comparative analysis of SSH research systems. A total of 12 case studies of national research systems (9 EU countries and 3 international). For each case study, an in-depth documentary analysis was conducted, as well as interviews (total of 96) with relevant stakeholders.

- A contribution to the FP7 Ex-post evaluation with an extensive evaluation of the impacts and outcomes obtained from previous FP6 (last call) and FP7 SSH projects. The methodology followed for the evaluation focused on the exploitation and analysis of E-CORDA data, project reports from the SESAM database, other existing reports and evaluations, project websites, personal interviews with EC officers, more than 70 phone and/or Skype based interviews with Main Researchers or project team members, and an Online questionnaire.

- Identification of Success stories on achieving impact and other that have not achieved the expected success. 20 projects for each impact-type have been identified as having impact. They were scored according to specific criteria.

- A total of 23 case studies on SSH research projects: 14 top stories (5 for social impact and 3 for scientific, political and ERA impacts) and 8 which did not achieve expected success. This has allowed to better identify factors and strategies to increase the impact of projects.

- Launch of the Social Impact Open Repository (SIOR), unprecedented data source at international level in which researchers display, cite and store the social impact of their research results. SIOR is already connected to ORCID and it is highly instrumental to funders and research institutions. SIOR has projects from all disciplines (going beyond SSH) and worldwide regions.

- A pilot study for a system of indicators for scientific impact. This has involved an analysis of a large-scale sample of FP6 and FP7 projects, for which a large-scale database on FP6 and FP7 SSH projects (n=419) containing the necessary interlinked datasets of projects and their research outcomes was delivered. Different analytic strategies were used to test indicators on different impact dimensions, and to identify different “impact profiles” among projects.

- A set f 42 interviews to policy makers on the relationship between SSH research and policy-making, and a Delphi Panel with researchers, policy makers and endusers, focused on the agreements on appropriate indicators for political impact of research.

- An extensive expiration of the presence of SSH outcomes in social media and the relationship between open access publications and social impact. This analysis has also provided methodological approaches for identifying actual social impacts of research in the social media.

- Two Open Seminars with the Integrative panels gathering both experts, researchers and citizens discussing on the impacts of SSH.

- The first international conference on Social Impact of Sciences (SIS2016) was held in Barcelona, extending the debate promoted from SSH to all scientific fields. Three Nobel prizes and some of the most important funding agencies have participated in the Conference.

- Articles published in Peer-review journals, including one article in Nature.

- Synergies with relevant stakeholders, agencies, programmes and individual researchers. Among them, we highlight ORCID, as elected an IMPACT-EV member in the Board of Directors, and the involvement of IMPACT-EV researchers in the the launch of the European Alliance for Social Sciences and Humanities (EASH). The work conducted with agencies and researchers (during the evaluations of projects and the case studies) has resulted in new collaborations.

Potential Impact:
The project has evolved in parallel of, on the one hand, an increase of the concern for both the future of the social sciences and humanities’ research, and on the other hand, the move towards assessment of all the different impacts of research. In this context, IMPACT-EV goes beyond providing a system and indicators for the analysis of impact; actually this project is simultaneously transforming the way social sciences’ research are developed and presented to society.

The launch of the Social Impact Open Repository (SIOR) and te Social Impact of Sciences conference have advanced eon the awareness of the need for appropriate tools for viewing and valuing social improvements derived from SSH research projects, as well as the indicators and kind of evidences that are needed. With them, IMPACT-EV has also opened the floor for a public debate on research social impact, which has gone beyond SSH.

In the final stage, the development of IMPACT-EV research will:

- Define indicators for scientific, political, social impact as well as impact on promoting the European Research Area.

- Pilot the indicators and processes of evaluation for projects, in collaboration with national funding agencies.

- Held a Final Conference in Brussels, in which there is expected an involvement of experts, researchers, stakeholders and end-users.

As a whole, IMPACT-EV will take a key step further to not only improve the processes of SSH research evaluation in all the stages, but also to enhance the way SSH contribute to the improvements of our societies.

List of Websites:
www.impact-ev.eu

Reported by

UNIVERSITAT DE BARCELONA
Spain
Follow us on: RSS Facebook Twitter YouTube Managed by the EU Publications Office Top