Final Report Summary - MERCI (Starting Grants – Monitoring European Research Council’s Implementation of Excellence)
Executive Summary:
In February 2009, the European Research Council (ERC) launched the MERCI project (“Monitoring European Research Council’s Implementation of Excellence”) to monitor its “Starting Grants” (StG) funding scheme. The StG programme addresses “excellent” postdocs with up to 1.5 million Euro for the duration of up to five years to set up or consolidate their own research group in order to pursue independent, investigator-driven “frontier research”. Due to the fact that the StG programme has only been established in 2007, the MERCI project puts a strong emphasis on the programme’s implementation and how it works in practice. Thereby MERCI focuses on the individual perspective and aims to draw a broad picture concerning direct and indirect effects of the StG funding scheme. MERCI does not only evaluate the programme’s objectives, but questions the objectives themselves by assessing if the tailoring of the StG funding scheme is suitable to meet the specific needs of postdocs. The project was implemented as an ERC Coordination and Support Actions (CSA) project and had been completed in 2014. It was carried out by a consortium of four partners: the Institute for Research Information and Quality Assurance (iFQ), TU Dortmund University, Bielefeld University and Humboldt University of Berlin.
MERCI delivered its results by drawing upon large empirical evidence based on three StG cohorts and more than 1000 ERC applicants. Results shed light on the applicants’ research funding strategies and their motivation to apply to ERC in comparison with other existing research funding opportunities. They depict the role of the ERC grant in the overall research funding portfolio of ERC applicants. They investigate if ERC reaches “excellent” young researchers by analysing their selection in terms of bibliometric indicators and look at the extent to which the StG effectively facilitates postdoctoral researchers' independence and their advancement on the academic career ladder. The results also provide insights on the sustainability of ERC funding in terms of the StG recipient’s position at the host institution and continuation of the StG research group after the end of the funding period.
Some selected MERCI findings:
- The relevance of the StG project in the researchers’ overall project portfolio differs across research domains. While in the SH domain the StG is essential for realising a specific research idea that would otherwise not be possible, in LS and PE domains an ERC grant is often embedded in larger projects and presents an integrative part of a more diversified funding portfolio. In some specific research fields, due to its thematic openness and flexibility, the StG tends to substitute for recurrent institutional funding.
- Bibliometric analysis reveals that past publication performance of approved and rejected StG applicants differs only moderately. Most of the StG applicants already show an above-average output prior to the application. In the analysed domains (LS and PE) over 90% of applicants have published at least six articles in international journals and over 75% of all applicants have authored at least one highly cited paper. Both hold for rejected and selected applicants. This evidence gives a clear indication of recognition of the ERC's principle of scientific excellence and demonstrates an effective self- or pre-selection attitude amongst potential StG candidates.
- Receiving a StG usually is followed by a higher level of autonomy in terms of allocation of material resources, human resources as well as lab and office space, while it has less effect on teaching activities and institutional co-decision. Compared to their peers at the same career level StG applicants (approved as well as rejected) report very high levels of scientific independency already at the point of applying for StG. This suggests that the StG serves as an instrument to complement already existing scientific independence by adding financial autonomy to it.
- The comparison of researchers’ time budgets reveals that approved applicants report a substantial surplus in time for research compared to rejected applicants. While the grantees on average dedicate 46% of their overall working time to genuine research, the respective share for the rejected applicants is 10 percentage points lower. Even when controlling for potential moderating factors (e.g. teaching load, position, research field and country group) receiving the StG results in a significant increase in research time if compared to rejected applicants.
- In the assessment of their working conditions approved applicants systematically report higher levels of satisfaction across all different aspects of work. The most significant difference shows up in the assessment of long term career perspectives where ERC grantees show much higher satisfaction.
Project Context and Objectives:
The diagnosis of the ERC reads that “Europe currently offers insufficient opportunities for young investigators to develop independent careers and make the transition from working under a supervisor to being independent researchers in their own right [...]” which “leads to a dramatic waste of research talent in Europe.” (http://erc.europa.eu/starting-grants). To encounter this structural drawback, the ERC introduced the StG funding scheme to promote “excellent” postdoctoral researchers (from 2 up to 12 years after obtaining their PhD) with up to € 1.5 million for the duration of up to 5 years in order to set up or consolidate their own research group. This endowment should enable the postdocs to pursue investigator-driven “frontier research”. The StG programme addresses, hence, different dimensions: It offers young researchers the opportunity for a research-oriented period allowing to independently conduct a large-scale research project and to develop leadership skills by leading their own research group at a host institution of their choice. Furthermore, it aims at creating positive framework conditions to foster mobility between or within countries and thereby harnessing the diversity of European research talents and channeling funds to the most promising researchers. The demand for the StG funding scheme is immense, which may indicate that the ERC addresses an important problem indeed and that the funding scheme is particularly lucrative – both in terms of the duration and the amount of funding. There is no doubt that since its short history, the ERC funding scheme is accompanied by a considerable prestige. Besides the high amount of funding, the prestige is presumably derived from the high standards of eligibility requirements and the rigorous selection process as well as the low acceptance rates (between 3 and 16 percent) fostering the ‘exclusivity’ of StGrantees.
Due to the fact that the StG programme has only been established a few years ago, the MERCI project (launched by the ERC in 2009) puts a strong emphasis on the programme’s implementation and how it works in practice. Thereby MERCI focuses on the individual perspective and aims to draw a broad picture concerning the questions whether the StG programme succeeds in attracting up-and-coming “excellent” young researchers from all over the world and in accomplishing its objectives. Furthermore, MERCI addresses direct and indirect effects of the StG funding scheme. Here, direct effects refer to the working conditions and the career development of StGrantees whereas indirect effects address rather structural changes (e.g. diffusion of standards in national funding systems) and changes in organisational respectively institutional settings. All in all, the MERCI project tackles four broad topics: 1) the funding strategies of StG applicants and their motivation to apply for a StG, 2) the ERC selection process, 3) the experiences of StG recipients with their host institution and their working conditions and 4) the outcome and sustainability of the StG funding.
In order to gain a comprehensive view on the StG programme implementation as well as the career development attributable to the StG funding, a triangulation approach and a comparative design with rejected StG applicants were chosen. The special gain of MERCI’s empirical approach does not only emerge from a parallel implementation of qualitative (semi-structured interviews) and quantitative methods (online survey in a panel design and bibliometric analysis), but also from the functional interlacing of partial respectively preliminary results, the development of instruments and the interpretation of empirical data. This step seems crucial considering the main problem of quantitative data in application-oriented contexts – such as evaluations – to effectively interpret the measured values of indicators in the whole context of the evaluation. For answering the question whether the programme performs well and whether the StG funding has any effects on the career development of young researchers, sound empirical evidence and a comprehensive gathering of individual aggregate data considering changes over time were required. Therefore, one building block of MERCI was a panel approach consisting of two waves of standardised online surveys allowing for a longitudinal design with several cohorts of approved and rejected StG applicants. The first panel wave was conducted at the beginning of the StG funding (respectively one year after the StG application as for rejected applicants) and the second wave in the last part of the StG funding period (respectively 3.5 years after the StG application). A short Intermediate survey ran between the first and the second wave survey.
MERCI can draw upon large empirical evidence based on three surveyed StG cohorts (StG 2009, 2010 and 2011 cohort) with more than 1,700 valid cases for the first wave survey and 500 for the second wave survey. For the qualitative interview study 40 interviews were carried out (29 with approved applicants and 11 with rejected ones of the StG 2009 cohort). The bibliometric analyses were conducted for approved and rejected applicants of the StG 2007 and 2009 cohorts.
For elaborating of how the StG funding scheme works in practice, MERCI aimed to compare approved and rejected StG applicants, reveal cohort specific effects, take into account differences across nationalities, research fields and positions and, not least, triangulate empirical results gathered from the different in order to gain a comprehensive understanding.
Project Results:
Below, selected empirical findings are presented along the four main topics of MERCI:
1) Funding strategies of StG applicants and motivation to apply for a StG
1.1) Regardless of their research field, their current host country or their funding status, the motivation to apply for a StG is mainly driven by the endowment which the grant offers. This on the one hand refers to the amount of funding and its duration but also to its thematic openness and freedom to set own research priorities. Contrasting the motivation to apply for a StG with the motivation to apply for other funding bodies reveals that these are also the characteristics which are perceived as outstanding. The reputation of the StG also poses an important, but slightly less relevant, motive. Whereas it is almost consent among all applicants that the attractivity of the StG mainly arises from its generous endowment and flexibility, for a minor but relevant group a StG application proves to be an essential attempt to ensure the funding of the own position or to compensate the lack of other funding opportunities. These motives have been found most relevant for respondents from Humanities, in Eastern European and Scandinavian countries but also for the rejected applicants. The relevance of the StG project in the overall project portfolio of researchers differs across research domains. While in Social Sciences and Humanities the StG is essential for realising a long-harboured research idea, in Life Sciences and Physical Sciences and Engineering an ERC grant is often embedded in larger projects and presents an integrative part of a more diversified funding portfolio. While in Social Sciences and Humanities the StG funding is crucial for the realisation of the research project as well as for the career prospects of the Principal Investigator (in some cases even the employment of the Principal Investigator would have been endangered without the StG), in other research fields the realisation of the StG project idea is still deemed possible without the ERC funding – maybe but not necessarily with some small adjustments due to a smaller funding budget.
1.2) Due to its thematic openness and flexibility, in some specific research fields the StG tends to substitute for recurrent institutional funding: In Life Sciences receiving a StG obviously does not come along with a general shift in the relevance of funding sources since the applicants usually make use of a large variety of other sources. Here, the StG apparently neither compensates a general lack of funding which would restrict the general expansion of research activities nor does it substitute specific other sources. In Natural Sciences and Engineering, the StG rather tends to substitute recurrent funding, whereas the number of projects funded by third parties remains almost stable. Furthermore, in Natural Sciences the StG leads to an increase in the overall number of projects, whereas this trend is not observable in Engineering. In Social Sciences the composition of projects does hardly differ between approved and rejected applicants, meaning regardless whether the respondents receive a StG or not, third-party funding, recurrent funding and individual fellowships are of equal relevance and the StG project is ‘added’ to the existing set of projects with evoking any general shift in funding sources. Given that in Humanities among the Grantees the share of projects funded by individual fellowships and recurrent funding is substantially lower than among the rejected applicants (whereas the number of projects based on third party funding remains almost equal), one may conclude that the StG funding in this field primarily substitutes recurrent funding and fellowships.
1.3) The survey results as well as the qualitative interviews reveal that only a minority of the StG proposals are written without any external support. Advice by colleagues or ERC’s National Contact Points significantly increases the chances to succeed – but only to a minor extent. Whereas the survey findings suggest that individual researchers’ acceptance rates are not very strongly moderated by the availability and usage of external support, the qualitative interviews suggest that especially informed peers might provide useful input for a successful application. Even in view of an expansion of institutionalised support, informal advice by experienced peers still plays the most important role for preparing applications. However, the support focuses rather on the question how to design and adjust a proposal so that it fits the norms. This observation in conjunction with the increasing efforts on behalf of the institutions to ‘pre-select’ promising candidates, to offer trainings for oral presentations and other specific courses leads to the conclusion that at least in some cases the StG becomes a ‘collective endeavor’.
2) ERC selection process
2.1) The bibliometric analysis reveals that past publication performance of approved and rejected StG applicants differs only moderately. Most of the StG applicants already show an above-average output prior to the application. In the analysed domains (Life Sciences as well as Physical Sciences and Engineering) over 90 percent of applicants have published at least six articles in international journals and over 75 percent of all applicants have authored at least one highly cited paper. Both hold for rejected and selected applicants. This evidence indicates the strong recognition of the ERC's principle of scientific excellence and demonstrates an effective self-selection attitude amongst potential StG candidates. Furthermore, in order to have a look at the internationality of approved and rejected StG applicants both the international cooperation and international perception were analysed. The MERCI results indicate that for both aspects approved applicants reach higher indicator values than rejected applicants.
2.2) Respondents were asked to assess the ERC evaluation process along a set of criteria which are expected to reflect different dimensions respectively stages of the evaluation process ranging from the organisation of the application and the review process to contracting after a positive evaluation. A principal components analysis of the criteria confirms the underlying three dimensional structures of the assessment dimensions. The survey findings suggest a quite balanced contribution of formal-administrative and review-related criteria to researchers’ overall satisfaction with the evaluation procedure and that subsequently equal priority should be given to both aspects when striving for further improvements of the StG programme.
2.3) Compared to the evaluation process of other funding bodies (European Commission, foundations, national and subnational level governmental funding bodies, e.g. ministries), the StG programme does not show weaknesses with regard to the application phase, and is rather characterised as average. While information provided in terms of documentation seems sufficient, in contrast to other funding agencies a need for improvement for the StG programme line probably consists in the speed and transparent information on the status of evaluation process itself. As for the remaining funding bodies the quality of the evaluation reports is evaluated most critically.
3) Experiences of StG recipients with their host institution and their working conditions
3.1) The StG is seldom used to evoke mobility: More than 80 percent of the MERCI respondents prefer to stay not only in the same country where they lived when applying for the StG, they even stay at the same institution. Not even 11 percent of the StG recipients use the grant in order to move to an institution in another country. When regarding the type of host institution it is obvious that the majority of respondents prefer to work at a university. Hence, in many cases the StG funding is used to improve the research conditions at the university where StGrantees were already working when applying for the grant.
3.2) Familiarity with the research institution and their reputation are of utmost importance for the choice of a StG host institution. Overall, three out of four StGrantees deem the reputation an important or very important factor, but especially in Humanities and Life Sciences reputation counts for most. Best contractual conditions and research infrastructure are in general less relevant but very important criteria for the quite small group of StGrantees who decide to leave their current host institution and country for realising their StG project.
3.3) Receiving a StG usually is followed by a higher level of autonomy in terms of allocation of material resources, human resources as well as lab and office space, while it has less effect on teaching activities and institutional co-decision. Compared to their peers at the same career level, approved as well as rejected StG applicants report extraordinary high levels of scientific independency. This may suggest that the StG serves as an instrument to complement already existing scientific independence by adding financial autonomy to it. The comparison of researchers’ time budgets reveals that the StG comes along with a substantial surplus in time for research compared to rejected applicants. While the grantees on average dedicate 46 percent of their overall working time to genuine research, the respective share for the rejected applicants is 10 percentage points lower. Even when controlling for potential moderating factors (e.g. teaching load, position, research field and country group) receiving the StG results in a significant increase in research time if compared to rejected applicants. In the assessment of their working conditions, approved applicants systematically report higher levels of satisfaction across all different aspects of work. The most significant changes compared to the rejected applicants show up in the assessment of long term career perspectives and with regard to reputation at the host institution and in the scientific community.
4) Outcome and sustainability of the StG funding
4.1) The development of skills and competencies acted as an outcome dimension of the StG funding. To get a proxy for the perceived developments of individual skills – independently of the career stage and the position a respondent holds – approved and rejected StG applicants were asked to rank their own level of competence compared to colleagues at the same career level. The respondent’s overall tendency to rate their own competencies as “above average” is remarkable, but points (again) to a strong self-selection among the applicants. Due to the fact that both approved and rejected applicants perceive their abilities to carry out research independently as very strong compared to colleagues on the same career level, it seems plausible that strong research skills might be a precondition for the application rather than an outcome of the StG programme. Furthermore, among the approved StG applicants skill development is concentrated on a small set of competencies, namely leadership, acquisition of research funding and networking skills, whereas in the reference group a broader set of skills has been mentioned and the picture appears much more heterogeneous.
4.2) The survey data suggests substantial vertical mobility on the organisational career ladder among MERCI respondents. Between the StG application phase and the second wave survey, the share of respondents holding a full or associate professorship doubled, while the share of those holding an assistant professorship or group leader positions remained stable. In addition, the findings show that the StG recipients obtained full or associate professorships in close temporal conjunction with funding decision. This provides evidence to the hypothesis that promotion is received in reward of the successful StG application. In general, for numerous applicants the StG serves as an official confirmation of their scientific recognition and helps them to progress on their organisational career while it is less relevant as an instrument to achieve scientific independence.
4.3) As for the sustainability of the StG funding, the survey data indicates that the majority of StGrantees intend to stay at their StG host institution (68 percent), 24 percent are still unsure and 9 percent intend to change the institution. 82 percent of the StG recipients have a permanent position in sight, 9 percent have a temporary one and only 1.5 percent does not have further employment opportunities and the remainder did not make any agreements so far. The individual perspective of the further employment at the StG host institution differs between research fields. Regarding the StG research group, also the majority of them will most likely continue to exist in a similar composition at the current institution (62 percent). In 16 percent of the cases the research group will most likely be dissolved and 4 percent of the groups will most likely move to another research organisation. It is striking that for 17 percent of the respondents the future of their research group is quite unclear. Here, differences across research fields come into play: compared to other research fields, in Humanities as well as in Social Sciences, the future of the StG research group is most frequently less sustainable or unclear.
Potential Impact:
Without doubt, the ambitious aim of the ERC’s StG program can only be reached if the ERC is considered to be a learning organization that is continuously being informed about how its funding schemes works in practice and which effects they bring along. Due to the fact that the StG programme has only been established a few years ago, MERCI – as an ERC CSA-project – puts a strong emphasis on the programme’s implementation. Thereby MERCI focuses on the individual perspective and aims to draw a broad picture concerning the questions whether the StG programme succeeds in attracting up-and-coming “excellent” young researchers and in accomplishing its objectives namely promoting young researchers to carry out research independently and to conduct ground-breaking frontier research.
Furthermore, the MERCI project addresses direct and indirect effects of the StG funding scheme recognising that the unsettled nature of a programme in its beginning years makes it difficult to actually measure “impact” in the sense of long-lasting consequences and long-term changes. Here, direct effects refer to the working conditions and the career development of StGrantees whereas indirect effects address rather structural changes (e.g. diffusion of standards in national funding systems) and changes in organisational respectively institutional settings. Hence, MERCI does not only evaluate the programme’s objectives, but question the objectives themselves by assessing if the tailoring of the StG funding scheme is suitable to meet the specific needs of postdocs.
Undoubtedly, MERCI’s objects of evaluation depend on many contingent factors and are, thus, faced with a complex and multidimensional endeavour. The StG programme targets a quite heterogeneous group of postdoctoral scientists which makes it rather challenging to derive conclusions about genuine effects which are exclusively attributable to the ERC funding. Heterogeneity in this context does not only refer to the Principal Investigator’s characteristics (e. g. with respect to discipline, age or postdoc experience), but also touches on the institutional setting as well as the national research system he/she is embedded in.
In order to try to provide valid answers to the evaluation questions, MERCI chose both a triangulation approach and a comparative design with rejected StG applicants as control group. The special gain of this empirical approach does not only emerge from a parallel implementation of qualitative (semi-structured interviews) and quantitative methods (online survey in a panel design and bibliometric analysis) but also from the functional interlacing of partial respectively preliminary results, the development of instruments and the interpretation of data.
During the project duration a continuous exchange with experts from different fields (experts on methodology, science policy, evaluation experts etc.) was assured and (preliminary) project results were disseminated by several channels. The main emphasis was placed on conferences – scientific conferences as well as conferences with a broader audience – to ensure a continuous exchange with relevant stakeholders from different target groups (science, experts on evaluation, science policy etc.). Furthermore, empirical key finding gathered by the different method lines were discussed across different topics with experts from the MERCI consortium and the ERC in several workshops.
In addition to the regular reports to the ERC, a closing workshop was scheduled. In this final workshop, not only the results of the project were presented, but also the experiences of the project partners involved were exchanged. It took place in summer 2014 to guarantee that the results of the discussion could still be integrated in MERCI's Synthesis Report.
For a complete list of MERCI's dissemination activities see Section 4.2
In February 2009, the European Research Council (ERC) launched the MERCI project (“Monitoring European Research Council’s Implementation of Excellence”) to monitor its “Starting Grants” (StG) funding scheme. The StG programme addresses “excellent” postdocs with up to 1.5 million Euro for the duration of up to five years to set up or consolidate their own research group in order to pursue independent, investigator-driven “frontier research”. Due to the fact that the StG programme has only been established in 2007, the MERCI project puts a strong emphasis on the programme’s implementation and how it works in practice. Thereby MERCI focuses on the individual perspective and aims to draw a broad picture concerning direct and indirect effects of the StG funding scheme. MERCI does not only evaluate the programme’s objectives, but questions the objectives themselves by assessing if the tailoring of the StG funding scheme is suitable to meet the specific needs of postdocs. The project was implemented as an ERC Coordination and Support Actions (CSA) project and had been completed in 2014. It was carried out by a consortium of four partners: the Institute for Research Information and Quality Assurance (iFQ), TU Dortmund University, Bielefeld University and Humboldt University of Berlin.
MERCI delivered its results by drawing upon large empirical evidence based on three StG cohorts and more than 1000 ERC applicants. Results shed light on the applicants’ research funding strategies and their motivation to apply to ERC in comparison with other existing research funding opportunities. They depict the role of the ERC grant in the overall research funding portfolio of ERC applicants. They investigate if ERC reaches “excellent” young researchers by analysing their selection in terms of bibliometric indicators and look at the extent to which the StG effectively facilitates postdoctoral researchers' independence and their advancement on the academic career ladder. The results also provide insights on the sustainability of ERC funding in terms of the StG recipient’s position at the host institution and continuation of the StG research group after the end of the funding period.
Some selected MERCI findings:
- The relevance of the StG project in the researchers’ overall project portfolio differs across research domains. While in the SH domain the StG is essential for realising a specific research idea that would otherwise not be possible, in LS and PE domains an ERC grant is often embedded in larger projects and presents an integrative part of a more diversified funding portfolio. In some specific research fields, due to its thematic openness and flexibility, the StG tends to substitute for recurrent institutional funding.
- Bibliometric analysis reveals that past publication performance of approved and rejected StG applicants differs only moderately. Most of the StG applicants already show an above-average output prior to the application. In the analysed domains (LS and PE) over 90% of applicants have published at least six articles in international journals and over 75% of all applicants have authored at least one highly cited paper. Both hold for rejected and selected applicants. This evidence gives a clear indication of recognition of the ERC's principle of scientific excellence and demonstrates an effective self- or pre-selection attitude amongst potential StG candidates.
- Receiving a StG usually is followed by a higher level of autonomy in terms of allocation of material resources, human resources as well as lab and office space, while it has less effect on teaching activities and institutional co-decision. Compared to their peers at the same career level StG applicants (approved as well as rejected) report very high levels of scientific independency already at the point of applying for StG. This suggests that the StG serves as an instrument to complement already existing scientific independence by adding financial autonomy to it.
- The comparison of researchers’ time budgets reveals that approved applicants report a substantial surplus in time for research compared to rejected applicants. While the grantees on average dedicate 46% of their overall working time to genuine research, the respective share for the rejected applicants is 10 percentage points lower. Even when controlling for potential moderating factors (e.g. teaching load, position, research field and country group) receiving the StG results in a significant increase in research time if compared to rejected applicants.
- In the assessment of their working conditions approved applicants systematically report higher levels of satisfaction across all different aspects of work. The most significant difference shows up in the assessment of long term career perspectives where ERC grantees show much higher satisfaction.
Project Context and Objectives:
The diagnosis of the ERC reads that “Europe currently offers insufficient opportunities for young investigators to develop independent careers and make the transition from working under a supervisor to being independent researchers in their own right [...]” which “leads to a dramatic waste of research talent in Europe.” (http://erc.europa.eu/starting-grants). To encounter this structural drawback, the ERC introduced the StG funding scheme to promote “excellent” postdoctoral researchers (from 2 up to 12 years after obtaining their PhD) with up to € 1.5 million for the duration of up to 5 years in order to set up or consolidate their own research group. This endowment should enable the postdocs to pursue investigator-driven “frontier research”. The StG programme addresses, hence, different dimensions: It offers young researchers the opportunity for a research-oriented period allowing to independently conduct a large-scale research project and to develop leadership skills by leading their own research group at a host institution of their choice. Furthermore, it aims at creating positive framework conditions to foster mobility between or within countries and thereby harnessing the diversity of European research talents and channeling funds to the most promising researchers. The demand for the StG funding scheme is immense, which may indicate that the ERC addresses an important problem indeed and that the funding scheme is particularly lucrative – both in terms of the duration and the amount of funding. There is no doubt that since its short history, the ERC funding scheme is accompanied by a considerable prestige. Besides the high amount of funding, the prestige is presumably derived from the high standards of eligibility requirements and the rigorous selection process as well as the low acceptance rates (between 3 and 16 percent) fostering the ‘exclusivity’ of StGrantees.
Due to the fact that the StG programme has only been established a few years ago, the MERCI project (launched by the ERC in 2009) puts a strong emphasis on the programme’s implementation and how it works in practice. Thereby MERCI focuses on the individual perspective and aims to draw a broad picture concerning the questions whether the StG programme succeeds in attracting up-and-coming “excellent” young researchers from all over the world and in accomplishing its objectives. Furthermore, MERCI addresses direct and indirect effects of the StG funding scheme. Here, direct effects refer to the working conditions and the career development of StGrantees whereas indirect effects address rather structural changes (e.g. diffusion of standards in national funding systems) and changes in organisational respectively institutional settings. All in all, the MERCI project tackles four broad topics: 1) the funding strategies of StG applicants and their motivation to apply for a StG, 2) the ERC selection process, 3) the experiences of StG recipients with their host institution and their working conditions and 4) the outcome and sustainability of the StG funding.
In order to gain a comprehensive view on the StG programme implementation as well as the career development attributable to the StG funding, a triangulation approach and a comparative design with rejected StG applicants were chosen. The special gain of MERCI’s empirical approach does not only emerge from a parallel implementation of qualitative (semi-structured interviews) and quantitative methods (online survey in a panel design and bibliometric analysis), but also from the functional interlacing of partial respectively preliminary results, the development of instruments and the interpretation of empirical data. This step seems crucial considering the main problem of quantitative data in application-oriented contexts – such as evaluations – to effectively interpret the measured values of indicators in the whole context of the evaluation. For answering the question whether the programme performs well and whether the StG funding has any effects on the career development of young researchers, sound empirical evidence and a comprehensive gathering of individual aggregate data considering changes over time were required. Therefore, one building block of MERCI was a panel approach consisting of two waves of standardised online surveys allowing for a longitudinal design with several cohorts of approved and rejected StG applicants. The first panel wave was conducted at the beginning of the StG funding (respectively one year after the StG application as for rejected applicants) and the second wave in the last part of the StG funding period (respectively 3.5 years after the StG application). A short Intermediate survey ran between the first and the second wave survey.
MERCI can draw upon large empirical evidence based on three surveyed StG cohorts (StG 2009, 2010 and 2011 cohort) with more than 1,700 valid cases for the first wave survey and 500 for the second wave survey. For the qualitative interview study 40 interviews were carried out (29 with approved applicants and 11 with rejected ones of the StG 2009 cohort). The bibliometric analyses were conducted for approved and rejected applicants of the StG 2007 and 2009 cohorts.
For elaborating of how the StG funding scheme works in practice, MERCI aimed to compare approved and rejected StG applicants, reveal cohort specific effects, take into account differences across nationalities, research fields and positions and, not least, triangulate empirical results gathered from the different in order to gain a comprehensive understanding.
Project Results:
Below, selected empirical findings are presented along the four main topics of MERCI:
1) Funding strategies of StG applicants and motivation to apply for a StG
1.1) Regardless of their research field, their current host country or their funding status, the motivation to apply for a StG is mainly driven by the endowment which the grant offers. This on the one hand refers to the amount of funding and its duration but also to its thematic openness and freedom to set own research priorities. Contrasting the motivation to apply for a StG with the motivation to apply for other funding bodies reveals that these are also the characteristics which are perceived as outstanding. The reputation of the StG also poses an important, but slightly less relevant, motive. Whereas it is almost consent among all applicants that the attractivity of the StG mainly arises from its generous endowment and flexibility, for a minor but relevant group a StG application proves to be an essential attempt to ensure the funding of the own position or to compensate the lack of other funding opportunities. These motives have been found most relevant for respondents from Humanities, in Eastern European and Scandinavian countries but also for the rejected applicants. The relevance of the StG project in the overall project portfolio of researchers differs across research domains. While in Social Sciences and Humanities the StG is essential for realising a long-harboured research idea, in Life Sciences and Physical Sciences and Engineering an ERC grant is often embedded in larger projects and presents an integrative part of a more diversified funding portfolio. While in Social Sciences and Humanities the StG funding is crucial for the realisation of the research project as well as for the career prospects of the Principal Investigator (in some cases even the employment of the Principal Investigator would have been endangered without the StG), in other research fields the realisation of the StG project idea is still deemed possible without the ERC funding – maybe but not necessarily with some small adjustments due to a smaller funding budget.
1.2) Due to its thematic openness and flexibility, in some specific research fields the StG tends to substitute for recurrent institutional funding: In Life Sciences receiving a StG obviously does not come along with a general shift in the relevance of funding sources since the applicants usually make use of a large variety of other sources. Here, the StG apparently neither compensates a general lack of funding which would restrict the general expansion of research activities nor does it substitute specific other sources. In Natural Sciences and Engineering, the StG rather tends to substitute recurrent funding, whereas the number of projects funded by third parties remains almost stable. Furthermore, in Natural Sciences the StG leads to an increase in the overall number of projects, whereas this trend is not observable in Engineering. In Social Sciences the composition of projects does hardly differ between approved and rejected applicants, meaning regardless whether the respondents receive a StG or not, third-party funding, recurrent funding and individual fellowships are of equal relevance and the StG project is ‘added’ to the existing set of projects with evoking any general shift in funding sources. Given that in Humanities among the Grantees the share of projects funded by individual fellowships and recurrent funding is substantially lower than among the rejected applicants (whereas the number of projects based on third party funding remains almost equal), one may conclude that the StG funding in this field primarily substitutes recurrent funding and fellowships.
1.3) The survey results as well as the qualitative interviews reveal that only a minority of the StG proposals are written without any external support. Advice by colleagues or ERC’s National Contact Points significantly increases the chances to succeed – but only to a minor extent. Whereas the survey findings suggest that individual researchers’ acceptance rates are not very strongly moderated by the availability and usage of external support, the qualitative interviews suggest that especially informed peers might provide useful input for a successful application. Even in view of an expansion of institutionalised support, informal advice by experienced peers still plays the most important role for preparing applications. However, the support focuses rather on the question how to design and adjust a proposal so that it fits the norms. This observation in conjunction with the increasing efforts on behalf of the institutions to ‘pre-select’ promising candidates, to offer trainings for oral presentations and other specific courses leads to the conclusion that at least in some cases the StG becomes a ‘collective endeavor’.
2) ERC selection process
2.1) The bibliometric analysis reveals that past publication performance of approved and rejected StG applicants differs only moderately. Most of the StG applicants already show an above-average output prior to the application. In the analysed domains (Life Sciences as well as Physical Sciences and Engineering) over 90 percent of applicants have published at least six articles in international journals and over 75 percent of all applicants have authored at least one highly cited paper. Both hold for rejected and selected applicants. This evidence indicates the strong recognition of the ERC's principle of scientific excellence and demonstrates an effective self-selection attitude amongst potential StG candidates. Furthermore, in order to have a look at the internationality of approved and rejected StG applicants both the international cooperation and international perception were analysed. The MERCI results indicate that for both aspects approved applicants reach higher indicator values than rejected applicants.
2.2) Respondents were asked to assess the ERC evaluation process along a set of criteria which are expected to reflect different dimensions respectively stages of the evaluation process ranging from the organisation of the application and the review process to contracting after a positive evaluation. A principal components analysis of the criteria confirms the underlying three dimensional structures of the assessment dimensions. The survey findings suggest a quite balanced contribution of formal-administrative and review-related criteria to researchers’ overall satisfaction with the evaluation procedure and that subsequently equal priority should be given to both aspects when striving for further improvements of the StG programme.
2.3) Compared to the evaluation process of other funding bodies (European Commission, foundations, national and subnational level governmental funding bodies, e.g. ministries), the StG programme does not show weaknesses with regard to the application phase, and is rather characterised as average. While information provided in terms of documentation seems sufficient, in contrast to other funding agencies a need for improvement for the StG programme line probably consists in the speed and transparent information on the status of evaluation process itself. As for the remaining funding bodies the quality of the evaluation reports is evaluated most critically.
3) Experiences of StG recipients with their host institution and their working conditions
3.1) The StG is seldom used to evoke mobility: More than 80 percent of the MERCI respondents prefer to stay not only in the same country where they lived when applying for the StG, they even stay at the same institution. Not even 11 percent of the StG recipients use the grant in order to move to an institution in another country. When regarding the type of host institution it is obvious that the majority of respondents prefer to work at a university. Hence, in many cases the StG funding is used to improve the research conditions at the university where StGrantees were already working when applying for the grant.
3.2) Familiarity with the research institution and their reputation are of utmost importance for the choice of a StG host institution. Overall, three out of four StGrantees deem the reputation an important or very important factor, but especially in Humanities and Life Sciences reputation counts for most. Best contractual conditions and research infrastructure are in general less relevant but very important criteria for the quite small group of StGrantees who decide to leave their current host institution and country for realising their StG project.
3.3) Receiving a StG usually is followed by a higher level of autonomy in terms of allocation of material resources, human resources as well as lab and office space, while it has less effect on teaching activities and institutional co-decision. Compared to their peers at the same career level, approved as well as rejected StG applicants report extraordinary high levels of scientific independency. This may suggest that the StG serves as an instrument to complement already existing scientific independence by adding financial autonomy to it. The comparison of researchers’ time budgets reveals that the StG comes along with a substantial surplus in time for research compared to rejected applicants. While the grantees on average dedicate 46 percent of their overall working time to genuine research, the respective share for the rejected applicants is 10 percentage points lower. Even when controlling for potential moderating factors (e.g. teaching load, position, research field and country group) receiving the StG results in a significant increase in research time if compared to rejected applicants. In the assessment of their working conditions, approved applicants systematically report higher levels of satisfaction across all different aspects of work. The most significant changes compared to the rejected applicants show up in the assessment of long term career perspectives and with regard to reputation at the host institution and in the scientific community.
4) Outcome and sustainability of the StG funding
4.1) The development of skills and competencies acted as an outcome dimension of the StG funding. To get a proxy for the perceived developments of individual skills – independently of the career stage and the position a respondent holds – approved and rejected StG applicants were asked to rank their own level of competence compared to colleagues at the same career level. The respondent’s overall tendency to rate their own competencies as “above average” is remarkable, but points (again) to a strong self-selection among the applicants. Due to the fact that both approved and rejected applicants perceive their abilities to carry out research independently as very strong compared to colleagues on the same career level, it seems plausible that strong research skills might be a precondition for the application rather than an outcome of the StG programme. Furthermore, among the approved StG applicants skill development is concentrated on a small set of competencies, namely leadership, acquisition of research funding and networking skills, whereas in the reference group a broader set of skills has been mentioned and the picture appears much more heterogeneous.
4.2) The survey data suggests substantial vertical mobility on the organisational career ladder among MERCI respondents. Between the StG application phase and the second wave survey, the share of respondents holding a full or associate professorship doubled, while the share of those holding an assistant professorship or group leader positions remained stable. In addition, the findings show that the StG recipients obtained full or associate professorships in close temporal conjunction with funding decision. This provides evidence to the hypothesis that promotion is received in reward of the successful StG application. In general, for numerous applicants the StG serves as an official confirmation of their scientific recognition and helps them to progress on their organisational career while it is less relevant as an instrument to achieve scientific independence.
4.3) As for the sustainability of the StG funding, the survey data indicates that the majority of StGrantees intend to stay at their StG host institution (68 percent), 24 percent are still unsure and 9 percent intend to change the institution. 82 percent of the StG recipients have a permanent position in sight, 9 percent have a temporary one and only 1.5 percent does not have further employment opportunities and the remainder did not make any agreements so far. The individual perspective of the further employment at the StG host institution differs between research fields. Regarding the StG research group, also the majority of them will most likely continue to exist in a similar composition at the current institution (62 percent). In 16 percent of the cases the research group will most likely be dissolved and 4 percent of the groups will most likely move to another research organisation. It is striking that for 17 percent of the respondents the future of their research group is quite unclear. Here, differences across research fields come into play: compared to other research fields, in Humanities as well as in Social Sciences, the future of the StG research group is most frequently less sustainable or unclear.
Potential Impact:
Without doubt, the ambitious aim of the ERC’s StG program can only be reached if the ERC is considered to be a learning organization that is continuously being informed about how its funding schemes works in practice and which effects they bring along. Due to the fact that the StG programme has only been established a few years ago, MERCI – as an ERC CSA-project – puts a strong emphasis on the programme’s implementation. Thereby MERCI focuses on the individual perspective and aims to draw a broad picture concerning the questions whether the StG programme succeeds in attracting up-and-coming “excellent” young researchers and in accomplishing its objectives namely promoting young researchers to carry out research independently and to conduct ground-breaking frontier research.
Furthermore, the MERCI project addresses direct and indirect effects of the StG funding scheme recognising that the unsettled nature of a programme in its beginning years makes it difficult to actually measure “impact” in the sense of long-lasting consequences and long-term changes. Here, direct effects refer to the working conditions and the career development of StGrantees whereas indirect effects address rather structural changes (e.g. diffusion of standards in national funding systems) and changes in organisational respectively institutional settings. Hence, MERCI does not only evaluate the programme’s objectives, but question the objectives themselves by assessing if the tailoring of the StG funding scheme is suitable to meet the specific needs of postdocs.
Undoubtedly, MERCI’s objects of evaluation depend on many contingent factors and are, thus, faced with a complex and multidimensional endeavour. The StG programme targets a quite heterogeneous group of postdoctoral scientists which makes it rather challenging to derive conclusions about genuine effects which are exclusively attributable to the ERC funding. Heterogeneity in this context does not only refer to the Principal Investigator’s characteristics (e. g. with respect to discipline, age or postdoc experience), but also touches on the institutional setting as well as the national research system he/she is embedded in.
In order to try to provide valid answers to the evaluation questions, MERCI chose both a triangulation approach and a comparative design with rejected StG applicants as control group. The special gain of this empirical approach does not only emerge from a parallel implementation of qualitative (semi-structured interviews) and quantitative methods (online survey in a panel design and bibliometric analysis) but also from the functional interlacing of partial respectively preliminary results, the development of instruments and the interpretation of data.
During the project duration a continuous exchange with experts from different fields (experts on methodology, science policy, evaluation experts etc.) was assured and (preliminary) project results were disseminated by several channels. The main emphasis was placed on conferences – scientific conferences as well as conferences with a broader audience – to ensure a continuous exchange with relevant stakeholders from different target groups (science, experts on evaluation, science policy etc.). Furthermore, empirical key finding gathered by the different method lines were discussed across different topics with experts from the MERCI consortium and the ERC in several workshops.
In addition to the regular reports to the ERC, a closing workshop was scheduled. In this final workshop, not only the results of the project were presented, but also the experiences of the project partners involved were exchanged. It took place in summer 2014 to guarantee that the results of the discussion could still be integrated in MERCI's Synthesis Report.
For a complete list of MERCI's dissemination activities see Section 4.2
