The work performed in the first reporting period encompasses the finalisation and implementation of the RCT, which will run over the course of approximately one year. The Action began with an intensive design finalisation phase with scientific partners at the University of Minnesota Carlson School of Management and RCT consultants from Innovation Growth Lab by Nesta, in which the research design was finalised and translated into an actionable plan. In short, the final research design is a randomized, double-blind, two-arm trial testing the impact of a relative ranking score in addition to feedback on project application on improving project outcomes, future projects, and sustainability aspects in research and innovation projects funded under FFG’s funding program General Programmes (Basisprogramm). The sample comprised firms receiving funding from General Programmes, specifically SMEs and first time applicants, who are randomized into either the control group or the treatment group. The intervention, which only the treatment group receives, consisted of the firm’s relative ranking score for each of the 4 categories in the feedback. A relative ranking score ranked the respective firm’s performance in a specific category among all the firms in their sample. The control group received the same feedback sheet, only without the relative ranking scores. The trial has been designed to take place over the course of one year, admitting firms into the sample in line with the funding decisions for the General Programme, thus our trial has a unique “trickle sample”. The main outcomes being measured are the impact of the intervention on success of project implementation which is evaluated by the project evaluators at project end.
To summarize, the results of the RCT indicate that the type of feedback (with or without a benchmark score) does not have a significant influence on the project’s success. At least calculated with the amount of end evaluations we have so far. Since the original sample comprised 164 firms, not all projects could be included in this analysis due to various project delays. This indicates that the lack of significant differences between the treatment and control group can be a result of the small sample size. So far, the type of feedback does not have a significant influence on project success that would make any impacts detectable right now. Results will be reanalysed by our University partners in a year when most of the end reports will be available.
