Periodic Reporting for period 2 - FeedS First (Feedback for Small Companies and Firsttimers (FeedS First))
Période du rapport: 2021-04-01 au 2023-04-30
SMEs are the backbone of the European economy and innovation is key to maximising their productivity and viability in the long term. As an innovation agency, the FFG provides funding to SMEs to undertake innovation activities in their firms, however currently funded SMEs do not receive information on how their innovation projects could be improved. This is mainly because it is essential to provide support fast & easy. Providing feedback on the other hand needs time.
FeedsFirst is thus using experimental, evidence-based approaches to investigate the efficacy of feedback to SMEs' innovation projects on innovation activities in SMEs. By using experimental approaches, in this case a randomised controlled trial, we are able to gather robust evidence on the actual effectiveness the feedback is on fostering firms' innovation activities.
To summarize, the results of the RCT indicate that the type of feedback (with or without a benchmark score) does not have a significant influence on the project’s success. At least calculated with the amount of end evaluations we have so far.
FeedsFirst is thus using experimental, evidence-based approaches to investigate the efficacy of feedback to SMEs' innovation projects on innovation activities in SMEs. By using experimental approaches, in this case a randomised controlled trial, we are able to gather robust evidence on the actual effectiveness the feedback is on fostering firms' innovation activities.
To summarize, the results of the RCT indicate that the type of feedback (with or without a benchmark score) does not have a significant influence on the project’s success. At least calculated with the amount of end evaluations we have so far.
The work performed in the first reporting period encompasses the finalisation and implementation of the RCT, which will run over the course of approximately one year. The Action began with an intensive design finalisation phase with scientific partners at the University of Minnesota Carlson School of Management and RCT consultants from Innovation Growth Lab by Nesta, in which the research design was finalised and translated into an actionable plan. In short, the final research design is a randomized, double-blind, two-arm trial testing the impact of a relative ranking score in addition to feedback on project application on improving project outcomes, future projects, and sustainability aspects in research and innovation projects funded under FFG’s funding program General Programmes (Basisprogramm). The sample comprised firms receiving funding from General Programmes, specifically SMEs and first time applicants, who are randomized into either the control group or the treatment group. The intervention, which only the treatment group receives, consisted of the firm’s relative ranking score for each of the 4 categories in the feedback. A relative ranking score ranked the respective firm’s performance in a specific category among all the firms in their sample. The control group received the same feedback sheet, only without the relative ranking scores. The trial has been designed to take place over the course of one year, admitting firms into the sample in line with the funding decisions for the General Programme, thus our trial has a unique “trickle sample”. The main outcomes being measured are the impact of the intervention on success of project implementation which is evaluated by the project evaluators at project end.
To summarize, the results of the RCT indicate that the type of feedback (with or without a benchmark score) does not have a significant influence on the project’s success. At least calculated with the amount of end evaluations we have so far. Since the original sample comprised 164 firms, not all projects could be included in this analysis due to various project delays. This indicates that the lack of significant differences between the treatment and control group can be a result of the small sample size. So far, the type of feedback does not have a significant influence on project success that would make any impacts detectable right now. Results will be reanalysed by our University partners in a year when most of the end reports will be available.
To summarize, the results of the RCT indicate that the type of feedback (with or without a benchmark score) does not have a significant influence on the project’s success. At least calculated with the amount of end evaluations we have so far. Since the original sample comprised 164 firms, not all projects could be included in this analysis due to various project delays. This indicates that the lack of significant differences between the treatment and control group can be a result of the small sample size. So far, the type of feedback does not have a significant influence on project success that would make any impacts detectable right now. Results will be reanalysed by our University partners in a year when most of the end reports will be available.
The current preliminary results show that the tested types of feedbacks do not have a significant effect. The control with historic data indicate, that the automated feedback does not have an effect. This might lead to the conclusion that giving feedback for all projects might not be the best way to provide feedback. FFG will further exploit possibilites of giving written feedback, in the case that evaluators indicate useful feedback. Furthermore, follow-up evaluations of feedback mechanism show, that the interest for feedback is highest in scientific organisations and less high in businesses.
So providing feedback seems not to be useful in any case and therefore spending time and resources to provide feedback for every project, might not be worth it. The results indicate, that one should consider the type of organisation and type of project, when deciding if written feedback is given.
So providing feedback seems not to be useful in any case and therefore spending time and resources to provide feedback for every project, might not be worth it. The results indicate, that one should consider the type of organisation and type of project, when deciding if written feedback is given.