CIVIL SOCIETY ORGANISATIONS IN DESIGNING RESEARCH GOVERNANCE

Final Report Summary - CONSIDER (CIVIL SOCIETY ORGANISATIONS IN DESIGNING RESEARCH GOVERNANCE)

Executive Summary:
Civil Society Organisations (CSOs) are increasingly playing a vital role in research. In the context of the EU policy agenda on responsible research and innovation (RRI) and its focus on public engagement, this role is likely to continue to grow in the future. At present, however, CSO participation remains limited. Despite decades of research on public engagement in general and the role of the public in research in particular, there is little insight into the role of CSOs in research, their motivations, the factors that promote or inhibit it and the ways it can be assessed and evaluated.
Within the CONSIDER project (Civil Society Organisations in Designing Research Governance) we have focused on collaborative research projects, especially EU-funded multi-stakeholder projects (such as FP7 and Horizon 2020). We focused on current collaboration practices, observing different research teams in a real time context.
CONSIDER has involved a conceptually sound and empirically rich investigation of the role of CSOs in research.
The CONSIDER findings have been informed by:
• A conceptual clarification of the role of CSOs in research
• A survey of all EU Framework 7 (FP7) research projects
• An in-depth investigation of 33 cases of research projects including CSOs
• Expert and stakeholder perspectives obtained through a set of workshops and community-related exercises, and the development of a network of associates.
Our findings show that approximately one in four EU-funded projects (under FP7 in 2012) collaborated with at least one CSO. The added value provided by CSOs was about improving the societal relevance, enhancing policy impact, or representing the interests of a given societal group (such as patient groups, children etc.). However, CSO participation is not on the radar of some researchers.
The CONSIDER recommendations focus on the following key roles:
• Researchers
• Civil society representatives
• Policymakers
• Funders
There are a wide variety of reasons for including CSOs in research:
• To better align research with societal challenges
• To provide an opportunity for participatory research with the relevant communities of interest
• To improve the translation of research results back into society, including through innovation
• To improve the research (and its benefits) through the input of citizens’ expertise and context
Despite these many advantages, our research has clearly shown that CSO participation is no panacea and not necessarily recommended in all cases. In order for CSO participation to be successful, the different project partners need to understand each other and be clear about the purposes that such engagement is meant to achieve.
Further information regarding the project’s findings, including details of all of the recommendations, further specific case studies, quotes and examples, is available online at www.consider-project.eu/guidelines-landing-page

Project Context and Objectives:
Project Context and Main Objectives
Including Civil Society Organisations in research can offer many advantages, including higher acceptance of research outcomes and better quality of research findings. Despite the potential importance of CSOs, little was known until recently about the practice of including them in research projects. The CONSIDER project has addressed this gap through theoretically sound and empirically rich research which included a survey of all FP7 projects and more than 30 in-depth case studies.
For many policymakers, civil society engagement in research and innovation is about improving public support for research. For many scientists, on the other hand, it’s more about improving the research itself. For many CSOs both aspects are equally important. Against this background, conflicts fuelled by different expectations are quite normal. Therefore, the reasons for desiring CSO involvement may not be consistent with the mechanisms for promoting, implementing and evaluating it. Relating those motivations and mechanisms to the experiences of collaborative research participants, CONSIDER has identified a set of important obstacles and enablers that could be used in developing a more informed approach for integrating CSOs in European research. Detailed guidelines for policymakers, funders, CSOs and researchers are available on the CONSIDER website.

CSO participation is not the "one best way" to do research; it is useful in some settings but not in others. But when it is desirable, additional effort will be needed to facilitate effective cooperation among project partners.
CONSIDER’s comprehensive survey of collaborative research projects in the EU’s Seventh Framework Programme (FP7) found that a significant percentage of such projects involves participation by civil society organizations.
One reason frequently cited in favour of CSO involvement in research is that it can help “democratize science”, giving citizens a voice. In making this argument, however, it is important to be aware that CSO members are generally not lay citizens; they tend to be skilled and educated and may have research experience.
CSO involvement can be seen as one way of improving public engagement in research. In addition to the inclusion of professional CSOs as agents of civil society, many research projects involve citizens in specific actions (such as consensus conferences), few include individual citizens not organized in a collective group. (Examples are projects dealing with citizen science, the social sector and the arts). It is very difficult to involve citizens directly in EU projects. Therefore, CSOs can provide direct access to citizens’ views, and in many cases act as a skilled mediator between the research team and citizens’ inputs. The majority of EU-funded research projects with CSO participation feature CSOs that address specific interests such as patients, industry, agriculture, fishing, etc. Hence, CSO involvement in research tends to be delegative or representative rather than direct. There is evidence that they contribute to giving citizens a voice, thus deepening democratic involvement; however they can represent relatively narrow interests. Multiple perspectives and further changes to the structure and involvement of CSOs in research are required in order to achieve more general democratic aims.
Moreover, given the high degree of professionalization required to effectively take part in EU policy processes, there is a clear bias towards highly professionalized CSOs. CONSIDER’s research shows that the dominant mode of participation in FP7 research projects privileges an institutionalised type of civil society and supports the development of such CSOs. This creates a certain dilemma because professionalization limits the bottom-up character of grass-roots activists, including movements in opposition to public authorities, which are essential features of civil society if it is to fulfil a legitimising and communicative role. Nevertheless, CSOs are able to inject debate and creativity into research and thereby fulfil the role of critical voice and change agent.
Work Performed and Main Results
CONSIDER conducted the most thorough investigation to date into practices surrounding CSO participation in European research. Virtually all FP7 research projects were contacted and asked to indicate their involvement of CSOs. Projects reporting CSO involvement were contacted a second time and asked for more detailed information concerning organization of their research and mechanisms of CSO inclusion. The CONSIDER partners then conducted 33 case studies of collaborative research projects with CSO involvement, 20 of which were EU-funded.
During the course of the case studies, 107 interviews were carried out with relevant actors, which included project coordinators, CSO representatives, academic researchers and funders. Project outputs (documentation and dissemination) were also rigorously examined. Grounded Theory Methodology (which means that no analytical categories are developed before analysing empirical materials) was employed for the case study analysis. Collected data were shared amongst the project team and initially analysed for key themes using Nvivo software. Further data analysis was performed using Multiple Component Analysis (MCA) applied in two rounds: the first identified different modes of CSO participation in the given research projects; the second round identified difficulties faced by collaborative research efforts involving CSOs.
Additionally, CONSIDER collected highly valuable qualitative input through 10 expert workshops organized with relevant stakeholder communities throughout Europe. The CSOs involved in these workshops represented a variety of interests (patient groups, parents’ groups, environmental protection, etc.) and were joined at various stages by policymakers, researchers, funders and policy administrators. The workshops thus provided a rare opportunity for facilitated open dialogue between stakeholders involved in collaborative research. The first six workshops focused on best practices and testing CONSIDER’s key assumptions. The final workshops contributed to the drafting and revision of CONSIDER’s guidelines aimed at efficiently and effectively engaging civil society in research and innovation.
Frequently reported difficulties identified by CONSIDER include:
• Organisational challenges such as bureaucracy, lack of funding, and time and human resources constraints.
• Conflicting perceptions of project objectives (leading to disputes that often go unresolved)
• Low CSO status in consortium hierarchies
• Insufficient coordination and lack of clarity in task division
• Divergent views regarding methodology not only among CSOs and academic researchers but also among academic researchers themselves (interdisciplinarity)
• Clash of commercial and scientific interests
• Language barriers
• Concerns among academic researchers that CSO participation may weaken scientific legitimacy of research projects, jeopardizing potential to enhance academic reputations
Bureaucracy, lack of funding and constraints on time and human resources are the most significant barriers to CSO involvement in research. These barriers hinder participation especially by smaller organizations, which rarely have dedicated staff to deal solely with administrative tasks.
The role CSOs play in a given research consortium is a key parameter for understanding potential for conflict. The more negligible their role is (low recognition, non-equal status, little influence on agenda setting, late stage input, etc.) the more likely it is that conflicts will occur. Conflicted projects with CSO participation typically include only one CSO whose involvement is not regarded as strategic. Often the CSO contribution is planned for the end of the project and the CSO “value added” tends not to be recognized. Even in cases where CSOs are involved from the beginning, the structure of the project may exclude them from internal decision-making and limit their capacity to affect outcomes.
Final Results and Potential Impacts
CONSIDER’s empirical findings suggest that two factors (or dimensions) are decisive in determining a CSO’s position within collaborative research projects: social interaction and knowledge production.
Social interaction in this context refers to the extent of CSO engagement in a given research project. In projects “driven” by CSO involvement, CSOs interact intensively with other consortium partners. At the other end of the scale, a CSO’s position is “distant” from the core of the project, implying a minimum degree of social interaction. Between these extremes are “balanced” arrangements in which the level of social interaction among CSOs and other project partners is largely equal.
Here are some examples of roles performed by CSOs in research projects according to the social interaction scheme:
● CSO Distant: A CSO may be a member of an advisory board, a subcontractor or a research object; its input to the project is limited. Therefore the interactions between the consortium and the CSOs are very limited.
● CSO Balanced: A CSO may be a project member, a work package leader or an initiator of the project. Its influence on the project is balanced compared to the influence of the researchers involved. If this is the case, CSOs were normally involved in the main decision-making processes and take over roles of researchers.
● CSO Driven: A CSO assumes a dominant position, leading a research project. In addition to the activities mentioned above, the CSOs prepare the decision-making processes and are vested with a strong veto-position.
Meanwhile, in terms of knowledge production, CSOs can have either a “focused” or a “transformative” role in a research project, defined as follows:
● Focused: A CSO’s role in the project is specified to a well-defined task in the process of knowledge-production and dissemination, but they have no access to the core of knowledge-production. Such a focused role can be to improve the outreach and/or contextualize the project results based on the CSOs’ practical experience.
● Transformative: A CSO’s involvement leads in practice to a situation of co-research. This can be designed to achieve specific research goals, identify blind spots or define the research problem. The CSO may also inform the development of the methodology.
Combining these variables yields a set of six possible project types:

Figure 1: CONSIDER Model of CSO Involvement in Research
Each of the six project types identified in the table above has its own specific governance challenges. The CONSIDER Guidelines describe these challenges and offer detailed suggestions for addressing them. The intention is to provide relevant stakeholders with empirically defined project types and corresponding governance models that will be useful in designing future research calls that anticipate CSO participation.
On the basis of theoretical and empirical insights, the CONSIDER consortium developed a set of guidelines for key stakeholders of CSO involvement in research (see website). Some of the highlight guidelines are:
● Policymakers: Rethink scientific excellence to accommodate CSO participation.
● Funders: Give CSOs a voice in shaping the research agenda.
● Researchers: Get down to earth and involve CSOs.
● CSOs: Dare to take the initiative and reach out to researchers.
● Evaluators and reviewers: Consider the public relevance of the project

Impact
The CONSIDER guidelines were disseminated via dedicated dissemination documents (policy document, stakeholder-specific guidelines) and other media, including an animated video. These were disseminated broadly, in particular to related projects and interested stakeholders.
Importantly, the question of CSO involvement in research and innovation, which is a key component of the European agenda on responsible research and innovation, retains high policy interest. The project findings will feed into a follow-on study that will undertake a social network analysis and agent-based modelling of policy intervention which will be fed back to the European Commission.
Overall, CONSIDER provides a cornerstone of any future research policy concerning CSO involvement and research. This will lead to better quality of research as well as higher legitimacy of research outcomes.

Project Results:
Main Results
Engaging civil society actors in research is an integral aspect of the broader agenda of promoting responsible research and innovation in Europe. Integration of Civil Society Organisations (CSOs) in research projects is one way of achieving the aim of ensuring broader participation in research. This is an aim that is widely promoted by research policy makers and funders. Despite the broadly positive attitude to such CSO involvement, there was little knowledge of the practice of CSO involvement prior to the CONSIDER project.
The CONSIDER project was the first project that undertook a comprehensive study of the role that CSOs can have in research projects. To this end it engaged in the following activities:
• Conceptual clarification of the role of CSOs in research
• Survey of all FP7 research projects to ascertain the role of CSOs
• In-depth case studies of more than 30 research projects including CSOs
• Expert and stakeholder perspectives obtained through a set of workshops and community-related exercises, and the development of a network of associates.
In this document the main activities, results and outcomes of the project are described. The description is organised according to the internal logic of the project as expressed in its work packages.
Civil Society Organisation Participation in Research Governance
The first work package provided a theoretical means to structure the problem area and inform the subsequent analysis within the project. The first task was to reflect on the methodological underpinnings of the project. This was achieved via the creation of a glossary as a means to establish a common set of terms and concepts for the project, permitting the definition of key terms and concepts to be thought of as critical by each partner and allowing efficient communication between partners. The glossary is an open document and contains 29 concept or notion definitions written by different consortium members. It gives an overview of the main issues in debate inside the CONSIDER team and establishes a common set of terms and concepts. Some of the central issues that are dealt with are the following:
- CSOs scope: are they not for profit organizations, or should that definition include profit organizations like industries as well?
- Power: is this notion directly and uniquely bound to state exercise, or also related to other contexts such as firms?
- Research project: is it a planned and ‘one way to do research’ or a step by step development adjusting permanently to the context it is embedded in?
- Methods and models: how do we intertwine the grounded theory and the normative grid of analysis?
Those different discussions and conceptual connotations highlight the possible blind spots the CONSIDER team needed to be aware of and which are tackled along the way. This document is designed for CONSIDER consortium internal use.
The second activity was to undertake a theoretical review. Against the background of a description of some crucial historical antecedents, this theoretical account developed by the University of Namur in deliverable D1.2 permits the analysis of the institutionalisation of CSO involvement, via a conceptualisation of contemporary politics. Deliverable D1.2 enables addressing one of the core questions concerning CSO involvement in research design, going beyond the mere description of CSOs in research governance: how CSO engagement in research can be evaluated. Without a normative, policy-aware and analytic grounding, any means of assessment will risk being partial, ad hoc and framed by unacknowledged presuppositions. This deliverable begins to address this question by setting the groundwork for the grid of analysis, developed throughout WP1. This grid emerges from theoretical analysis and frames later research through focussing it on our research question.
It was initially explored how best to account, on normative grounds, for the role of civil society in new modes of research governance. A normative approach is crucial, as by such a means we can see the structures that underlie various approaches to CSO involvement. This has involved a kind of tour of the theoretical landscape. The point is to focus specifically on the role of civil society and to take into account the tensions and complementarities between participatory approaches and deliberative approaches and consider the institutionalisation of CSO participation.
The theoretical analysis has:
- provided a historical review of the dynamics that have made very salient both the role of civil society and legitimacy concerns in research and research governance; this is vital so as to be able to locate CONSIDER research in a relevant vein;
- dealt with this issue of how to conceptualise politics today – the necessity to acknowledge the opening process and the necessity of analysing its theoretical foundation – in short what is the actual theoretical grounding regarding civil society organisation participation and what are salient criteria to assess participatory experiences;
- determined the characteristics of our object of analysis and justified the relationship between conceptual approaches and empirical field study;
- created the groundwork for a grid of analysis.
The analytical grid is the result of an analysis of the theoretical background to civil society participation in research design. It is a distillation from more detailed research into, and critical analysis of, underlying themes in policy, history, society and philosophy as they appear in the (European) drive for participation in research. The grid permits a principled study of relevant cases and grounds tools of assessment that can inform policy design.
Essentially, in applying an analytical grid to the field of CSO participation in research design, one can shine a light on the (perhaps unstated) assumptions that underpin any given position. Using an analytical grid is like being a reverse engineer, or acquiring an object and deriving its design on the basis of its appearance, rather than designing from scratch an object with a purpose. In so doing, one can gain insights into the intentions that shape particular forms, which can then feature in detailed evaluations of those very forms.
The key steps taken towards the development and implementation of an analytical grid are:
1) Determining the domain for analysis
2) Specifying the parameters relevant to study within the domain
3) Creating a grid for analysis for our empirical findings by taking into consideration
a) Delimiting the field of inquiry
b) Defining the research problem
The grid of analysis was central to beginning to take these steps and delimiting the research in general in order to create coherent research as well as a strong foundation on which to base CONSIDER’s ultimate means of assessment for participatory practice, and for policy advice.
Action Implementation
Overall research problem Delimit field to a question: how do actors define and reach their expectations related to defining public interest when constructing norms in research projects? Apply the question to the overall problematic to yield pertinent areas for study
Theoretical background
(Including assessment of other applied research) Give content to question Analysis of theoretical backgrounds to pertinent areas of study, e.g. Aggregative democracy, Deliberative democracy, Governance models, actor selection, Participatory Approach aims
Grid
(Parameter determination) A derivation from the above two areas A table of pertinent areas of study that can inform field analysis to yield insights related to the question; also acts as a background against which to choose case studies
Table 1: Method illustration

The grid of analysis was deduced from a set of concepts most relevant to the research question. The research question represents a focus on a determined field within an overall problem area. The subsequent analysis in Deliverable D1.3 demonstrates that each of these parameters, criteria or concepts is extremely diverse. Whilst they are concepts frequently called upon by participants in research projects, there is no consistent understanding as to what their significance is.
These notions can serve as parameters for a grid of analysis, because, with respect to the research question, they are pervasive and their presence, absence and construal represent all the possibilities for addressing the question in terms of the analysis undertaken. They appear here relatively unexplained – their justification comes in the argumentation and reasoning that constitute the bulk of this deliverable.
How do actors define and reach their expectations related to defining public interest when constructing norms in research projects?
Norms & Values What norms? Whose? What values? Whose? Presupposed, ignored, excluded, constructed?
Expectations Of researchers Of CSO participants Of funders / and other stakeholders
Governance approach Hierarchical, consultation, co-construction? Aggregative, deliberative, dialogical?
Public interest Cui bono? (who benefits?) How is it progress rather than simple sectoral advance? Capacitation
Means of expressing interests Mode of participation?
Dialogue? Round table, focus group, questionnaire? Impact: when are the means deployed – start, during, end, throughout? Open ended or discrete? Conflict resolution mechanism?
Research and its background Funding source, aims, intentions? Political context (widely construed). CSO involvement for what?
Table 2: Grid of Analysis
Having identified the problem of our project, the problem becomes how to select the parameters. Inductive methods can’t easily reveal their own presuppositions. One methodology can reveal differences in outcomes when contrasted with another. A moment of judgement is required as to which is better, and with respect to what. This judgement call draws upon material not within the research methodology itself. We can't choose the parameters based purely on the fact that we have found them either in literature, assessment of prior practice, nor in our empirical work via induction. Instead, we try to determine what we are looking for through emphasizing the role of construction – we need a criterion or criteria of choice among the potentially unbounded parameters that will be available. We have to use a criterion of choice grounded in our wide appreciation of the overall objective of research we want to reach.
The final activity of WP focused on determining governance models. Its main deliverable D1.4 Governance Models outlined the theoretical and methodological background of the CONSIDER project, especially in terms of how they are related and how they will be applied to the analysis of the case studies, reflecting a combination of a normatively grounded, ‘deductive’ approach on the one hand (cf. Analytical Grid), and a more empirical, inductive approach (mainly related to the Grounded Theory Methodology) on the other.
In sum, through determining the limits of the current approaches with regard to CSO participation in the actual (especially European) political and research, via the characterization of patterns and models, the theoretical aspect of the CONSIDER project paved the way to overcome the common problem that in thinking about CSO participation, the conditions for the effective application in context are usually ignored.
The practice of CSO participation
The second work package on the practice of CSO participation analysed empirical data collected in the light of the theoretical background presented above and according to the methodology definition and observation tools defined in deliverable 2.1. The data collection strategy was focused on primary data through a questionnaire sent by email to all FP7 project coordinators present in the CORDA database on the first of March 2012. In addition the WP has undertaking in-depth case studies of 33 research projects involving CSOs to understand the factors influencing the success of CSO involvement in research.
This WP was thus central in CONSIDER as it produced the data needed about CSO participation in real practice. The data collection strategy provided rigorous and up-to-date data, thus enabling the analysis of the CSO participation landscape. Furthermore, it also contributed to our understanding of research projects, while giving insights into actor’s practices through observation and analysis of project practice, thus going beyond discourse analysis. It enabled the consortium to find out how participants communicate and view implicit assumptions and normative issues of CSO participation.
WP2 was central to the project achievements. It aimed at gathering the main data needed to build our research governance model (WP 3) and recommendations (WP 4).
The initial step of the empirical investigation was to develop the principles of data collection based on collaboration of the theoretical reflection undertaken in the first step. It suggested using relevant parameters as the starting point for a theoretical reflection and for an empirical exploration of research projects including CSOs. One important aspect of this work was to develop the foundations of a case study protocol. Making use of the results of survey 1, the project team developed variables and questions. Most of these questions became part of survey 2. Others formed the first interview questionnaire for our qualitative data analysis. This questionnaire was applied to the pre-test cases. The reflection of the pre-test-experiences led to an improved questionnaire, to a data analysis strategy including the decision of using NVivo a qualitative data analysis software and to the main structure of the reports which were written on each of the analysed research projects. Moreover, in cooperation with the project team UL decided on the relevant FP7 cases which came out from survey 2.
The second task in exploring the role of CSOs in research was to explore on a high level the overall rate of CSO participation. The strategic objective of this task was to gather data through questionnaire analysis of all FP 7 projects referenced in the European Commission Database “CORDA” (which is the database underlying the EU CORDIS service). The first wave of the initial survey mailing started on the 12th of June 2012. The email answers we received were classified and matched with the original contacts. Then phone calls and reminders were sent. A second wave of mails was sent on the 27th of November 2012. The second wave of questionnaire was sent on 21th of December 2012.
For the first time the 17000 FP7 projects have been surveyed with regards to their involvement of CSOs. Initially only two questionnaires were planned. LU actually sent three different questionnaires. The first one was sent to 17032 FP7 project coordinators abed we received 2959 responses completed. Then the second and third questionnaires were sent to 492 Project coordinators and CSOs who worked together. CSOs appeared to be already active in 21% of the FP7 projects. We gathered information about CSOs roles, expectations, and achievements, as well as team work habits and tools. Those results were discussed and their validity tested internally and externally.
Informed by the initial results of the survey, the consortium could collectively define case study selection criteria. During a project meeting in Prague, we defined a strategy of case study selection of FP7 projects including CSOs and of other research projects including CSOS. LU constructed a matrix of all FP7 Projects identified through the first survey as CSOs inclusive, according to two main factors: the intensity of their collaboration and their leadership (CSO or scientific lead).
To select non FP7 projects LU also suggested to get an overview of existing participatory research practices and projects in different countries. From this overview LU, KIT, DMU and Euclid had to suggest between two and four non FP7 projects pertinent for our case study protocol (defined in task 2.1).
Prior to the collection of data from the selected case studies, the consortium went through a process of developing the principles of data collection, data analysis and collaboration in these steps. Each of the partners involved in the case studies (UL, KIT, EN, DMU) selected one research project that they were familiar with, outside of the set of cases for the main data collection as a pre-test or pilot test. This pre-test was used to test the data collection methods, develop an interview schedule, create a case study template and explore ways of collaboratively analysing the data using processes and principles of data analysis developed by the research partners.
One aspect of this work worth mentioning in detail is the agreement to use the qualitative data analysis software NVivo (server version 10) for data analysis. The consortium went through a process of exploring possible data analysis tools. It was found that NVivo had the most appropriate functionality. In addition it provides the possibility of working collaboratively. This was deemed to be important to ensure the consistency of data analysis and the possibility of having an immediate insight into the work of the partners. The consortium therefore decided to buy the NVivo 10 server version, which was not foreseen in the original budget plan but which is central for a quick and high quality data analysis.
The result of this work is a collection of 33 case studies (counting the pilot cases). The data collected in the context of the case study research includes more than 100 in-depth interviews as well as numerous further data from the 33 cases. In order to facilitate cross-case analysis and to communicate about the cases, each case was written up in a structured document following the same structure, thus allowing a detailed comparison of important aspects, such as barriers or enablers, roles of CSOs or the external environment. Due to the level of detail provided in these cases, an analysis of them could allow for the identification of respondents. The consortium had received ethical permission to undertake the research on condition of anonymity of respondents. The consortium can therefore not publish the case studies but has to rely on the publication of aggregate analysis drawing from all cases.
The analysis of the material was described as a separate task in the DoW but in practice constituted an integral part of the data collection and interpretation process. In line with the principles of Grounded Theory which emphasises the important of constant comparison between data and interpretation, the analysis of empirical findings formed part of the collaborative work of WP2. The consortium set up a detailed process including weekly meetings to ensure that data analysis was undertaken consistently across the four different partners involved in the empirical work (UL, EN, KIT, DMU).
The outcome of the analysis of the material was a selection of 33 case studies. In line with the allocation of time and effort to the different partners. The analysis was described in D2.3 and can be seen in the case studies. In addition the higher level analysis across the different cases resulted in the formulation of hypotheses about the role of CSOs in research and the identification of variables of relevance to CSO involvement. These insights contributed to the development of the model of CSO involvement as developed in the next step.
Model of CSO participation
The development of a model of CSO participation was the focus of WP 3 which functioned as a central point connecting WP 1, WP 2 and WP 4. Parallel to analysing the qualitative empirical materials which were collected for the case studies, the consortium reflected on how empirical insights could lead to a refinement of the theoretical groundings of CSO participation. These thoughts are reported in Deliverable 3.2 which attempts to overcome theoretical approaches of participation which merely reduce participation to auto-justification or a sort of alibi. This was possible because CONSIDER gained the empirical insights concerning the reasons for and processes of CSO participation that revealed that CSO involvement is much richer and more complex than previously described in most of the literature. However, significant efforts were needed to structure these insights in such a way that we could connect the very broad empirical experiences: The consortium’s empirical team held three Model Building workshops between December 2013 and March 2014 (Brussels, Brussels and Leicester). There, the consortium defined several steps of finding its path to model-building and deducing mixed ideal-real types of CSO participation in research. The outcome was a model which contained empirically and theoretically based variables of CSO participation and linked these variables in the context of an overarching process related to three main dimensions social context, project governance and research results. Further, by applying different heuristics and performing various comparison activities, the project was able to define six different project types where all 33 cases analysed could be classified accordingly. Each of the types contains a specific governance challenge which the classified projects needed to react to. Coming from this, we could compare the different project strategies to master this challenge which in the end allowed us to develop specific recommendations accordingly.
The starting point of the development of the model was the integration of the analytical grid developed in WP 1 and the empirical insights from WP2. This is to say that the first step was to synthesise the normative and the empirical-analytical elements of the project up to that stage in order to arrive at a soundly-based picture of the present state of the art regarding CSO involvement.
The results of work undertaken in this first task (3.1) are presented in Deliverable D3.1. Significant aspects of this deliverable are definition of a set of objectives, hypotheses and parameters that have informed the choice of case studies as well as the case study protocol including interview schedule and data analysis approach.
The hypotheses that allowed for a focused engagement in the research are:
1. There is a variety of practices of CSO participation in research governance.
2. The participation of CSOs in research is embedded in a set of assumptions and procedures which affect the achievement of internal or external expectations.
3. Research governance of CSOs participation in research has to accommodate tensions between e.g.:
a. Public interest
b. Research
c. Policy / politics / governance
d. External and internal expectations
The main parameters which were broken down into significantly more detail that is described in D3.1 are:
1. Actors
2. Impact, outcome of the project
3. Research project
4. Conflict resolution
5. Practice of participation
6. Evaluation
7. Context
Deliverable3.2 the Report on the analysis of theory and practice of CSO participation in research governance was completed in January 2014. It built on WP1 and WP2 in order to develop a model of CSO participation in research. The task explored the way in which theory and practice of CSO participation in research relate to one another.
The deliverable built on deliverable 3.1 which started to synthesise the normative approach and the empirical-analytical data analysis of the project in order to develop an understanding and insight into CSO involvement in research governance. The deliverable demonstrated this by analysing empirical data that had thus far been collected. This included data from some case studies and the survey which was then interpreted it into the Analytical Grid highlighted in deliverable 1.4. The task assessed and identified gaps and limits in current CSO governance theory and used the Analytical Grid as a diagnostic tool and the grounding for the development of a model of CSO participation in research. The model involved critical variables identified in WP1 and used these to analyse the empirical results from WP2, with the aim of identifying matches and mismatches between theory and (CSO) participatory practice.
The consortium then started working on model building by discussing empirical results and perspectives of data interpretation. After having analysed most cases in autumn 2013, the project held a first model meeting in Brussels in December, where it defined the path to model building. It was necessary to determine a common idea of how to reach a model of CSO participation and define the necessary steps. The consortium began developing hypotheses which were implicitly inscribed in the codes and categories which were used to analyse our empirical data. These 115 hypotheses were discussed and we assigned them to all analysed cases to better understand their empirical validity. Then, at the second model meeting which took place in Brussels in the beginning of February 2014, the consortium discussed the structure of hypotheses and developed a hierarchy of top-level and sub-level hypotheses and by this determined the relevant variables which were deemed to be important for the governance of research projects involving CSOs. Following, these variables and their sub-variables were set in an order which led to the creation of so-called ID cards that contain all empirically proven relevant variables. To make it more precise: The ID cards contain all variables and sub-variables as well as possible shapes of these variables, e.g. the top-level variable, ‘construction of CSO’, with the sub-level variable ‘self-definition of CSO’ which was distinguished between the values ‘not-for-profit’, ‘independent’, ‘non-commercial’ and ‘don’t do research’. The ID-cards were finalized at the third model building meeting in Leicester and then applied to all our case studies. The variables inside the ID-card were then put in interrelation to each other leading to the ‘interrelation model of CSO participation in research projects’.
It was now possible to group all cases in accordance to selected variables, sub-variables or values and report the empirical insights of these actions of grouping and comparison of cases to the model structure. Doing this, the consortium repeatedly came across two relevant dimensions with their typical shapes: One is the ‘social interaction scheme’: This dimension describes the forms of cooperation within the project which can be differentiated between three variants of cooperation: CSO-driven (CSO is main actor and driver, e.g. CSO is PC), CSO-balanced (CSO has a clear voice in the project, e.g. CSO member of consortium) and CSO-distant (CSO has only limited influence, e.g. CSO no clear role / negligible). The other dimension is the ‘role of CSO in knowledge production scheme’: This describes the importance of the CSO in relation to the knowledge production-process in the project. The distinction is between focused importance (only one / two aspects named related to the CSO action) and transformative importance (broad range of activities with important impact of the whole knowledge production and application process and a broad scope of beneficiaries). By crossing these two dimensions we received six typical project types. Depending on the project type, we recognized that different challenges needed to be mastered accordingly. The following table summarizes the dimensions, project types, the allocation of our case studies and the individual governance challenges.

Role of CSO in Knowledge Production Scheme Social Interaction Scheme
CSO-distant CSO-balanced CSO-driven
Focused importance Transformative importance Focused importance Transformative importance Focused importance Transformative importance
Projects KIT E, KIT G, KIT M
DMU B, DMU F
UL D, UL E, UL B KIT J,
DMU H
KIT B KIT H
EN A, EN C
UL G,
DMU D, DMU J KIT F
EN B,
UL C, UL J, DMU E KIT C, KIT N, KIT B
UL I
DMU O
EN D KIT A, DMU C, UL K
Research-Governance Scheme Peripheral-marginal Peripheral-dominant Cooperative-restrictive Cooperative-inclusive Community-related Community-Based
Specific governing challenges Keeping the inclusion of the CSO facing their perceived limited importance Integration of CSOs while limiting their influence on the scientific agenda
(in cases of highly motivated and very influential CSO) Integration of CSOs in limited fields of engagement within a project Organising a working structure to bridge the distinct logics of professional orientation in the knowledge production process Allowing a CSO to drive the progress of a project but limiting its performance possibilities with regards to all kinds of roles possible Creating a working structure for cooperation for facing the non-academic boundary-conditions for knowledge-production
Table 3: Model of CSO engagement including interaction schemes and ideal types
Based on these insights, the consortium could deduce specific recommendation that should be reflected by everyone involved into a project setting with CSOs (link to WP 4).
According to the description above several significant results were achieved:
• Development of an innovative methodological approach that allows to capture and analyse the complex data structure of a multi-case-study approach
• Development of a list of hypotheses which can be used for all future research on participatory research projects
• Development of ID-cards containing all variables that seem to driving forces in the governance of participatory research
• Development of the ‘Interrelation Model’ containing a structured approach to all variables that seem to be driving forces in the governance of participatory research
• Development of six empirically based types of participatory projects
• Development of recommendations in accordance to the project types
Guidelines
On the basis of the model of CSO engagement
The Guidelines are the main practical outcome of the project. They condense the insights gathered by the consortium over the course of the project into a set of accessible and practical suggestions for individuals and groups involved in CSO engagement in research.
Developing the guidelines was a long and iterative process which to a significant degree ran in parallel to the work undertaken in WPs 1, 2 and 3 and that fed into WP5. Development of guidelines and feedback on various drafts and versions led to the version presented at the final event in January 2015.
It is recognised that these guidelines are subject to further development after the end of the project and can only be seen as a snapshot subject to better understanding and further experience. For this reason it was decided to host the final version of the guidelines on the project website which not only makes them more broadly accessible, but also ensures that they can be commented on and further developed. The latest version of the guidelines is available at:
www.consider-project.eu/guidelines-landing-page
Prior to the development of the actual guidelines the project considered the requirements for such guidelines. The requirements for guidelines depend on the specific needs of different stakeholders, notably CSOs, researchers and research institutions and policymakers. In order to capture these requirements the consortium decided to use the in-depth interaction with these stakeholders during the case study data collection to specify stakeholder-specific requirements.
The main result of the task was an agreement on a methodology of eliciting requirements for guidelines. Instead of developing guidelines on the basis of the consortium’s understanding of potential users’ needs, it was decided to explicitly include such potential users in the development of the guidelines. This agreement on a methodology will allow for the further development of the requirements and of the actual guidelines themselves in a parallel and iterative way.
Results from surveys, workshops, case studies and theoretical analysis were used to identify priorities for Guidelines being developed for separate stakeholders (especially researchers, policymakers/funding bodies and civil society organizations).
Working hand in hand with task 4.1 the development of guidelines resulted from a synthesis of work carried out in the project. Guidelines were developed on the basis of the conceptual and empirical insights of the project. This included analysing results from surveys, workshops, case studies and theoretical analysis that resulted in the development of specific guidelines as outlined in D.4.1. The general methodology employed in developing the guidelines as well as their formulation at the end of the project in January 2015 are described in deliverable 4.1 Guidelines Handbook.
The result of this process is a set of guidelines for five main stakeholder groups:
• Researchers
• CSOs
• Policymakers
• Funders
• Evaluators
For each of these stakeholder groups there is a brief overview of their role and motivation as well as a set of specific guidelines. Each of these guidelines is structured into an explanation of the guideline’s principles, an indicative example and a quote.
The development of the guidelines was undertaken iteratively with activities aimed at gathering the feedback from particular user groups. In July 2014, CONSIDER held its last three expert workshops in Brussels. These meetings were dedicated to the revision of the set of guidelines and recommendations developed by the CONSIDER consortium for all relevant stakeholders to efficiently and effectively engage civil society in research & innovation (researchers, policy-makers and funders, civil society organisations).
Holding these three consultation workshops has proved very fruitful and informative. It allowed all project partners to ensure that the guidelines are practical from the point of view of potential users. By holding these workshops during the second half of the development of the guidelines document, the consortium wanted to maximise the effectiveness of the specialist input into the discussions, but allowing time for the modification and development of the guidelines according to the feedback received.
Overall, the document was received positively and judged to be extremely valuable by all the participants. During the three workshops, very useful input was provided by the participants allowing the consortium to adapt the document to the needs of the potential end-users.
An overall criticism was that the initial document lacked actionable guidance and must be clear in its limitations and focus. Further, as empirical knowledge is the key selling point of CONSIDER, it was also outline that the document should include key figures, quotes, statistics and diagrams to engage the audience. In terms of content, we were able to get feedback on concrete points to improve and clarifications to provide (terminology used and changes of phrasing). We were encouraged to stress certain points such as the various reasons that can motivate CSO involvement in research projects for both the researchers and the CSOs themselves. Further, it was made clear that the final document should list the practical steps allowing such collaboration and explicitly state the administrative procedures linked to research projects, especially EU-funded projects. It should also include an approximate evaluation of the time and personnel needed to collaborate on such initiatives.
In terms of its presentation, it was agreed that both printed brochures and an electronic version, such as a web-platform, would strengthen impact. Currently the electronic version of the recommendations, a dedicated website, is being developed by the consortium. Thanks to these three expert workshops, we were able to identify a limited number of key recommendations per category of stakeholder that were the most relevant and action-oriented ones.
The results of this task fed into the deliverable 4.1 and are available in deliverable 4.2 Expert workshop proceedings.
The guidelines are available at the following URL:
http://www.consider-project.eu/guidelines-landing-page

Potential Impact:
Potential Impact
Wider societal Implications
The purpose of the project was to contribute to a better understanding of CSO involvement in research. This topic, while always of relevance, turned out to gain in importance over the lifetime of the project. The rise of the EU agenda on responsible research and innovation (RRI) towards the end of FP7 and its integration as a cross-cutting activity in H2020 together with its significant budget line meant that the component activities of RRI have gained considerably in terms of visibility and importance. One key aspect of RRI and arguably the most prominent in terms of promotion by the EU is public engagement. While there are numerous ways of implementing public engagement, one way of doing so that is particularly suitable for European-funded research is to integrate CSOs into research projects. It is therefore to be expected that future research policy will seek ways of improving such processes and the CONSIDER findings and guidelines are geared towards supporting such policy development.
There are numerous indicators of likely policy impact that the CONSIDER work will have. On the one hand, its final even was well attended and policymakers including the President of the European Economic and Social Council as well as the Director of the Joint Research Centres publicly stated their support for the integration of CSOs in research. At the same time CONSIDER was well connected with related projects which explicitly aim to anchor public engagement and RRI in current policy, notably the Engage 2020 and RRI-TOOLs project.
Finally, it is important that the EC developed a tender project that, largely based on the CONSIDER outcomes, will undertake a social network analysis of FP6 and FP7 and then engage in agent-based modelling to identify policy interventions. This project will feed into the mid-term review of H2020 and point the way towards policy in the post-H2020 research programme.
The CONSIDER project consortium understands that social and policy outcomes need to be supported by targeted dissemination activities to ensure that the knowledge gained can be made available to individuals and groups in a position to make decisions. The extensive dissemination and exploitation activities are described in the next section.
Dissemination and Exploitation
The study and engagement of all the CONSIDER audiences (most importantly CSOs, academics and policy-makers) was a key focus of the project dissemination from the beginning. CONSIDER aimed to be a reflective project that applied principles of participation to its own work. Therefore, the consortium involved these stakeholders in the development of research questions, deliverables and the general inputs on the state of play of CSO involvement in research.

Several dissemination initiatives were delivered, with the objective of reaching out to all of the project’s audiences through several targeted activities, which included:
- Published materials (Brochures, posters, flyers)
- Website and social networks
- Ten consultation workshops with CSOs, policy-makers, industry members and the academic community
- Videos
- Two Policy Briefs
- EU Presidency Events
- A Network of Associates
- Articles in non peer-reviewed magazines
- The EU Participant Portal dissemination activities
One of the key points of the dissemination strategy was to seek out for the permanence of the research findings and recommendations even after the completion of CONSIDER. Several dissemination outputs were deliberately designed as resources that could prevail and be a source of information for the ongoing consultation of CSOs, policy-makers and academics.
Key highlights within the dissemination activities included:
Final Guidelines brochure. An accessible and easy-to-read summary of the main recommendations of the project was prepared, and launched at the final event for the project. This brochure deliberately used the same design style and characters from the animated video in order to achieve consistency and connect the two elements

Website. During the second half of the project, the CONSIDER website (http://www.consider-project.eu/home) was redesigned and migrated to a Google sites application, in order to preserve all the research done and allow permanent access to it as a valuable source of consultation for the stakeholders after the project’s termination date.

Policy Briefs. The first policy brief was issued in M18 and provided an overview of preliminary findings and concepts of the project. The second policy brief was published in January 2015 (M36), and focused mainly on the findings of the extensive empirical research conducted by the consortium.

Consultation workshop events. CONSIDER organised several consultation workshops, including briefing sessions and dialogue panels, and were conducted with the following aims:
• To raise the profile of the CONSIDER project within key relevant stakeholder groups.
• To gather feedback and external input from those stakeholder groups on initial plans and findings, in order to ensure that the final documentation is relevant to and recognised by the people whom it will affect e.g. Commission staff, academic researchers, practitioners and CSO staff themselves.
• To ensure active and in-depth dissemination and discussion of the main project findings within the wider research and policy communities.

The workshop locations, focus and design were decided based on the following criteria:
• Achieving a geographical spread across Europe, close to key areas of relevant stakeholder groups.
• Covering a range of purposes: information seeking, testing research tools, dissemination and feedback on findings, or testing the guidelines and recommendations.
• Accessing a variety of event types (ranging from existing conferences to bespoke policy-linked working groups).
• Involving all the different CONSIDER target audiences.

A total of 10 workshops (one of them a parallel session) were held during the project. Further details regarding each of these workshops, including a dedicated report outlining the workshop content, participants, focus of discussions and conclusions, are provided at http://www.consider-project.eu/workshops.

Copenhagen (M27) Prague (M14) Lille (M20)

Copenhagen (M27) Brussels (M30) Prague (M14)

Animated video. The CONSIDER animated video is intended to ensure that the CONSIDER results are presented to stakeholders in a way that captures the attention of policymakers, funders, researchers and civil society. The aims of the animated video are to:
• Disseminate key findings from the CONSIDER project within the video itself.
• Generate interest amongst relevant groups to look further into our findings.
• Give the project a legacy and provide inspiration for the various EU-funded Responsible Research and Innovation (RRI) projects starting under the EU’s current Horizon2020 (H2020) framework programme.
We see the video being used in a number of different ways. For example it might be played at:
• The start of relevant conferences and meetings to set the scene for subsequent discussions,
• Incorporated into training sessions on ‘Responsible Research and Innovation’ run by other EU-funded projects or the EU Commission more broadly.
• Used within briefing sessions by the National Contact Points (NCPs) throughout Europe to assist in updating researchers at a national level
By making it available for download from the CONSIDER website as well as disseminating it through standard video channels (such as YouTube) and using a Creative Commons license we have also ensured it can be used as a stand-alone resource for individuals to access.
The video itself is available from the project homepage (http://www.consider-project.eu/home)
EU Presidency Events. The CONSIDER consortium sought to have a distinct presence in various relevant events, so as to promote its results and create awareness to civil society and to research community. In particular the consortium made an effort to engage with policymakers and key stakeholders participating in high profile international events regarding research and innovation policies.
In this respect the CSOs, the researchers and the authorities, like national and international funders, research administration authorities and policy-makers at EU and at national level were approached through the presentation of the CONSIDER project results in relevant conferences organised in the context of the Presidencies of the Council of the EU, with the aim to create a platform for discussion and comments on the project’s findings and their future exploitation and wide promotion.
Dissemination actions were undertaken in connection to events organized by the rotating Presidencies of the EU Council, between 2012 and 2014:
• Nicosia, Cyprus (September 2012)
• Vilnius, Lithuania (November 2013)
• Athens, Greece (June 2014)
• Rome, Italy (November 2014)
Network of Associates. The Network of Associates was developed in order to “inform the research, broaden the knowledge base, gather feedback, and facilitate dissemination activities”. The Network of Associates reached the target number of 100 people from a wide range of countries and sectors as was intended from the beginning of the project. However, during the progress of the project it became noticeable that the network became a relatively inactive platform despite the Consortium’s efforts to maintain momentum. Members preferred to use it for occasional information-exchange purposes rather than more regular multi-directional contact. As a result, after looking again at the members and overall functioning of the network under WP4, the Network of Associates was migrated from a Google group to a regular newsletter/mailing list, as a more inclusive way to reach out to more people in the manner that they found most useful. In this platform, any member could contact Euclid Network to add or suggest content to the newsletter, which proved a more effective way of managing the network.
The consortium then collated all of the Network of Associates members in the Google group along with all other CONSIDER contacts, into a single contact list (approximately 200 contacts) which became the project’s primary information dissemination list. After the final event, the second Policy Brief, the project Guidelines brochure, and the two videos were disseminated through the mailing list. Euclid Network, as a member of the consortium, will continue to manage this group after the end of the project as an RRI network for CSOs, thus giving CONSIDER a legacy that should enlarge the communication channels within the Commission.
CONSIDER final conference. The European Economic and Social Committee (EESC) and the CONSIDER project organised the conference “Civil society in Research and Innovation” as the final conference of the project. It was held at the EESC premises, in Brussels, on 28 January 2015.
Bringing together around 200 representatives from across Europe and beyond, the event aimed to stimulate debate between representatives of the main stakeholder groups involved in integrating civil society into research. Policymakers, civil society representatives and academic researchers discussed the recent work carried out by the EESC and the CONSIDER project, and looked at both current practices and potential future directions.

The Opening Session started with a speech by Mr Henri Malosse, President of the EESC and was followed by a video message by the Commissioner, Carlos Moedas, who highlighted that civil society involvement in Research and Innovation is not only a very important topic for him, but highly relevant for the priorities of his mandate as a Commissioner.

Carlos Moedas video message at CONSIDER’s final conference
(URL: http://www.eesc.europa.eu/?i=portal.en.videos&itemCode=34610)
Prof. Stahl spoke on behalf of the CONSIDER project. He presented some of the most important outcomes of the project, as well as the animated video.

Following the opening session, a series of panels were set up to discuss the conclusions of the project, as well as to highlight the possible pathways for CSO participation and RRI research in the near future. These panels were discussed by the project’s stakeholders: high-level EC members and policy-makers, recognised academics, and CSOs with relevant expertise.

Panel 1: Current state of CSO participation in R & I
• Mr Jef Ongena, President of the Belgian Physical Society
• Mr Lars Klüver, Engage2020 Project coordinator and Director of the Danish Board of Technology
• Mr Ulrich Schurr, Managing Director, Institute of Bio and Geosciences, Research Center Jülich
• Mr Stefan Bergh, Secretary General of the Swedish Cancer Society and President of Euclid Network
Chair: Mr Terry Martin, Journalist and Executive Director of the Science Policy Interface Agency

Panel 2: What are the future developments?
• Mr Herbert Reul, Member of the European Parliament
• Ms Hilary Sutcliffe, Director of MATTER
• Mr Jasper Roodenburg, NWO, Dutch research council
• Ms Annika Thies, Director of the Brussels Office, Helmholtz Association
Chair: Mr Erik Svensson, Member of the EESC

Closing Speech
• Mr André Martinuzzi, Head of the Institute for Managing Sustainability, WU Vienna University of Economics and Business
• Mr Martin Siecker, President of the Section for Single Market, Production and Consumption, EESC

List of Websites:
Web Address and Contact Details
www.consider-project.eu
Contact details

Coordinator
Professor Bernd Carsten Stahl, De Montfort University, Leicester, UK.
E-mail: bstahl@dmu.ac.uk
Tel: +44 116 207 8252

Project logo

Diagrams or photographs illustrating and promoting the work of the project
The CONSIDER consortium:

For further photos see website
Videos
Animated video giving an introduction to CSOs in research
https://www.youtube.com/watch?v=3AXBAzRnTdA
Presentation of European Commissioner, Carlos Moedas, on the occasion of the CONSIDER final event
http://www.eesc.europa.eu/?i=portal.en.videos&itemCode=34610
Overview video of CONSIDER project
https://www.youtube.com/watch?v=4Mrm0LcpM7I

List of all beneficiaries with the corresponding contact names
De Montfort University (DMU) - Leicester, United Kingdom; contact: Bernd Carsten Stahl
Signosis Sprl. (SIGNOSIS) - Brussels, Belgium; contact: Elena Tavlaki
University of Namur (UNamur) - Namur, Belgium; contact: Philippe Goujon
University of Lille (LU) - Lille, Fance; contact: Martine Revel
Karlsruhe Institute of Technology (KIT) - Karlsruhe, Germany; contact: Stefan Boeschen
Euclid Network (EN) - London, United Kingdom; contact: Stephen Barnett
University College London (UCL) - London, United Kingdom; contact: Karen Bultitude
Science-Policy Interface Agency (SPIA)- Berlin, Germany; contact: Terry Martin

Final Report Summary - CONSIDER (CIVIL SOCIETY ORGANISATIONS IN DESIGNING RESEARCH GOVERNANCE)

Related documents

Share this page

Download