Skip to main content
European Commission logo print header

How to achieve sustainable water ecosystems management connecting research, people and policy makers in Europe

Final Report Summary - AWARE (How to achieve sustainable water ecosystems management connecting research, people and policy makers in Europe)

Executive summary:

The project AWARE, funded by the European Commission (EC) under the Seventh Framework Programme (FP7), has developed and tested a new way of connecting scientists, citizens and policy makers to enhance the effectiveness of their participation and cooperation in the water science and policy assessment process. Connectivity is ensured by means of a knowledge brokerage concept applied at the European Union (EU) and local level. A pilot test of such knowledge brokerage concept has been successfully realised, focussing on three coastal areas of Europe (respectively the southern North Sea coast and the river basins of Seine, Somme and Scheldt, the Gulf of Riga in the Baltic Sea and the Goro Lagoon in the Po Delta area, in the Adriatic Sea).

To begin with, a European level workshop gathered all project partners including scientists, a panel of 30 randomly selected citizens from the pilot case study areas (10 citizens were recruited for each area) and various stakeholders and decision makers. In the first European workshop, participants gathered the basic scientific knowledge of the topic on focus, i.e. coastal water deterioration, and of the EU legislation aiming to ensure the good ecological status of coastal waters across Europe, namely the EU Water Framework Directive (WFD) and related policies. This initial workshop was then followed by workshops and public conferences organised in parallel in each case study region, enabling each group of citizens to catalyse the results of the knowledge brokerage, producing local assessments of the sustainable coastal water management topic in the form of 'Citizens declaration'. The local citizen declarations were then combined into one set of common recommendations (the AWARE 'European Citizens Declaration') during a final European workshop and presented at the AWARE European Conference held on 9 June 2011 in Brussels, at the European Economic and Social Committee, with the participation of European stakeholders and policy makers. The whole AWARE process has been evaluated and the evaluation results discussed with a restricted number of water managers and practitioners at the evaluation workshop held on 20 October 2011 in Brussels, at the Austrian Delegation to the EU.

Results and lessons learned from the project are summarised in the AWARE brochure available at http://www.aware-eu.net.

By mainstreaming the AWARE approach, a better multi-level governance of water policy issues can be achieved linking central and decentralised levels across Europe and ensuring a constant interface and brokerage of scientific and citizens/experiential knowledge at all levels. Indeed, as highlighted also by the AWARE experience, nowadays the dialogue between scientists, policy makers and the end users targeted by water policies works 'to say the least' unevenly across different countries and different levels of governance. One recommended solution is therefore to replicate the AWARE experience, engendering new informal process of consultation on key sustainability issues which can enable a more productive public-policy interaction.

In the water sector, such an awareness raising process could help to bridge the gap between the citizens' locally specific and experiential knowledge, perspective and understanding of the topics, that of the elected representatives and the more technical perspectives hold by the water managers and the experts involved in the water policy formulation process.

Project context and objectives:

Broadly speaking, the AWARE methodology engage panels of randomly selected citizens from all countries of Europe and all walks of life to make a critical evaluation of research goals, outcomes and management options, focussing on their societal acceptance. The evaluation is undertaken by means of brokerage activities among different forms of knowledge - i.e. between the different scientific disciplines needed to understand complex issues, the citizens' everyday life (non expert) knowledge, the different stakeholders' interests and the way decision makers tackle societal challenges. One of the key characteristics of the AWARE method is indeed linking together scientists and citizens early in the process in order to provide a common understanding of the issue at stake. Thus, lay citizens gain confidence to discuss the issues with other stakeholders and form independent opinions.

Bringing together scientists, citizens and decision makers in knowledge brokerage activities "by means of structured citizens' conference processes, where citizens' panels discuss and present their recommendations (Citizens' declarations) to policy makers and society at large" is at the heart of the proposed approach. This is a new way of connecting citizens, experts, stakeholders and policy makers in order to:

1. share a common understanding and awareness of the complexity of environmental and societal challenges
2. discuss how research and new innovative solutions may help to tackle those challenges now and in the medium-to-long term future
3. deliberate about how various research outcomes could or should be taken up by governments or citizens themselves, e.g. by incorporating sustainability into planning or adopting more environmentally-friendly behaviours, respectively.

The pilot experiences conducted so far show that the method works, delivering important benefits for all the actors involved:

1. the scientists learn to communicate results in a tailored manner to citizens, as well as policy makers and stakeholders. They also can broaden their research interests by accepting new inputs and discovering a public interest in their results, beyond the academic fora
2. the citizens quickly learn key environmental concepts, change their mind becoming aware of the complexity of the challenges ahead and reflect on how to tackle them with more systemic approaches. They may better assess which policies would be needed for solving complex problems and choose to support their politicians in tackling challenging decisions and policies, thanks to the better understanding and greater commitment gained throughout the process
3. the various stakeholders benefit from the insights and opinions of the scientists, citizens and policy community usually assessed in the more neutral context 'i.e. not heavily influenced by special interests' of 'citizens juries'
4. the policy makers 'through meeting the citizens and hearing their proposals' can share common visions on societal challenges that could not be solved with simple policies, would require systemic and long lasting actions to be implemented beyond electoral terms and a deeper consensus and commitment of all actors involved. Policy makers also gain further confirmation that decisions successfully involving all actors affected, through increased awareness for instance, are more effective in their implementation and outcomes.

The specific issue addressed in the AWARE project was the deterioration of coastal waters in Europe and how EU funded research and EU and local policies may help to reduce deterioration and achieve a good ecological status of waters by 2020. This environmental goal is a core objective of the EU WFD (EC, 2000).

This piece of EU legislation provides a coherent framework for the implementation of policies and the assessment of water quality across EU Member States, including river basins, transitional and coastal waters. The outcome expected from a consistent application of the EU water policy assessment process is the achievement of good environmental (ecological and chemical) status and the related benefits at a proportionate cost or the maintenance of the water body in a moderate quality status with reduced benefits, due to the disproportionate costs of achieving a better quality status. In this respect, the AWARE pilot experiments can supply a model for future implementation of participatory assessment of the water policies, whereby lay citizens and stakeholders are involved together with experts and policy makers in AWARE-like processes. These may be especially helpful to assess the 'proportionality' of costs against benefits in specific case studies circumstances, because the criteria to be used in the assessment are not uniquely technical, requiring instead the explicit consideration of social norms and values. However, the method is general and may be applied to different societal challenges.

To start with, the AWARE method recognize that there are different ways to effect an interaction between scientists, policy (managers) and the public, whether citizens and/or stakeholders. The traditional way is to treat the three fields as entirely autonomous, interacting only within the established formal procedures of democratic societies, i.e. in the framework of public inquiries as prescribed by law or by delegation through elections.

Today it is however more common to follow what has been called a 'participation-limited' adaptive management approach (Kusel et al.,1996) which supports the close interaction between scientists and policy but not with citizens or stakeholders. This is pretty much the approach that characterises the EU WFD, as the common framework built up to follow the implementation of the Directive is actively promoting a closer interaction between scientists and policy managers across EU member states, through the setting up of several thematic working groups. But as the WFD strategists themselves admit, there are shortcomings in their approach, especially with respect to the use of research results from studies other than those commissioned by national governments; and with respect to inputs from stakeholders and civil society.

The integrated adaptive management approach takes a step further and tries to stage and learn from a closer interaction between science, policy, stakeholders and citizens. Such an approach is not necessarily always suitable, but if applicable it displays three main benefits:

1. it recognises and uses the public as sources of information and knowledge,
2. it builds trust and broadens support and
3. it generates ideas and questions paradigms thus contributing to both learning and creativity in problem-solving. This is what the AWARE process is all about.

Project results

The AWARE citizens conference process

The AWARE process has been concretely performed by recruiting a transnational panel of final water users, i.e. 30 citizens randomly selected from three coastal areas of Europe. The citizens' panel has been engaged in a number of workshops with scientific experts, stakeholders and decision makers to assess the best scientific knowledge available, the local water management practices and the EU water policy framework and to formulate their recommendations.

The mandate to the citizens was to deliberate their assessment of and recommendations for interfacing experts, citizens and policy makers, to achieve a better management of coastal environments in Europe. This assessment has been presented and discussed in the AWARE European Conference 'Linking research to policy in the water sector', held on 9 June 2011 in Brussels, at the European Economic and Social Committee.

The panel of 30 randomly selected citizens was the catalyser of the overall process: they met with scientists in a first European workshop 'to be acquainted on the topic, the process and their mandate' then split in sub-groups of 10 citizens for each pilot area and met again scientists, stakeholders and decision makers in local workshops and conferences and finally they convened together all again at the EU level to prepare and discuss in a final conference their assessment of the coastal water management topic and the issue of connectivity between science, society and policy makers. The citizens' assessment and recommendations are presented in the "AWARE Citizens declaration".

The role of the AWARE citizens' panel was akin to that of a 'citizens jury'. Indeed, citizens juries bring together a panel of randomly selected and demographically representative citizens for a period of a few days to discuss specific policy issues. Citizen juries tend to have on average 25 members. The deliberations are conducted by a neutral facilitator and often involve experts on the given issue(s) to deepen the debate.

Also in AWARE, the recruitment process of citizens was mostly dominated by the challenge of building a representative sample of the population concerned and to find people with sufficient English proficiency as well as interest to the topic. The language condition had to be met to ensure that citizens could communicate, in some cases on the regional level already, at least on the European level. Besides this, the selection of citizens was based on their motivations and ideas about coastal water quality. The recruitment of citizens occurred differently among the three case studies. In Sacca di Goro and in the North Sea cases the response rate to the widely lead recruitment campaign was low. In the Gulf of Riga the response to the recruitment campaign was comparatively more positive, potentially due to the activities of the project partner that took on this task, a local non-governmental organisation (NGO) 'Baltic Environmental Foundation', knowledgeable about the most appropriate dissemination sites that would reach the targeted public.

As it concerns the living interaction throughout the AWARE process among all the participants, relevant knowledge has been provided and brokered in different formats and measures: expert knowledge was provided mainly by scientists, tacit and local knowledge mainly by stakeholders and local policy makers. Citizens also provided local knowledge, as well as personal experiences of the state of the coastal water resources. The process was also complemented by two rounds of on-line surveys, targeting different stakeholders at local level and a wider scientific and stakeholder community at the European level and interviews to keynote decision makers, again at local and European levels.

As it concerns the scientific information delivered in the process, in the Sacca di Goro case, the Elinor Ostrom's general framework for analysing sustainability of socio-ecological systems was used in the knowledge exchange process. In addition, the analytic hierarchy process (AHP) multi-criteria method has been used in this case study to evaluate different management options.

In the southern North Sea case integrated river basin-coastal water models have been used to test alternative scenarios with different measures to reduce nutrients from diffuse sources (agriculture) affecting coastal water quality and the scenario assumptions and results have been discussed with the citizens and the stakeholders convened in a local conference held in Dunkerque, on 7 January 2011.

In the Gulf of Riga case the local workshop, held along the gulf coast, was followed by the local conference after only one day. This was due to the more difficult logistics of arranging meetings with a citizens' panel from both Latvia and Estonia. However, the time in between proved sufficient for the scientists to adapt models and scenarios with the input from the citizens and for these latter to finalise the citizen declaration started during the workshop and prepare for the deliberations with the policy makers and other participants at the conference.

In all the case studies, the moderators of the participatory project were carefully selected among the project consortium's experts, bearing the advantage that instead of being recruited solely for the matter of one event, they were involved in the process from the beginning on. Additionally, training was provided by consortium's scientists and invited experts and prepared with intense exchange of knowledge of the AWARE purpose and context preceding the meetings.

In-built evaluation of the AWARE pilot experience

As a pilot project funded by the EC in order to test a new knowledge brokerage method, AWARE was subject to a careful evaluation by a team 'formed from the project consortium' taking the role of independent observers of the participatory process and its outcomes.

The objective of the AWARE evaluation team has thus been to observe the ways in which project partners, most of whom are scientists, learned from the interactions with citizens, stakeholders and policy makers about how to move towards a more integrative science-policy-public interface. In particular, the evaluation has described how elements of the design and preparation phase affected the development of the process, considering as relevant sources of information the outcomes of the participatory moments (discussions, results), the evaluation of participants (questionnaires) and the role played by the partners in the interactive dynamics.

The final goal of the evaluation was to understand to what extent the knowledge brokerage was effective, according to the AWARE project scope. The analysis therefore concentrated on the level of awareness and critical knowledge achieved, the level of satisfaction regarding the design and content of workshop sessions and specific aspects of the workshops and conferences.

On a whole, the evaluation of the AWARE process has been positive but with two distinct weaknesses. The first one concerns the interaction between the citizens and the scientific experts. The second weakness at the local level was the difficulty to involve policy makers. However, an alternative way to enable a more productive citizens-policy makers interaction is to trigger a continuous informal process of confrontation on key sustainability science-policy issues between the policy makers and the citizens involved in AWARE-like awareness raising process, to help bridging the gap between the citizens 'street-level' information, perspective and understanding of the topic and that of the elected representatives.

The AWARE outcome and lessons learned

The main outcome of the AWARE process is the 'Citizens declaration'. Besides the citizens' assessment, as summarised in their declaration, the AWARE case studies showed that all participants gained new and significant understanding and insights on coastal water management through participating in a set of local workshops and conferences. They exchanged views on a broad range of issues relating to the short and long term health of coastal waters. The groups in all three case studies also addressed specific problems related to agricultural policy, water quality and pollution and socioeconomic trade-offs. The following sections collect the most significant comments from different participants - citizens' panels, scientists, stakeholders and policy makers attending the local and the European conference, interviewed or answering to the on-line surveys - and illustrate a number of lessons learned as tips for future applications of the AWARE methodology.

Lessons about engaging citizens

The recruitment of citizen panels was dominated by the challenge of building a representative sample from the population concerned and to ensure sufficient English proficiency as well as interest in the topic. The language condition had to be met so that all citizens could communicate not just across borders in the transboundary cases, but also at the European level. In addition, the selection of citizens was based on their motivations and opinions about coastal water quality and management. As the case study descriptions highlighted above, the citizen recruitment was different in the three areas. In the Sacca di Goro and in the North Sea cases the response rate to the widely disseminated recruitment campaign was low. In Sacca di Goro especially, the selection of the 10 citizen panel was influenced by the need for sufficient English language proficiency, a prerequisite hardly owned by residents in the small Goro community, but needed in order to allow an acceptable level of exchange and discussion among the three panels at the European level. Compared to the other two cases, the response rates from the Gulf of Riga were more positive. This may be explained by the Baltic Environmental Forum's (BEF) experience in public communication and dissemination: advertising the AWARE project in the largest job and volunteering portal in both Latvia and Estonia contributed significantly to the fact that the Gulf of Riga citizen panel was selected at the desired rate of 1 member in 10 applicants.

Tips for future participatory projects

1. Instead of traditional open hearings a better response and feedback may be gathered through a random selection of individuals that form a citizen panel; these should then be part of the entire policy consultation process. This requires a careful selection procedure using a call for citizens appropriately disseminated in print and online to the target audience; the collection and evaluation of citizens' application forms; and the selection of panel members and deputies from the pool of candidatures received with the support of software ensuring fair opportunity to be selected and a balanced composition of the panel.
2. The commitment of the citizens selected as panellist need to be ensured at the very beginning of the process, by signing a letter of commitment where the terms and conditions for their participation are established and a nominal fee to compensate for their time is agreed to be paid at the end of the process. Any travel and accommodation cost needed to attend transnational workshop must be covered from the project budget.
3. Citizen panels should aim to be representative of the socioeconomic structure of the case study they represent, however considerations about language proficiency and a basic level of interest in the sustainability topic addressed are a priority. The ability to speak a common language is crucial when working with transboundary and cross-European citizen panels: the presence of language interpreters would greatly reduce the effectiveness of interactions between participants and would significantly increase costs. Substantial time however, is still needed to clarify for those involved the terminology of relevant environmental laws and directives.

Lessons about engaging stakeholders and policy makers

The engagement process of stakeholders presented challenges not in terms of language skills, as they have been involved only in the local knowledge brokerage processes, but in terms of achieving participation from the whole range of relevant organisations. For this purpose, it proved to be a successful approach to use a matrix dividing stakeholder organisations into four groups depending on their level of influence and their level of interest. In this way, not only actors with high interest and high influence were identified and involved, but also those with high interest but low levels of influence, as they were considered as important and in need of empowerment.

It is interesting to note that in the AWARE project the task of engaging the various stakeholders rested in most cases with the scientific project partners. This gave increased credibility to the engagement efforts, from the point of view of the stakeholders, albeit it was not an easy task for partners mostly used to interacting in academic networks rather than advocacy and policy communities. In all case studies the local knowledge brokerage events were well attended by stakeholders. However, there was a notable lack of involvement from industry representatives, which was noted by the actors involved, by the parties interviewed throughout the AWARE process and during the evaluation process. Even so, some differences were noted across the case studies.

Finally, although the local workshops and conferences were attended by policy makers it was often difficult to actively involve them both at the local and at the European levels. More importantly still, the nature of the interaction between policy makers and the citizen panels remained on the level of political statements rather than producing a true exchange of ideas. Comparing this with the discussion between the citizen panel and the other actors involved, it is clear that there is room to improve the way in which the policy and the public interacts. In fact, this gap appears to be at a systemic problem rather than an organisational weakness of the AWARE project. Unlike science, the realm of policy-making is more concerned with representing and/or weighting different issues and interests, than with understanding natural or social phenomena. Moreover, it seems difficult to engage policy makers on topics and processes that require long-term consideration and that might span beyond their mandate. The active involvement of policy makers in the process however, is crucial as they only have the legitimacy to take any eventual decision, on the basis of the deliberations formulated by a randomly selected small group of citizens.

Based on the above, some additional tips for future participatory projects are the following:

1. engaging stakeholders from across the low-high interest and low-high influence continuums is crucial in order to achieve a balanced exchange of knowledge, views and information
2. the participatory process gains credibility by tasking scientists and trusted regional NGOs with the stakeholder engagement
3. the participation of a permanent 'policy and science advisory group', as in AWARE, can provide significant feedback and positive inputs both during the knowledge brokerage events and during the evaluation. Members of this group should be key actors in the study areas, have a relatively high interest in the process and come from different backgrounds.
4. industry representatives are a key actor - when they are missing from the discussion a wide array of needed knowledge is lost, which has repercussions on the process and outcomes. Reaching this target audience in future projects may include bilateral consultations with industry representatives around concrete outcomes and recommendations.
5. a suggested alternative to enable a more productive public-policy interaction is to engender a continuous informal science-citizens-policy makers interface process of consultation on key sustainability issues.

Lessons about engaging scientists

The AWARE experience and in particular some of the interviews undertaken, show that there is a gap in the dialogue between scientists and policy makers, as well as between policy makers and the general public especially on complex topics that require a technical background. As for the first gap, the dialogue is often unidirectional, with policy makers asking the scientists for advice but with scientists not always directing their research to answer policy questions. This dialogue seems to work much better at the EU rather than at the national and local levels and this fact claims for a better connectivity between the body of research on which the EU itself relies on and national and regional research programs and advice to policy makers. As for the second gap, the interface between policy makers and the public lacks efficiency in part because technical knowledge of lay citizens is generally low, thus hampering productive dialogue on complex sustainability policy issues.

Another barrier may be that communicating scientific knowledge to a lay audience is a difficult task both for scientists, who would need public communication expertise to which they are often not used to, and for citizens, for whom workshop attendance alone may not be enough to acquire a complete scientific knowledge. The AWARE process has demonstrated, however, that a well-structured participatory process where citizens meet scientists with a clear purpose, to discuss a specific sustainability challenge and with enough time and commitment available, can greatly help to overcome this barrier. An important outcome of the process was that the citizens became somewhat more like scientists and scientists more like citizens: AWARE built a common language between the two groups, based on a common understanding of complex issues and on increased awareness gained in a neutral forum.

In the Sacca di Goro case, Nobel laureate Elinor Ostrom's general framework for analysing sustainability of socio-ecological systems was used in the knowledge exchange process. In addition, the AHP multi-criteria method was used as an evaluation tool to measure the mutual distance of the stakeholder groups from a common vision of the Goro system, as well as the priority of actions to be implemented for improving the social, environmental and economic situation of the same system. The AHP analysis of the stakeholders' answers was an object of discussion during the local Italian conference, serving to link the workshop and the conference.

In the Gulf of Riga the connection between the local workshop and conference was strengthened by having only a one day break in between. Although the time proved sufficient for the case study scientists to adapt existing models and scenarios with the input from the citizens and workshop and for these latter to prepare for the deliberations with the policy makers and stakeholders at the conference, more time would have been useful for the citizen panel to further develop the local citizen declaration, potentially allowing for meetings outside of the planned project activities.

The scientific background and participatory modelling for the North Sea case study was provided by the North Sea team partners particularly on the basis of previous and on-going studies such as the Liteau, Thresholds, Timothy and PIREN-Seine research programmes. Although the modelling goals of the participatory process were achieved across all case studies, the final evaluation comparison between the three case studies showed that the process and the outcome, in the form of local citizen declarations, may have benefited from a lengthier consultation with scientific and policy experts, both in terms of the group cohesion and in terms of the concreteness of the citizens' recommendations.

Tips for future participatory projects:

1. citizens' input can help scientists to focus on a more comprehensive view of the problem at stake, avoiding the pitfalls of compartmentalisation
2. including the opinions of stakeholders and citizens enriches scientific models and scenarios and helps develop more robust results. Systematic approaches should thus be developed to promote this type of interaction.
3. citizen-scientist interactions benefit from a regular consultation process across time, during which knowledge and information can be exchanged; trust built; and a 'common language' based on understanding of complex challenges and mutual awareness can be developed.
4. complementing participatory workshop interactions with public conferences helps maintain actors' motivation and interest in the process and provides an ideal public forum for the presentation of the achieved results and a the opportunity to built up a consultation around citizens' recommendations.

Lessons about organising the knowledge brokerage process

Knowledge has been provided throughout the AWARE process by all the participants in different forms and measures: expert knowledge was provided mainly by scientists, tacit and local knowledge mainly by stakeholders and local policy makers. Citizens also provided local knowledge, as well as personal experiences of the state of the coastal water resources. The AWARE activities were thus specifically designed to allow an exchange between these different types of knowledge and for learning to occur between the different actors.

The knowledge brokerage events were organised in a similar manner across case studies, using sessions to present specific expert knowledge from scientists or stakeholders, followed by a moderated discussion between presenters and citizens, taking into account the various opinions represented. As part of the monitoring evaluation, the sessions were carefully documented and minutes were made available. In fact the transparency of the information proved a key requirement in building confidence in the process itself.

Following the local workshops, local public conferences were designed to disseminate the knowledge gained and exchanged to a wider audience: in all case studies around 50 participants attended the conferences, including policy makers who commented positively upon the outcomes and the innovative approach of the AWARE process. The time and space allocated to the interaction between different types of actors are important aspects to consider in order to build trust between different groups of actors: in general it was noted that citizens seemed to trust scientists from the beginning, but more time and opportunities for interaction were needed to increase the trust between citizens and policy makers and between policy makers and scientists.

Regarding the interaction between citizens and the scientific experts, the latter were asked to make presentations regarding models or the state of the environment clear and understandable for a lay audience and, with a few exceptions mostly coinciding with invited external actors, this proved to be quite successful. In fact the evaluation team noticed a marked decrease in the amount of 'community-specific jargon' used by the different actors as the participatory process progressed. The greater challenge derived from the unavoidably incomplete scientific information conveyed by the experts to the citizens, due to clear time constraints. In some cases even small bits of incomplete information resurfaced in the form of erroneous assumptions in the citizen deliberations.

Tips for future participatory projects

1. good moderation by a team of communication or social experts that is trusted and involved throughout the participatory process is essential
2. citizens' panels need time to consult not only with experts during workshops and conferences, but also among themselves, if they should produce concise, concrete and cohesive statements on a given sustainability issue
3. facts and figures presented by experts should be easily understood and available
4. expert knowledge should be shared and embedded continuously into the knowledge brokerage process.

Lessons about delivering outcomes

As mentioned above, in the AWARE project the outcomes of the knowledge brokerage process were the three local citizen declarations and the common European citizen declaration.

The evaluation of the outcomes showed that participants gained new and significant understanding and insights on coastal water management through participating in the workshops and conferences. They exchanged views on a broad range of issues relating to the short and long term health of coastal waters. They also addressed specific problems related to agricultural policy, water quality and pollution and socioeconomic trade-offs. Additionally, thanks to the process, the citizens involved in AWARE committed to taking personal actions to protect the environment and the experts gained new perspectives to apply in their research field. A clear and positive outcome was an increased awareness for all participants of the complexity of the coastal water management systems, as well as a deeper understanding of the need to engage the whole spectrum of actors in a continuously adaptive process to produce truly sustainable benefits.

Additionally, while managing the expectations of the actors involved, in particular of the citizens' panels, a real challenge was to find the right 'entry points' for changing current policies towards sustainability. While it was crucial that all actors engage honestly and openly with each other during the knowledge brokerage process, participants perceived the outcome to have limited capacity for concrete change. Such perception was even more pronounced at the level of the final outcome, i.e. the European citizen declaration. Thus, it seems that such a knowledge brokerage process may be more appropriate at the local or regional level, where long-term informal interactions between all the actors are more practical.

Although mentioned above, it bears repeating that trying to pre-determine the direction in which discussions will move, by providing only specific sets of expert information for instance, is not useful. In addition, the evaluation of the AWARE process also showed that it is crucial to allow sufficient time for the citizen panels to consult, during and after the participatory moments such as the workshop. This is necessary in order to achieve the best possible outcome, in the form of citizen declarations.

Tips for future participatory projects:

1. in addressing complex sustainability issues, the outcomes of a knowledge brokerage process are closely affected by the extent to which the whole spectrum of actors is involved
2. in a knowledge brokerage process engaging citizens' panels, it is necessary to address expectations regarding the outcomes of the process and the concrete possibilities of implementing certain options. The willingness of individual policy makers to communicate openly and take up insights from a body of lay citizens is a key prerequisite to achieving concrete impacts on policy processes and decisions. However, an important barrier is the mismatch that often occurs between the responsibility that the local policy makers have of taking care of concrete actions at local level and the public budget that they directly govern.
3. all the participatory events should be planned to maximise their communication effectiveness, including clear definition of roles and objectives. Facilitation should guarantee inclusion and proper balance of all the participants, time for discussions, understandable information, appropriate and respectful ways of interaction. Giving ample time for the participants to consult in the co-creation process is crucial for a cohesive outcome that has the full support and ownership of the citizen panel.
4. achieving a common basic knowledge of the issue at stake and using a commonly-understood language are key aspects to bring all actors at the same level and, thus, develop synergies. Informal types of knowledge may be accepted by some participants while more formal knowledge is required by others. One effective solution proven in AWARE was to develop a multi-language glossary of technical terms to help the citizens' panels in understanding and comparing different terms.

Ex-post evaluation of the AWARE approach

As illustrated in the sections above, the goal of the AWARE project has been to demonstrate a new way of 'connecting research, people and policy makers in Europe to achieve sustainable water ecosystem management'. The project has used a variety of methods and activities to achieve this aim.

Personal interviews with decision makers in water management were conducted in particular at the EU and international level, to assure a more personal approach and to get a more complete ex-post evaluation of the AWARE approach that could not be received via the online survey, as well as to capture further insights and recommendations from stakeholders that may not have participated in the AWARE workshops or conferences.

More in detail, the objectives of these interviews were to:

1. gain feedback on the perception of the policy and decision makers on active citizens' involvement and especially on their perception of the AWARE approach;
2. identify the reaction of policy and decision makers to the results produced by AWARE, specifically to the citizens' recommendations for EU policy makers;
3. produce recommendations concerning the enhancement of the connectivity between research and policy-making through participatory processes.

The institutions targeted by the interviews at EU and international level included the EC, international and national research institutions, national ministries in charge of water and the environment, national water agencies, as well as European and international water organisations and platforms, such as the Euro-Mediterranean Information System on know-how in the Water sector (EMWIS), the European Federation of National Associations of Water and Wastewater Services (EUREAU), the European Water Partnership (EWP), the Water supply and sanitation Technology Platform (WssTP) and the Eureca Cluster Acqueau. The main conclusions of these interviews regarding the AWARE process are summarised below:

1. all interviewees valued the AWARE experiment in a positive way, but highlighted the lack of involvement of the general public.
2. the question of whether to involve stakeholders from the industry was raised a few times by the interviewees: in their opinion, this type of stakeholders have so far been missing in the process; however it was recognised from the design of the AWARE process and from the interviewees' own experience that involving industry actors in such a process is in general not an easy task; one suggestion was to try to take the AWARE results to federations in countries where these are likely to arrange regular contacts between industry actors, policy makers and science
3. in general, the interviewees think that such kind of procedure would have little-to-no impact on coastal water quality and sustainability; some of them prospected an indirect effect; some others deemed a strong intervention by local or national government as necessary to achieve a significant impact.
4. the approach is felt to be applicable to other policy areas; however the general feeling is that citizen involvement is effective and efficient at local level and as the level goes up to national or European level the citizens lose touch with the scale of the problem, which become more and more vague and impersonal, or with little pertinence/impact to their personal experience and everyday life.

About the AWARE outcome it should be noted that:

1. the reaction of the interviewees to the recommendations presented in the AWARE citizens declaration was unanimous in that, although positive, such recommendations are for the most part not well addressed at the EU level.
2. it is generally felt by the majority of interviewees that there is a gap in the dialogue between scientists and policy makers, which is not optimal and needs improvement; the dialogue is often unidirectional, with policy makers asking the scientist for advice but not with scientists directing their research to answer policy makers' questions.
3. it is also generally felt that the policy makers/public interface is not so efficient as it should be; scientific knowledge of the lay citizens is generally low, thus hampering further the possibility of dialogue; the contact between local managers and the general public improves as we move from the EU to the local level, as in the case of the river basin management plans (RBMPs), for which public involvement is a mandatory requirement explicitly outlined in the WFD.
4. all interviewees welcomed however the approach and the outcome of the AWARE process; many recommended the use of social networks or local scale dissemination to disseminate ideas, awareness and to foster participation by a larger public.
5. as it concerns the prospects for delivering a significant impact at EU policy level, a fundamental requirement would be to institutionalise and make the application of the AWARE method more systematic. This would be useful especially to establish trans-boundary activities and connections between policy makers, scientists, stakeholders and citizens at the level of international river basin organisations. The AWARE approach needs however endurance to be successful, because the good ideas need to be implemented at all levels and acting on a single project basis would be not enough.

A further and final step of ex-post evaluation was to convene a workshop, inviting the members of the AWARE PSAG and a number of selected water managers to attend a full day meeting organised at the Austrian Permanent Mission to EU, in Brussels, on 20 October 2011. The workshop was attended by Frèderique Martini (ONEMA, France), Rolands Bebris (Ministry for Environment and Regional Development, Latvia), Jésus Manuel Gago Pineiro (Directorate General Environment, Marine & Water Industry), Lorenzo Galbiati (Agència Catalana de l'Aigua, Spain), Eric Mino (EMWIS/SEMIDE), Bruno Rakedjian (MEEDDAT, France), Sofie Vanhooren (Coordination Centre for ICZM, Belgium), Elena Giusta (ISPRA, Italy), Marta Valente (Ministry of Environment, Italy), Francesco Tornatore (River Po Authority, Italy), Rene Reisner (Ministry of Environment, Estonia), Lisa Struebbe (European Water Partnership) and Jean-Paul Vanderlinden (Observatoire the Versailles Saint-Quentin).

The AWARE partners (ISIS, JRC and the evaluation team: Adelphi Research and Poliedra) presented the AWARE method and pilot experience in the morning, together with the results of the in-built evaluation and two working groups were formed in the afternoon, to discuss follow-up recommendations for action to be taken for future implementation of the AWARE approach at EU and local level.

A key question addressed in the ex-post evaluation workshop was how to achieve a snowball effect, scaling-up the AWARE pilot project to produce a wider impact. Two directions of possible impact have been envisaged, the directions are:

1. to expand the application of the AWARE method to other sustainability topics and related science and policy frameworks. Indeed, while AWARE addressed coastal water management specifically, it should be made clear that the methodology could also be useful to deal with other sustainability and social challenges that unfold in the long-term. AWARE-like knowledge brokerage processes particularly useful for issues that transcend electoral timelines, as they contribute to build a more robust understanding of complex undertakings and a greater commitment towards durable actions and policies. Future European projects inspired by the AWARE method can bring therefore researchers and citizens from across Europe together in collaborative research experiences to address cross-cutting societal and sustainability challenges that Europe is currently facing, including energy and climate change and sustainable transport among others.
2. to deepen the approach in the water sector, in order to support integrated coastal zone management (ICZM) across Europe with a more effective participation of citizens and stakeholders in the common implementation of the WFD. The current situation of participation in the WFD context and future prospects open by more intensive methods of participation as that experienced in the AWARE pilot, have been discussed in depth in the afternoon session of the ex-post evaluation workshop.

Potential impact:

The AWARE pilot experience has been translated into a more general concept of 'AWARE solution for participatory assessment of the WFD implementation'. The solution considers the whole water policy and research field which is regulated from the EU level, where the WFD has been formulated, through the transposition of the Directive in the member states down to the level of river basin organisations, in charge of the RBMPs targeting the good ecological status of river and coastal waters by the year 2016 and finally down to the bottom level of local authorities (municipalities), that usually have the responsibility to implement local plans and actions on their territories.

Nowadays, public participation is mostly organised, where it exists, in the form of public consultation open to local population and stakeholders representatives and aimed to assess the implementation of local water management plans, without any formal procedure that would help the citizens to address the water management process earlier, at the level of formulation and/or implementation of the river basin management plans, or even more ambitiously at the level of EU policies formulation or implementation.

The geographical and institutional distance of the average European citizen from the 'Brussels polity' centre has created a large gap, which would need now new participatory assessment procedures and a systemic approach - supported by a strong top-down political will at EU level - to be filled in. The AWARE solution, if backed by a consistent political support, may represent such a systemic approach. It may be used to formalize the citizens' participation to the impact assessment of the EU water policy (as well as other policy and research fields addressing grand societal and environmental challenges), by recognising that there are three distinct layers of participation to be connected:

Layer A: Think globally. This is the layer already tested in the AWARE project, i.e. the participation of panels of randomly selected citizens from different coastal areas/river basins of Europe to a citizens' conference process organised at EU and local level. This interfaces scientists, lay citizens, stakeholders and policy makers, to be engaged in a common process of assessment of water management goals (namely achieving a good ecological status of coastal water), the related research evidence and the formulation of policies at EU and local level. Participation here is focussed on thinking to global or universal issues, e.g. deterioration of coastal waters in Europe, and how they are handled by the current policy framework at EU and regional level, what evidence is produced by research and how the brokerage of knowledge between scientists, citizens and policy makers may help to improve consistently the planning and management of water resources and ecosystems at local level. Layer A envisages the engagement of small groups of randomly selected citizens on sustainability topics of interest, to participate and produce their assessment at an early stage of policy formulation (i.e. at a stage where EU regulations and directives are in gestation and an ex-ante impact assessment is needed) or at a more advanced stage of policy implementation (i.e. when regulations/directives at EU level are issued and need to be translated at national and local level, requiring bottom-up citizens' empowerment approaches to enabling an effective implementation in the local contexts).
2. Layer B: Act locally. This is the layer where more conventional public consultation processes are usually set up, with the aim of building consensus on policies and programs already framed in the context of the RBMPs. Typically, participation is limited in this layer to the design, implementation and/or assessment of downstream project details and to decisions within the reach and boundaries of single local authorities.
3. Layer C: Be aware. The third layer is perhaps the most innovative and aims to build up a participative forum on Internet open to lay citizens, civil society representatives and stakeholders where public information related to periodic (annual or mid-term reviews) impact assessment exercises of the WFD implementation is provided and access is given to all interested users to provide their comments, discuss policy implementation aspects, publicly control and monitor the progress of policy implementation, etc. The focus of participation here is on raising and keeping alive the awareness of the public about the implementation of the water policy across Europe and let the citizens have more permanently the opportunity to control its progress and be aware of the actions required to avoid the deterioration of coastal water quality (an important aspect of quality of life).

Implementing this AWARE solution would be the main avenue to produce an important impact. To achieve this, we would need water managers and policy makers, at international, EU and river basin level, to appreciate and mainstream the AWARE pilot project results in their water policy environments. We would need also to connect the innovative citizens' conference approach (layer A) tested in the AWARE project with the other two layers, i.e. local public consultation processes ex Art. 14 of the WFD (layer B) and a participative WFD forum (layer C) that should be opened and maintained at the EU level, with affiliates in the different river basins of Europe (for instance, with a structure similar to the web-site used to support the covenant of mayors in the energy and climate change policy field, see http://www.eumayors.eu online).

To trigger the above potential impact, the AWARE solution has been disseminated in several scientific and policy conferences during the second project period, as illustrated in the second project periodic report and eventually has been included as one of the innovative solutions presented at the Sixth World Water Forum 'Time for solutions' in Marseille, in the workshop organised by Onema and the Office International dé l'Eau on 14 March 2012 on the topic 'Science and water policy interface: When science and innovation meet water policies'.

Among the main outcomes of the workshop is the intention, supported by several participants (including in particular the EC, Directorate General Research and Innovation - Environment, represented by Ms Luisa Priesta), to implement an action plan for improving the science-policy interface, especially in the context of the forthcoming EU Research Programme 'Horizon 2020'. Amongst the features recommended to frame this action plan (Martini, 2012), there is the need:

1. to build multi-stakeholders processes, i.e. platforms including research and policy, local authorities, business including small and medium sized enterprises (SMEs), NGOs and citizens at all relevant scales
2. to develop an online platform where citizens can engage in the process to give them the opportunity to influence policies
3. to recognise the importance of knowledge brokers to reconcile the two sides of policy and research
4. to develop financing and incentive mechanisms for research dissemination and uptake, finding in particular funding opportunities and people specialised to facilitate the transfer of knowledge
5. to ensure knowledge transfer across different geographic locations within consistent regions.

The above recommendations resonate with the main conclusions of the AWARE project.

Knowledge brokerage processes based on the AWARE-proven methodology should be used therefore to improve the science-citizen-policy interface in Europe and support decision making on sustainable water management and other sustainability issues. The following set of recommendations, based on the AWARE experience, are finally delivered here as a guidance for water managers that are applying - or are willing to apply - participatory processes, to foster the dissemination and increase the potential impact of the AWARE project.

1. Bring citizens, scientists, stakeholders and decision makers together in a participatory knowledge brokerage process to improve decision making at the local level and increase ownership of challenges affecting citizens across Europe. Implementing a knowledge brokerage process for improving decision making at the local level can help increase ownership: citizens and stakeholders have a higher incentive to participate around local issues where they will be able to track and measure the impact of the decision process. Nevertheless, working at larger, European, scales provides more relevance to the process for all actors involved, as it also includes the top-level policy framework for sustainability issues. Thus, depending on the objective of each participatory process, a compromise between these two perspectives, local/national and supranational, should be found. At all scales ownership and personal involvement is significantly increased by clearly defining the outcome that is expected from the knowledge brokerage process.
2. Focus on local level implementation while also feeding the lessons learned back into European level policies. A knowledge brokerage process may best influence the local implementation process of European Directives, since these provide some flexibility and room to manoeuvre at the national and local scales. The participatory process also benefits significantly from informal-formal regular opportunities for various actors to interact, which is easiest at the local levels. However, it is also important for local participatory process to provide feedback on the ways in which European Directives can be implemented, which will be useful at the EU level in the design of future policies.
3. Take into account cultural and empowerment differences when implementing a knowledge brokerage process, especially to ensure a fair level of trust in the process outcomes. Taking into account cultural and empowerment differences is important before implementing a knowledge brokerage process, especially regarding trust in the outcomes of such a process: it is important that the broader public, from among which a representative panel will be selected, believes that concrete results can be expected from the process, given clearly defined outcome goals. These outcomes are not necessarily a specific set of decisions, which are ultimately to be taken by legitimate powers at EU, national, regional or local level, but may be more broad and informal outcomes (actions or initiatives) that can help to determine policy strategies supported by a deeper consensus, built upon a greater awareness of the issues at stake from all actors concerned.
4. Bring citizens, scientists, stakeholders and decision makers together regularly to allow for trust building and effective learning and knowledge sharing. Implementing a knowledge brokerage processes in a geographically delimited area is more efficient: the necessary permanent or regular informal processes involving different types of actors to support decision making will be more effectively carried out. Such regular opportunities for interaction, focussed in time and space on a particular area and topic, also allow the activities to take place in a common language.
5. Involve local civil society actors to best reach and engage with the broad public. Involving local NGOs and social actors is important to reach and engage a broad public: because of their good knowledge of local societal structures, they are able to act as 'mediators' between actors. They can also act as catalysers of action and as 'multipliers' to ensure a wide dissemination that reaches all types of lay citizens - not only those with access to most resources and networks, or those groups of selected individuals that consider themselves especially environmentally-friendly.
6. Engage all relevant actors to the extent possible, particularly those from relevant industry or economic sectors. Try to reach actors from relevant industry or economic sectors by engaging on concrete topics with industry representatives and federations who are likely to arrange regular contacts between industry actors, policy makers and science. Future EU funded knowledge brokerage processes between scientists, citizens and policy makers should aim to connect with the European Innovation Partnership on water efficiency and other sustainability challenges.
7. Involve a team of scientists throughout the knowledge brokerage process. 'Scientific ambassadors' could communicate critical information to citizens, business representatives and policy makers alike. AWARE has shown that involving a team of scientists for each case study throughout the knowledge brokerage process is significant: scientific knowledge should be shared with a broader public and the academic expert teams should accompany the whole knowledge brokerage process, also in designing the output of the process with the citizens' panel, to ensure that relevant knowledge is accessed and processed accurately. To help with complex sustainability issues, 'scientific ambassadors' could 'translate' critical information for citizens, business representatives and politicians alike. These experts should have cutting edge knowledge of research advancements in a given sustainability domain, personal communication skills, a mind open to a broader dialogue and an understanding of the socioeconomic implications of their research. Increasing academic engagement with projects similar to AWARE is one way to do this. However, the scientific award system could, in addition to a scientific careen, recognise and encourage individual participation in this kind of participatory initiatives, as well as efforts done to tailor scientific results to wider target audiences.
8. Exchange best practices and share the lessons learned from already implemented knowledge brokerage activities with others implementing such activities at local or regional levels. To increase the relevance and appeal of participating in more innovative knowledge brokerage processes, existing processes also need to be identified and reviewed: exchanging and comparing results is important for action learning and for improving the visibility of knowledge brokerage activities at local and regional levels across Europe. Feedback on this kind of experience is also needed at the EU level, where active involvement of different parties through participatory processes is encouraged, particularly on sustainability issues.