Final Report Summary - EST-FRAME (Integrated EST Framework)
Executive Summary:
Exploring the needs for integration in assessment of emerging science and technologies (EST) was a central aim of the FP7 project Integrated EST Framework (EST-Frame). The EU, national governments, and a whole range of national and international institutions produce assessments of emerging science and technologies. These assessments take different disciplinary perspectives and may point in different directions for responsible governance of technologies, and there is currently limited integration of such advice, opinions and appraisals. The EST-Frame consortium, consisting of researchers from the Danish Board of Technology, Fraunhofer ISI (Germany), the Wageningen LEI Institute (the Netherlands), University of Nottingham (UK) and Oslo and Akershus University College (Norway, coordinator), responded to a call from the EU, through the FP7 Science in Society Programme for making recommendations for more integrated EST assessment.
The EST-Frame project established a threefold workplan for the task: 1) we studied how emerging science and technologies were assessed and governed in four case studies; 2) we studied the main methods and traditions for technology appraisals – and this we referred to as “assessment domains”; and 3) we identified major societal trends affecting assessment practices. Furthermore, we aimed at a differentiated analysis of what integration means at different levels and for different purposes.
The consortium engaged a range of stakeholders and assessment practitioners in order to discuss and review integration through a series of workshops. The results from the case studies, the study of the assessment domains, the trends analysis and the workshops are published in a special issue of the journal Science and Public Policy, June 2014. References to these articles, as well as the project’s full publication list, can be found at www.estframe.net.
In December 2014, the EST-Frame project presented the outcomes of the work to a policy-oriented audience. One of the notable conclusions was that there is a lack of dialogue between assessment communities on basic assumptions for assessment, and that assessment communities struggle with similar challenges specifically related to uncertainty, complexity and controversy around facts and values related to new technologies. The resulting recommendation is a process and learning approach named TranSTEP (TranS-domain Technology Evaluation Process). The central elements of a TranSTEP process include – through continual use of facilitated dialogue - to conduct a transdisciplinary situation analysis and transparent problem framing before setting out with assessments, to reflect commonly on method choices in current and potential new assessments to be integrated, and to integrate the results in a transparent and reflective way.
Although some integrated approaches already exist, such approaches are still not mainstream in technology appraisal. In the project’s final conference all practitioners and researchers linked and directly involved in EST assessment were invited to join in a longer term quest for achieving in practice more balanced and integrated assessment for responsible technology governance.
A full description of the TranSTEP is published on https://transtepapproach.wordpress.com
Project Context and Objectives:
The significant investment in science and the high levels of novel technological development and innovation currently witnessed globally raises questions about and constantly challenges the way in which technology governance is structured and conducted. Embedded within current EU policy goals, technology and innovation policy is intended to ensure that technology is developed in line with societal needs and is responsive to public concerns; at times responding to notable levels of public anxiety. Several forms of new technology development can be characterised as scientifically complex and novel innovations, presenting significant uncertainties related to the application of the technology. This means that policy makers and the wider public are largely dependent on expert interpretations of what risks and consequences the technologies may present to society or specific affected parties. These expert interpretations are provided in terms of risk assessments, impact assessments, ethical reviews, economic analysis, etc. In addition, different forms of public or stakeholder deliberation frameworks are applied to inform policy makers and build better understanding between societal groups regarding technology development.
However, individual analysis and assessment frameworks can often reveal only a partial presentation and assessment of social issues and hence different frameworks sometimes render different (and potentially conflicting) findings and conclusions. The selection of assessment framework may itself be a ‘value-based’ choice, as different frameworks rest on different premises, traditions of thought and ethics. The assessments, which can be conducted through the application of the different frameworks, may not converge into a stable and robust understanding of the social impact of the technology, but may point in different directions when it comes to responsible governance of technology. It is therefore important that the nature and choice of frameworks for technology assessments are developed to be more transparent, revealing the premises, potential biases, and strengths and weaknesses of the different models. On this background the European Commission, in the 2011 Science-in-Society programme, called for a more integrated approach to assessment and governance of emerging science and technologies (EST), to which the EST-Frame project is a response.
The notion of integrated assessment is not new. European Impact Assessments follow an integrated approach where economic, environmental and societal issues should be considered in assessments of European actions. Integrated Sustainability Assessment is a well-established approach on the side of environmental management. Integrated assessment of EST is therefore well placed in a landscape of integrated approaches. However, it is not immediately clear what an integrated EST assessment would consist in, although the subject matter indicates that scientific and social uncertainties and controversies would need to be handled by such an approach. This is the background for the EST-Frame project, which has carried out broad empirical studies in order to develop an integrated approach for assessment of emerging science and technologies.
In accordance with the overall concept presented above, the project has had the following aims and objectives:
Aim: To assist in making holistic and integrated assessments of emerging science and technologies resulting in socially robust technology development conducive to societal progress and increased quality of life.
In order to achieve the aim of the project we specified a number of objectives.
i. To undertake a study of four cases (nanotechnologies, synthetic biology, biofuels and security & emerging ICTs) to analyse the nature of the frameworks that are currently being used to assess the technology development and how the different assessments can be compared.
ii. To analyse the assumptions and results from the assessments of the four cases. Identify strengths and weaknesses of the frameworks.
iii. To identify different advisory contexts and in what contexts there is a need for integrated frameworks, and when more focused frameworks may be sufficient.
iv. To determine the role for an Integrated Assessment Framework in a policy context characterised by internationalisation of markets and technologies, global liberalist politics, and new forms of public-private partnerships.
v. To identify existing frameworks of a comprehensive or integrated nature.
vi. To develop an integrated assessment framework based on the analysis of the four cases and currently used frameworks.
vii. To apply the integrated assessment framework on the four cases and analyse how such an assessment differs from the earlier assessments. This includes organising four deliberative events where users, policy makers and stakeholders reflect on the totality of existing assessments in light of the integrated framework.
viii. To ensure end user relevance of the results by presenting them to a group of policy makers, advisers and key stakeholders, and revise the model based on their input.
ix. To disseminate the final results to end users and the academic community by a final project conference consisting of an expanded end user group, as well as communicating the results through the project web site, relevant other web sites, scientific journals and the media.
Project Results:
The project conducted four case studies of assessment and governance of emerging science and technologies, a mapping study of assessment frameworks currently in use, a study of policy trends affecting assessment and an analysis of integration needs in current EST assessment. We will here present the most important results from each of these research tasks.
3.1 THE CASE STUDIES
The four case studies followed the same protocol, structured into the following main sections:
• Description of the technology
• Description of the governance context. This context would be national (related to the responsible partner’s national context) as well as on the EU level.
• Identification/screening of assessments. A comprehensive search for relevant assessments in the selected country and the EU level.
• Selection of 8-12 assessments for closer review. These selected assessment should be policy relevant and represent different assessment perspectives
• Analysis of these assessments using the project’ analytic protocol; the process characterisation table and the purpose analysis table (see tables 1 and 2 in the pdf version of the final report), as well as a more general description of their context and significance.
• Qualitative comparison of the assessments and evaluation of the assessments situation revealed in the case study. This included the organisation of an expert workshop discussing the case study findings.
• Judgement on the needs for integration revealed in the case study (summarised in table 3 in the pdf version of the final report).
3.2. CASE STUDY ON NANOTECHNOLOGY IN FOOD AND AGRICULTURE
Nanotechnology has many different manifestations – in structures, as particles, and in degradable or non-degradable forms – and is surrounded by many uncertainties; nanomaterials exhibit properties not found at the macro-scale which may result in unpredictable safety problems and risks. As nanotechnology increasingly is being used in consumer products this will in time affect large number of consumers, workers and citizens in general, in addition to potentially impacting on the environment. Many therefore claim that nanotechnology cannot be widely implemented into society without public engagement, call for an inventory of nanofood and contact materials, and urge industry to communicate more with the broader public about the research and development they are doing in this field. Moreover, because of the potentially big gains to be harvested from nanotechnology, along with potential significant risks, there is a perceived need to carry out different kinds of risk assessments, impact assessments, economic assessments, ethics assessments, and so on. In this case study we studied a number of assessments with the field of nanotechnology in the food and agriculture sector, leading to a number of findings related to assessment and governance of nanotechnologies.
When selecting assessments for review we found that most nanotechnology assessments apply to nanotechnologies in general, and not specifically to nanotechnology in food (or agriculture). The assessments chosen for further analysis in this case study had primarily a focus on the Netherlands and Europe. We tried to encompass the perspectives of all relevant stakeholders and the multiple dimensions that can be impacted by the development of nanofood (economy, environment, safety, security, society and ethics). In total five assessments in the Netherlands and eleven assessments in other countries were selected and reviewed on several dimensions. For getting a more detailed picture of the assessments and the issues at stake in the debate on ‘nanofood’, interviews with twelve key stakeholders in the Dutch nanofood network were executed.
The study found that there is an equal assessment trend internationally (in the EU) as in the Netherlands. The first years of nanotechnology assessment have been dominated by deliberative efforts, while these kinds of approaches have not been so prominent in the latter years. Deliberative projects (like Nanologue, NEG, Nanodialogues, FramingNano etc.) started out with great ambitions, and achieved much, but on an overall level many felt that nanotechnology development and policy largely went on irrespectively of these activities. A vigorous public debate, outside of the organised exercises, was never achieved. Later, voluntary codes of conduct emerged, where industries (and researchers) would commit themselves to among other things be transparent and communicate uncertainty (EC 2008, Responsible NanoCode etc). Towards the end of the period we see more of the international institutions entering the stage and a shift towards risk assessments as well as guidelines for these. There is still a lack of methods and standards for risk research, and hence a lack of knowledge for assessing nanotechnology in a traditional scientific sense. Most of the evaluations acknowledge the lack of basic risk research as one of the major concerns.
The discussion of the findings in the case study highlighted that although it is understandable that environmental and human safety aspects are the most prominent concerns in debates on nanotechnology, this should not be the only focus in assessments. We believe that there needs to be an increased interconnecting of assessments of environmental and human risks, of economic costs and benefits and of other ethical and social issues. This is not only important for public opinion formation on nanofoods, but also for increasing sensitivity and trustworthiness of (scientific) risk assessments in the context of the uncertainties that seem inevitable with this new science and technology. The keyword in this context is learning, not least about the different perspectives and fundamental values that may be hidden behind concrete opinions on the health, environment and safety issues of nanotechnology. Looking at the results of our analysis of assessments and frameworks, a recommendation is to create and support networks on national levels that can overcome the organisational and institutional constraints that are an important cause for the weak links between the different kinds of assessments (scientific risk assessments, industrial foresights and economic analysis, public deliberation processes).
This study resulted in the following recommendations to policy makers:
1. By shaping a safe consortium or platform in which industry and other stakeholders can exchange views and information, or encouraging existing consortia and platforms to pay more attention to this kind of interaction and communication, a more mature ‘learning relationship’ with the (inevitable) ambivalence of citizen-consumers could be started off. Moreover, networks should be created on national levels that can overcome the organisational and institutional constraints that are an important cause for the weak links between the different social spheres in which nanotechnology assessments are taking place. Assessments should be seen not only as a way to inform decision makers, but as learning processes in itself. As such, they should be set up to also involve those involved in risk assessments, economic assessments and impact assessments, so that learning is facilitated across disciplinary boundaries. In order to be trustworthy risk and impact assessments need to be informed about the concerns of citizens and consumers.
2. There is a need for more application specific assessments. The large diversity of nanotechnology applications implies that specific assessments must be carried out for nanotechnology in food, packaging and agriculture. But even within this sector the diversity of applications is great. Assessments need to be tailored to specific issues within this broad field, and the participation and design of the assessments need to be adapted to the problem. This might involve setting up ad hoc assessment events, or encourage existing institutions to tailor assessments according to the specifics required by the topic. Such a problem/topic based approach will also by design avoid falling into the science trap identified by a number of authors; namely that assessments of emerging science and technologies become preoccupied with speculating on spectacular issues rather than addressing the most immediately relevant, but not so fascinating, applications.
3. There is a significant variation of the extent to which assessments address important contextual trends. This might be explained by the technology focus of most such assessments; the primary interest lies in describing the potential benefits and risks of the technology, as well as prescribing appropriate government responses to such. However, core contextual trends will impact on both the technology development, the magnitude and distribution of benefits and risks, and the ability to control these. Therefore we recommend that such trends are taken into account by assessors, and that decision makers request such contextual analysis when commissioning assessments.
3.3 CASE STUDY ON SYNTHETIC BIOLOGY
Synthetic biology is an emerging sub-discipline within the field of biotechnology, focussing on introducing engineering at various levels of biological systems. Although synthetic biology is still primarily a scientific endeavour, the huge potential and impact it could have on the economy, environment and society as a whole emphasises the importance of sound assessments. A review of a selection of current synthetic biology assessments on a number of aspects was elaborated, including issues related to purpose, applied methodological approach and created impact on policy making processes. Based on the identification of both strong and weak points within these assessments, some recommendations are suggested for future assessments of synthetic biology.
Of the in total 91 identified synthetic biology assessments, 11 assessments were selected for review and comparative analysis (4 German assessments, 7 European/international assessments). Criteria used for selecting assessments included geographical origin, methodological approach applied and analysed impacts. The reviewed assessments were conducted by various organisation types including research institutes and consortia, science and technology advisory organisations, NGOs, economic interest groups and funding organisations.
The assessments were systematically reviewed based on a developed analytical framework. Based on the assessment reviews, we would like to emphasise the following issues:
Concerning the aim and content of the reviewed assessments:
• The majority of assessments seem to be self-initiated; there is limited evidence concerning external commissioning. A main driver for many assessments is a need to position the respective organisation within the synthetic biology debate, or to react to active discussions within this field.
• The transparency and impartiality is in general good. Nevertheless, the reviewed assessments conducted by both NGOs and industry representatives seem to be less transparent and impartial compared to assessments conducted by other actor types.
• Considered impacts within synthetic biology seem to focus mainly on safety and security issues. However, issues related to economy, environment, society and health are also being discussed to a certain extent within most of the reviewed assessments. The topic of sustainable development does not seem to be very present in synthetic biology assessment.
Concerning the methodological approach of the reviewed assessments:
• Due to the emerging nature of synthetic biology, the majority of assessments are anticipatory. Considered time-scales within the assessments range from short term (<5 year) to long term (>15 year); however, none of the analysed assessments have applied structured tools for anticipation.
• Assessments are very much orientated towards scientific expertise. There is limited involvement of industry representatives, lay-people and policy makers in constructing synthetic biology assessments.
• The methodological approach of synthetic biology assessments is likely to be driven by the emergent nature of synthetic biology. The number of tools used within assessments, as well as methodological reflections, is limited. The majority of reviewed assessments rely to a large extent on literature review. In addition, some assessments use information derived from workshops, the majority of them being driven by expert judgement.
• Synthetic biology is seen as an international R&D quest, and is largely funded under existing biotechnology programmes.
Concerning the addressing of synthetic biology governance within the reviewed assessments:
• Although the majority of assessments state that synthetic biology governance could depart from currently active regulatory structures regarding genetic modification, it is often not clear how these structures are limited with respect to future synthetic biology activities and product development.
• The limited amount of observed regulatory activity might also relate towards the unclear situation regarding the need for market regulation versus liberalisation, as identified within the reviewed assessments.
• There is limited attention within the assessments concerning how obtained results have been, or could be, integrated within the overarching field of biotechnology governance. This situation may correlate with the limited policy impact of the reviewed assessments.
Although past assessments might have proven very useful and valuable at the time and within the context of their execution, the fast changing and dynamic field of synthetic biology might require different approaches for future technology assessments. At this point, there seems to be a need for improved dialogue between assessors and policy makers to clarify specific assessment needs that could improve policy relevance and policy impact. Our analysis implies that future assessments might need to consider more structured designs for anticipation, more inclusive participation of relevant stakeholders in methodological approaches and more systematic analysis of contextual variables and relevant trends.
Based on the synthetic biology case study results, a number of recommendations for policy makers are suggested:
• Be more pro-active in commissioning synthetic biology assessments: Since the majority of synthetic biology assessments seems reactive, in order to position an organisation into the synthetic biology debate, the link with and impact on policy making processes is not always evident. Governments could increase the effectiveness of assessments through communicating to assessing institutes policy maker needs with respect to synthetic biology governance, including the need to address relevant political, social, economic and scientific trends.
• Understand the time-scale of future synthetic biology developments: In order to communicate the needs for synthetic biology governance, policy makers need to have a clear understanding to what extent current governance structures apply to both current and future synthetic biology developments. A roadmap could be an instrument to anticipate decision making with respect to short- and long term synthetic biology developments.
• Stress the importance of public dialogue in assessing synthetic biology: Although a number of Science-in-Society research projects have been initiated by the EC to stimulate public dialogue concerning synthetic biology, the limited inclusion of lay people in assessing synthetic biology seems to limit the scope and applicability of such assessments.
• Synthetic biology is global: The international character of biotechnology needs to be appreciated in policy making. Activities with respect to governance outside the borders of Europe need to be closely monitored, in combination with the effect of the introduction of new governance approaches on the development of synthetic biology.
3.4 CASE STUDY ON BIOFUELS
Biofuel research and technology development has seen notable investment in the past decade at a European level. A proliferation of assessments has accompanied the intensification of research, although the body of assessments is very diverse and, often, poorly coordinated. A set of UK, European and international reports were analysed in a desk-study, interviews and workshop in order to examine the state of biofuel assessment and the potential for the application of integrated approaches.
One of the significant aspects of this case study is that it represents one of the few studies that has attempted to clearly characterise biofuel assessment approaches. Biofuels has developed significantly in the past decade and a proliferation of assessments has accompanied the intensification of research, although the body of assessments is very diverse and, often, poorly coordinated. Within the case study a set of UK, European and international reports were identified and analysed.
In terms of the assessment landscape, the assessments tend to focus upon environmental and, to a lesser extent, economic impacts with social impacts often overlooked. Most assessments position biofuel research and technology development in an ’opening’ stage and demand more knowledge and broader understandings of potential impacts. The assessments exhibited high scientific literacy and impartiality. Transparency of reporting varied amongst assessments and both participation and self-reflection are low. The body of biofuel assessments is diverse and the embedded assumptions often appear incommensurable. This indicated that it would be difficult to integrate findings.
The workshop findings confirmed that many practitioners consider the embedded and implicit assumptions in assessments to be a major problem for the body of biofuel assessments. The findings of the workshop suggest that a group could be established to deliver a report summarising the state of current knowledge about biofuels with particular reference to the embedded assumptions and methods. Participants suggested that the group should be independent but state-sanctioned, preferably at a European level. The procedural design and transparency of such a group would be a crucial factor in its legitimacy and success.
Based upon the findings of the analysis, recommendations were made for (i) assessment practitioners, (ii) research councils and (iii) policymakers. These primarily respond to the following problems identified in the body of biofuel assessments:
• Lack of transparency, affecting the utility of assessments for policymakers or further studies and also making the integration of findings difficult
• Insufficient attention paid to societal aspects
• Focusing upon specific technologies rather than specific problems
• Limited participation, particularly of non-experts, which may affect assessments’ sensitivity to different perspectives on technical impacts.
Recommendations for Assessment Practitioners:
• The desk study found that biofuel assessments are weak in the areas of transparency and self-reflection. The workshop findings confirmed that this failure to highlight and justify approaches, methods and boundaries negatively affects the utility of assessments, regardless of their scientific integrity (which is considered strong), and presents a serious barrier to the integration of knowledge. As such, we recommend that assessment producers consider the following practical points to improve the transparency and, ultimately, the utility of their work:
• Explicitly state the methods applied, justify their selection and examine their limitations. The assumptions embedded within these methods should be considered and described.
• Include as much data as possible (using annexes or on-line resources where necessary), stating any assumptions and clearly describing the analytical steps taken to produce the results.
• Explicitly state the boundaries of analysis. Consider the geographic scope, specific combinations of feedstocks and process technologies, and different types of impacts (e.g. GHG balances or societal aspects) to which the study is limited. Justifications should be provided where results are extrapolated beyond these boundaries.
• Consider the extent to which results depend upon the context of assessment, and how the results would be affected by, for example, new targets. While such changes are difficult to predict, clearly describing the data and methods applied could prolong the utility of an assessment which may otherwise be made obsolete as contextual features change.
• Reflect upon the limitations of the study and the conditions under which results remain valid.
• Participation, particularly of non-experts, is low in biofuel assessments. Although there are many forms of participation and participation per se may not appropriate for every study, increased participation may improve the legitimacy of assessments by incorporating the insights of publics and other stakeholders. This may also improve assessments’ reflexivity and strengthen their attention to societal aspects, each of which is also considered low.
• Consider how policymakers engage the academic literature and develop strategies to improve visibility, for example publishing targeted policy briefs on open access on-line sites.
Recommendations for Research Councils:
• The desk study, workshop and interviews reveal that assessments are insufficiently sensitive to social impacts. This is a difficult problem and is probably not limited to the study of biofuels. The situation may be improved by greater cross-domain or cross-funding council activity relating to biofuels, including grants for the examination of social impacts of biofuel development and support for dialogue across disciplinary boundaries. Some such activities are underway, cross-cutting events organised by the UK bioscience research council, BBSRC.
• Workshop participants find that the focus upon specific technologies such as biofuels, rather than specific problems such as land-use, makes it difficult to produce meaningful assessments which can adequately respond to contemporary challenges. This is exacerbated by the widely reported difficulties of publishing and building careers outside of disciplinary silos. The provision of funding to research the impacts of specific technological developments could discourage assessment practitioners from considering problems holistically. Research councils could respond by providing funding structures which encourage broad research into potential responses to challenges such as land-use change.
Recommendations for Policymakers
• The body of biofuel assessments is diverse and the embedded assumptions often appear incommensurable. This means it is difficult to integrate findings. The workshop participants suggested that a committee should be established to deliver a report summarising the state of current knowledge about biofuels with particular reference to the embedded assumptions and methods. It was suggested that the committee should be independent but state-sanctioned, preferably at a European level. The procedural design and transparency of such a committee would be a crucial factor in its legitimacy and success.
• Assessments are framed with reference to policy discourse. As climate change increasingly dominates policy discourse, GHG emissions increasingly dominate assessment spaces. Maintaining alternative discursive arenas such as rural development, energy security and land use may inspire assessments to consider a broader range of impacts. Even indicating a desire to consider more social perspectives on biofuel development, for example, may inspire activity amongst social scientists who are not otherwise not engaged with these issues.
• Policymakers could engage the academic community more systematically. Ad hoc approaches to the literature are not transparent and are unlikely to yield the best results. A closer relationship could be forged, for example, through partnerships with the teams of multi-disciplinary researchers working across topics or directly with research councils. The research community could improve their penetration by releasing policy briefs alongside their academic contributions or by running short policy-focused dissemination events.
3.5 CASE STUDY ON CLOUD COMPUTING
This case study described assessments in the field of cloud computing as well as their interrelations and their links with policy processes. The case study focused specifically on assessments influencing the European Commission's cloud strategy presented in 2012. The study's aim was to understand which kinds of assessments have played which role in the evolution of the field of assessments and to seek indications as to the formative effect – if any – the production of the cloud strategy has had on the assessment field.
As part of a broader investigation of “integration” in assessments and the possible need thereof, the case study focused especially on integrative functions in the field of assessment. Looking beyond methodology, the case study attempted to establish the formative effects of the cloud strategy production process on the assessment field as a whole as an “integrative” effect. The purpose was to lay the ground for a better understanding of the strategic situation facing any project attempting (normatively) to promote integration in assessments.
The main part of the case study took the shape of an outline of the field of assessments carried out under discourse-analytical inspiration. This part of the study shows that cloud computing assessments have historically grouped around seven “problem perspectives”, some of which are integrated under the strategic perspective of the cloud strategy. Importantly, efforts to promote and assess productive development paths alternative to those promoted by major industry actors are largely ignored. Instead, “conventional” perspectives such as security, privacy and legality are the main perspectives opposing industry hype, which are integrated in/by the political process. The main concerns driving integration of opposing evaluations of cloud computing, however, remains strategic and economical ones. “Integrative” perspectives in the broader sense of ethical/societal evaluations of the technology only emerge late in the process in reaction to political initiatives.
The study supplemented this assessment field analysis with analysis of process characteristics and purposes of individual assessments along with interrelations between assessments and relevant political documents.
In the course of the case study, seven consecutively emerging “problem perspectives” were identified from which cloud computing has been assessed since its emergence in the market in 2006. These were:
1. Individual strategy assessment (in reaction to hype)
2. Co-creative promotion of alternate development paths
3. Assessment of the technology's ontology, risk and reward
4. Security and privacy in a globalized world
5. Growth in a time of crisis
6. Societal strategy with regard to cloud computing
7. Integrative responsibility (in response to political actions)
One important example of such emergent relations between problem perspectives is that of “trust” emerging as a term bridging and combining the concerns of security and privacy assessments on the one hand and economic concerns on the other. “Trust” is neither technical, legal nor economic, but bridges all of these conceptual areas and provides a target for strategic coordination of perspectives, which might otherwise remain locked in opposition. As something of a dialectical creation, this concept acts therefore as a specific key to conceptual integration at a level, we might call “politico-epistemological”, i.e. it provides a simple framing (building trust in the cloud) for the complex of strategic problems facing decision-makers (balancing data security, privacy rights and carbon footprints against the primary and secondary economic benefits cloud computing seems to entail).
With regard to the question of integration, we observe an evolutionary maturation of the field of assessments as a whole and a continual widening of the range of topics dealt with in assessments. This is not so much due to systematic attempts at “integrative assessment”, but more due to mutual learning in the field and the fact that assessors read and digest already completed assessments. As such, the field of assessment has something of a closed circuit about it, with analysis points traveling from one assessment to the next.
To be sure, many possible points and perspectives are never represented or mostly ignored in assessments. We have mentioned for instance the absence of the perspectives of lay people and the marked silence in the field about alternative development paths such as open-source. Four interviewees stated explicit needs for further integration. Two of these expressed a need in the field for more stable platforms or methods for dialogue between stakeholders, where the power effects of major corporate actors shaping the dialogue could be neutralized. Two other interviewees expressed a need for better translation of “ethical” issues (a term they found more confusing than enlightening) into operable terms directly relevant to the topics under discussion in the wider process of societal appraisal of something like cloud computing.
Our reading of the field shows that it is possible to continually widen the range of topics being assessed. Such widening, however, takes place mainly by the promotion of issues through hype; through the forcing of issues through advocacy; or through the dialectical integration of opposing viewpoints. The only explicit attempts at systematically integrating a comprehensive spectrum of assessment topics in a single assessment we find in one ethical assessment and one technology assessment.
The main conclusions about the field of cloud computing assessments are:
1) The assessment field is dominated by assessments concerned with ontological and risk issues followed closely by strategic and economic assessments.
2) Assessments of privacy issues act as the main counterweight to these perspectives while ecological sustainability plays a minor role. Broader assessments of social sustainability and value discussions largely do not exist in the field.
3) Political action plans have a major formative effect on the assessment field.
4) Very few attempts exist at “integrative” assessment.
The main recommendations to policy makers consist in
• implementing deconstruction of framings in policy preparation, especially with regard to identifying business interests versus general societal interests
• seeking out visions alternative to those presented by business interests
• explicating methods for balancing / resolving conflicting viewpoints on new technology
• actively including citizens’ perspectives
3.6 FINDINGS ON INTEGRATION NEEDS ACROSS CASE STUDIES
See table 4 in the pdf version of the report.
This analysis importantly informed the process of identifying the criteria for our integrated assessment approach (presented in section 3.9 below).
3.7 FRAMEWORKS FOR ASSESSING SOCIETAL IMPACTS OF EMERGING SCIENCE AND TECHNOLOGIES
In this part of the work we collated and compared the work in the case studies and enriched this material with additional analysis of six established advisory domains: risk analysis, impact assessment, economic assessment, foresight, technology assessment (TA) and ethical assessment. In the case study and domain work all in all 1506 assessment reports were screened and 101 reports were reviewed according to the protocol. We also organised a practitioner workshop where we discussed with professionals involved in assessment about challenges and best practices of assessment. We documented that emerging science and technologies are assessed with a great width of methods and approaches. Several advisory domains are involved in producing such assessments and many assessments or reports are produced also outside the established advisory domains. Advisory domain assessments follow the conventions of their domains, and are imprinted with the cultures and traditions of their respective domains. Theses cultural assumptions influence the framing of the assessments, the choice of methods and how the assessments are reported.
Through the project’s analytic instruments we documented that the manifold of assessments have different intended purposes and functions. They frame their topics in different ways depending on their mandate, the traditions in the domains, the impact they intend to have, the participants included, etc. This means that there is a wide range of assessments answering different questions. We also documented that the advisory domains generally do not make use of systematic tools for situation analysis and method choice, though some such tools may exist. The domains also generally lack standards for transparent reporting of situation analysis, including framing assumptions, method choice and the unfolding of the assessment process.
We found that all domains have a focus on methodological development, and try to tackle the complexities and uncertainties implied by emerging science and technologies. However, they do not often discuss such challenges with practitioners from other fields. There are sometimes collaborative assessment efforts across the domains, for instance between TA offices and ethics committees. However, most often the domains do not collaborate to solve methodological challenges. Even if there are many overlaps between the domains in terms of both topics and methods, there is a lack of communication between the domains. There seems especially to be a communication gap between ethicists and economists in the context of emerging science and technologies. From the project practitioner workshop we learned that such cross-domain learning was appreciated.
Assessments from outside the established advisory domains may be just as important for policy as those from inside. However, we argued that there is a risk that these have less institutionalised mechanisms for methodological reflection and learning. Legitimacy of the input, methods and output is important for assessments coming from both inside and outside the established advisory domains. In this study we found no institutionalised mechanisms for judging the quality and legitimacy of assessments that influence policy. Moreover, there was very limited transparency with regard to the impact of the different assessments on policy.
Several integrated methods exist, but from the 101 assessments reviewed in this project, very few could be characterised as integrated on several dimensions. Moreover, for those assessments that scored high on several dimensions relevant for integration, it was generally hard (though with some notable exceptions) to trace this back to systematic methodologies. From the project practitioner workshop and from the literature studies we learned that there is no established understanding of what the right approach to achieving integration is. However, de facto integration happens when translating lessons from the assessments into practical EST policy. There is evidence from the project’s case studies and discussions with end users that this translation process is not entirely transparent.
In the analytic work of the project we found that there is no comprehensive acknowledgement and transparency of the value-laden assumptions underlying EST assessments. In many cases of EST there is no ‘innocent’ starting point for assessment; a starting point that is not contested. This applies to risk assessment, to economic assessment, and to ethical assessment alike, as well as to all other assessments.
There are many EST assessments focusing on specific EST dimensions in a general way, such as health, safety and environmental concerns (HSE) of nano particles, security concerns of ICTs, ethical concerns of synthetic biology, economic projections of bioenergy, etc. These are important for an initial mapping of issues. We identified fewer assessments trying to tackle concrete EST related policy problems, such as specific applications in a specific geographical context. With increased practical problem orientation comes increased complexity in variables and a need to consider all the above dimensions at once, with corresponding problems of delimiting the adequate scope of assessments. These are topics that need to be addressed in order to facilitate responsible technology governance in practice.
The work generated a number of recommendations to policy makers and assessment professionals:
1 Assessments should be transparent in their framing of the topic, situation analysis, method choice and practical process development. Only in this way is it possible to assess the quality and legitimacy of the assessments in their function as providing an evidence base for policy. Policy makers thus need to request such transparency. Transparency guidelines should be developed for advisory reports in general.
2 There should be increased interaction between the advisory domains in order to enhance learning and facilitate extended peer review. Such interaction should be facilitated by institutionalised instruments.
3 There should be increased focus on developing methods for tackling EST issues that have become practical policy issues. This inherently involves interdisciplinarity and broader involvement of stakeholders and/or the public. Policy makers should request such problem oriented assessment before the problems become acute.
4 In most cases, before a field of assesment reports can come to function as an evidence base for policy, some form of integration of the main lessons from these reports will be necessary. The legitimacy of this evidence base will increase when such integration is done in dialogical and problem-oriented interdisciplinary processes. Policy makers should work to institutionalise such forms of integrated assessment.
3.8 THE CURRENT AND FUTURE CONTEXT FOR EST ANALYSIS
Technology advising takes place in larger societal contexts and the study of these contextual developments were considered important in EST-Frame. These contexts can be understood as influenced by some significant mega/macro trends, as well as a variety of trends and developments at a meso and micro level. Assessment of emerging science and technologies must respond to such trends in order to be well targeted and policy relevant.
In this work macro trends were identified and analysed, with the most important aspects examined in more detail in order to consider the implications for technology advisory practices. The trends analysed were selected based on a study of reports from seven larger trends studies. The trends and contextual factors that were identified as the most significant were those that are:
a) Important to consider when designing, carrying out and evaluating assessments; and
b) Important for developing a framework for integrated assessment that is capable of being responsive to such developments.
The trends were profiled as:
1. Liberalisation and globalisation
2. New governance networks
3. Public-private partnerships (PPPs)
4. Citizen empowerment and public deliberation
5. Rapid technological change
6. Focus on sustainability and climate change
7. Economic change
8. Quantification
9. Policy integration as a response to the identified trends
In particular our analysis has examined whether or how the identified assessments (in the case and domain studies):
a) discuss the effect of the trends on technology governance
b) adapt their methods to the trends in order to have more impact on technology governance
c) are transparent about their assumptions regarding the future and how the trends influence any of the topics they address ( for example, ethical issues, economic projections, technology development, etc.)
The main lessons and recommendations for policy makers that emerged from this work are:
Greater levels of market liberalisation and public-private partnerships (PPP) would appear to indicate increased importance of economic assessments in decision-making. However, economic assessments are not dominating any of the chosen case studies, and PPPs are hardly mentioned. If it is the case that liberalisation is an important contextual factor then the effects of market liberalisation on responsible technology governance should be discussed more widely. Moreover, the economic assessments that underlie policy on PPPs should become more transparent and scrutinised as part of the assessments in EST fields. As liberalisation potentially affects the boundaries of technology governance assessments should discuss this reflectively to a larger extent.
New governance networks, especially on an international level, are often considered in assessments. However, assessments are rarely conducted at a global level. Because of the global character of the current grand societal challenges, there should be increased infrastructure and competence building for assessment at a global level.
Citizen empowerment is an important trend and this has typically been an important element within technology assessment (TA), but also impact assessments include some wider involvement. However, systematic infrastructure or instruments for the involvement of lay people and a broader range of stakeholders in all assessments seem to be lacking, as does funding instrument and capacity building initiatives for enabling the participation of potentially marginal or socio-economic weak groups (for instance consumer and societal organisations without much financial resources). Standards for transparency in such involvement processes, in order to be able to assess the nature and quality of the involvement, would be useful.
Rapid technological change is a trend that implies that significant uncertainties may result, both with regard to environmental and human health risks, and with regard to societal and economic impacts. These uncertainties must be characterised and applying the precautionary principle must be considered. Moreover, such rapid change also affects important societal values. These must be considered and appropriately addressed in an anticipatory way, so that societal, value based technology governance can be carried out at an appropriate early stage.
Increased focus on sustainability is seen in some technology fields, such as bioenergy. However, sustainability – and especially its social pillar - can be better operationalised. Sustainability needs to be addressed as an integrative concept, and not split into separate ‘silo’ assessments that are not integrated. If sustainability is to be an important policy goal, better tools for integration of knowledge need to be developed. Even at a time of economic crises, the balance of ecological, economic and social concerns need to be considered. In such balancing, quantification has a place, but it must be scrutinised and placed within a wider perspective.
Policy integration is a response to trends (such as liberalisation) that appear to reduce the opportunities for state steering. Integration in assessment can facilitate policy integration and, to the extent that assessments influence policy, transparency about assumptions, methods and the practical aspects of the assessment process is crucial for the legitimacy of the policy.
Overall, we have found that the policy trends may have significant impact both on the development of EST, the possibilities for responsible governance of EST and the setups and practices of EST assessment. This merits devoting much more attention to the trends in EST assessment in general and especially in assessment processes that intend to integrate the assessment evidence base in a field into practical, multi-dimensional policy recommendations.
3.9 CRITERIA FOR AN INTEGRATED FRAMEWORK
From the case and domain studies, dialogues with assessment professionals and stakeholders, and the analysis of the integration dimensions, the EST-Frame project was able to better diagnose the needs for integration currently unmet in the EST field. These basic needs were described as criteria for an integrated framework and were subsequently used for developing the integrated approach in the project.
The three basic criteria for an integrated framework were identified as:
a) It needs to be focused on governance problems. The project’s case studies and domain studies showed that rich, problem-oriented assessments were scarce. This problem does not necessarily need to be a topic on the political agenda. It can well a topic that important stakeholders believe should be subject to policy making.
b) It needs to give guidance on how assessments in a larger body of assessments relevant for a particular problem, can be integrated. It is noted that integration of existing assessments may be facilitated by convening an assessment team consisting of assessment practitioners from the different assessment traditions.
c) It needs to provide guidance on how to ensure sufficient transparency in assessments for such problem oriented integration can take place. If problem-orientation, trans-domain interaction and integrating lessons from existing assessments are to be done, the assumptions of the different domain representatives and assessments need to be transparent. Assessments with incompatible assumptions may not be possible to integrate. Moreover, the situation analysis and method choice of the integrated process must be explicit and reflective.
These criteria were used to develop the integrated approach that was tested in the project, the Integrated EST Framework, ultimately finalised as the Trans-domain Technology Evaluation Process, TranSTEP.
From the work on the criteria for an integrated assessment framework we developed the following recommendations to policy makers:
1. In order to facilitate responsible research and innovation emerging science and technologies must be assessed in their practical contexts of use, taking into account the richness of impacts that appear in such concrete situations. Trans-disciplinary and trans-domain assessments must be carried out in order not to fragment complex real-life situations into generalised, abstract reductions.
2. Transparency of all assessments – also disciplinary assessments - is necessary for their inclusion into the evidence base for technology policy. In order to know whether an existing assessment can provide valid and relevant knowledge for solving the governance problem at hand the situation analysis and method choices must be justified and transparent. The EST-Frame project recommends that all assessments of new technology clearly show their situation analysis and method choices.
3. Assessment institution directors and managers should increase their strategic focus on the development of "home-grown" approaches to problem-oriented transdisciplinary research, to develop transdisciplinary competences, to foster connections and interaction with other assessment domains, and to secure transparency in assessments with regard to situation analysis, dialogue and method choice.
4. Assessment commissioners, for example in European DGs and members state ministries and agencies, should help to foster problem-oriented transdisciplinary assessments by implementing an approach such as the Integrated EST Framework as a way of securing transparency with regard to situation analysis, dialogue and method choice in assessments and assessment-based policy-development.
5. Policy developers in European DGs and member state ministries and agencies should work to secure transparency in the use of assessments in policy-development through clearer presentation of the interpretations made of assessments and the conclusions drawn. Policy makers must ensure that the evidence base for EST related policy-making is integrated in a transparent and balanced way, taking into account the different framings, methods and approaches of the assessments making up the evidence base.
6. European and member state policy makers should work to secure the implementation of responsible research and innovation in the Horizon 2020 program. Such requirements would place demands on assessment researchers, encouraging that they apply the quality criteria of problem-orientation and transdisciplinarity and that assessment research is carried out in ways which secure transparency with regard to situation analysis, dialogue and method choice.
3.10 THE TRANSTEP APPROACH - TRANS-DOMAIN TECHNOLOGY EVALUATION PROCESS
The main outcome of the EST-Frame project is the TranSTEP approach. This was thoroughly tested and discussed with assessment practitioners, commissioners and stakeholders in altogether six workshops. The motto for this approach is: Get an integrated perspective on complex technology issues by teaming up across established assessment communities. The approach is described in the following way:
TranSTEP is a conceptual assessment approach targeted towards policy makers commissioning assessments and professionals conducting assessments.
TranSTEP is an approach to the assessment of technologies or technological applications that present challenges related to complexity, uncertainty and controversy over facts and values. In such situations the legitimacy of any assessment may be challenged with respect to who participates, how the assessment is conducted and the quality and meaning of the results. TranSTEP offers an approach where the range of participants is widened, the assessment process itself is made transparent and the output has been subject to broad review. In this respect TranSTEP aims to confront the difficult discussions as an integrated part of the assessment, conducive to better robustness and legitimacy of its output.
TranSTEP is a conceptual guide for practical work. It gives commissioners and facilitators the necessary leeway to blend tools and methods from different assessment traditions in a flexible manner without sacrificing transparency and accountability in the process.
TranSTEP focuses on the enhancement of communication and interdisciplinary learning between different domains of expertise, because fragmentation of expertise is one of the main barriers to integrate factual evidence, values and normative perspectives across these domains.
Using the TranSTEP approach involves initiating and facilitating assessment groups composed of people from different assessment backgrounds to integrate assessment perspectives on complex technology issues. Participants in such processes can be assessment practitioners from domains such as economics, risk assessment, ethics, foresight, impact assessment or technology assessment, or from outside these domains. This is why TranSTEP is called a trans-domain approach. The various perspectives to be integrated are found in the participants’ professional background. In order to ensure that all relevant perspectives are brought in, other actors should also be involved, such as representatives from industry and public research, private sector stakeholders, public sector decision-makers or administrators, NGOs or the public.
The breadth of possible participation, and the focus on learning, requires a high degree of process design reflexivity amongst the assessment initiators and participants. TranSTEP involves collaborative situation analysis, transparent method reflection, and a continual use of facilitated dialogue.
An important part of the process is to assess existing knowledge and identify gaps where the knowledge base may be improved. The work may also involve initiating and carrying out new activities in order to strengthen this base. The purpose is to integrate existing knowledge that is compatible with the group’s problem formulation into an evidence base (where evidence must be understood in a wide sense, also potentially including input from the public) for responding to this problem. The approach ultimately aims at the production of an integrated assessment by the assessment group with a firm rooting in science, society and policy.
In order to disseminate this approach in a more user-friendly way than a report we created a designated website: http://transtepapproach.wordpress.com/. Please consult this website to find the details of the approach.
We are dedicated to disseminate this approach widely in order to increase its impact. Please see the next chapter for details on the further exploitation of these results.
Potential Impact:
4.1 POTENTIAL IMPACT OF THE PROJECT
As shown above the project has had outputs of different character; new knowledge, analytic instruments and a practical approach for integrated assessment of emerging science and technologies. The potential socio-economic impact of these activities are hard to estimate, let alone quantify. Here we will instead qualitatively describe the potential impacts of the project’s output and the measures we have taken to increase them.
A. The impact of an analytic framework to evaluate assessments and provide internal quality control
The comparative assessment in WP1 contributed to a better understanding of the respective roles of the various technology assessment frameworks. This work resulted in an analytic tool (the process characterisation table) for better appraising the assessments of emerging sciences and technologies and for more reflected use of assessment frameworks (see table 1 above). This tool was used as an analytic framework in the project and can also be used in future research and to improve the quality of assessments. In order to increase the impact of this tool we have disseminated it widely, both in Deliverable 1.1 and in Forsberg et al. 2014 (see list of the project’s publications below).
B. Better understanding of the assessment of governance of the selected case technologies
The case studies and the processes testing the integrated framework on the technology cases has provided more holistic understanding of these respective technologies; the main issues to consider, further assessment needs, and governance needs. The impact of these case studies have been strengthened by the inclusion of key stakeholders in two workshops in each case study (confer sections 3.2 to 3.5 for the outcomes of these studies). The results of these studies have been disseminated in Deliverables 2.1 3.1 4.1 and 5.1 and in De Bakker, De Lauwere, Hoes and Beekman 2014, Van Doren and Heyen 2014 and Boucher, Smith and Millar 2014.
C. Better understanding of how EST assessments are conducted and how they respond to a dynamic societal context
The systematic comparison of different advisory domains and assessment frameworks, as well as the cross-domain dialogues organised in the project, have facilitated a better understanding of the common challenges facing assessment practices and practitioners. A potential longer-term impact of taking such a comprehensive perspective may be to professionalise assessment practices across domains. The trends study allows such practices to better respond to a dynamic environment. These cross-domain analytic frameworks may be important for further quality control of EST assessment practices. In order to increase this potential impact we have disseminated this work in Deliverables 1.1 and 1.2 as well as in Forsberg, Thorstensen, Nielsen and De Bakker 2014 and Van Doren, Forsberg and Lindner 2014.
D. Integrated assessment of four technology cases
During the work in the project we organised an integrated assessment workshop for each case study. These workshops were not full implementation of TranSTEP, but workshops testing the initial phases of an integrated assessment process. These workshops pointed to specific actions that need to be conducted in the future.
The workshop on integrated assessment of cellulosic ethanol pointed to the need to take further actions on the following issues:
• Actions must be taken to reduce scientific uncertainty over crucial sustainability issues of biofuels and uncertainty over long-term performance and cost of advanced biofuels.
• Advanced biofuels, such as cellulosic ethanol, need to be considered against different alternatives and competing uses of biomass need to be further explored.
• Methodological challenges involved in the assessment of the impacts of first-generation ethanol seem to remain valid for the case of cellulosic ethanol. A transdisciplinary system-wide approach is needed.
The possibilities for following-up on these action points are being explored by the University of Nottingham team.
The integrated assessment workshop on nano food ended up in recommendations for the follow-up actions to be taken by the Dutch consortium NanoNextNL. NanoNextNL is a consortium of more than one hundred companies, universities, knowledge institutes and university medical centres, aimed at research into micro and nanotechnology. Noting the problem of the ‘waiting game’ (an unresolved situation where no one wants to be the first mover) the consortium was advised to organise a ‘ societal incubator’ where nano food industry could discuss with key societal stakeholders in a protected sphere. This is now being implemented. Inspired by the activities of EST-Frame NanoNextNL has asked the Rathenau Institute (a Dutch Technology Assessment agency) to write a proposal for the development a ‘societal incubator’ that could explore and support a broader dialogue with societal stakeholders.
The integrated assessment workshop in synthetic biology ended up focusing on a specific case example, genetically modified/engineered mosquitoes and flies using DNA-synthesis. This was given a definite problem framing and a list of questions the envisioned integrated assessment should tackle. The assessment was not completed, but the design for follow-up actions were made and the participants appeared willing to continue to engage in this process.
The integrated assessment workshop on the European cloud strategy concluded with two recommendations:
- An integrated assessment process concerning the European Cloud could pave the way for the development of a set of guidelines to defined groups of stakeholders to apply cloud-based solutions for certain sets of services. If such development was undertaken, it should be matched by strategies for the continued development of certification schemes based on open standards.
- Europe’s recent experiences in connection with data protection and privacy and the difficulty of governing these in a digital society have brought attention back to a long standing need to develop strategies and capacities for the adequately flexible development of regulation. An integrated assessment process could explore ways of developing practical capabilities for more responsive forms of ongoing societal regulation of data use.
Follow-up actions on the integrated assessment of the European cloud strategy were desired by the participants and are planned by the Danish Board of Technology.
E. An integrated approach to EST assessment: TranSTEP
The development work in WP6 has provided an integrated framework, TranSTEP, conducive to a better and more balanced assessment of emerging sciences and technologies.
The potential impacts of using a TranSTEP approach for EST assessment are:
- Better understanding of complex technology issues: The TranSTEP approach helps to ensure that the rich nature of complex technology issues are captured.
- Better understanding of one’s own and others’ assumptions, view and values: Through the systematic situation analysis and method reflection undertaken in a TranSTEP assessment, public and private decision makers and stakeholders will better understand other actors’ assumptions, views and values. The learning that results from actually working together in the TranSTEP group will likely influence the participants’ own understanding of the issue and of their own assumptions and default working methods.
- Better legitimacy and trust for advice and decisions: The nature of complex technology policy issues means that facts and values often intertwine and will frequently be contested by experts, stakeholders or the public. If such contestation is invited into the assessment group from the start, the assumptions of the assessments are likely to be much more broadly deliberated and thus more robust when presented to a wider audience.
- Increased transparency in assessments: TranSTEP provides explicit situation analysis and method reflection, which will make it easier for those outside the TranSTEP group to evaluate the assessment and the integration of results from TranSTEP. Explicit situation analysis, method reflection and assessment design are important quality criteria for assessments in general and enhance the ability of decision makers, stakeholders and the public to evaluate whether they agree with the assumptions of the assessment and corresponding results, and will therefore facilitate integration of assessments in the future.
- That the most urgent issues are addressed: Addressing a broad range of perspectives BEFORE designing assessment strategies may help to tailor assessments that target the most urgent issues, and not only the issues that established domains most easily can address. This may increase the efficiency of assessment efforts, by avoiding spending time and money on resolving issues that are not the real problem.
The project has taken the following measures to promote the use of this integrated framework model by advisors and policy makers when making assessment decisions:
- It was presented to European policy makers, stakeholders and researchers at the project’s final conference in Brussels in December 2nd 2014
- We have built up awareness of our approach by engaging end users, researchers and stakeholders in workshops throughout the project, informing on the websites and presenting preliminary results in conferences (see the dissemination section below)
- We sent out an email informing about our results to all the end users, stakeholders and researchers we have been in touch with in the project, approximately 120 persons (December 2014)
So far (February 24th 2015) the TranSTEP website has had 3 500 views, from 277 individual visitors.
4.2 A EUROPEAN APPROACH
This project required a European approach because technology developments in the EST field are global and are primarily treated strategically at a European level. It would be hard for a European country to develop a national strategy on, for instance, synthetic biology, in isolation of the developments in Europe because the technology developers are globally connected and must be met with harmonised international governance approaches. The European Commission funds developments in these fields and the assessments coming out of the kind of integrated approaches proposed here will be directly relevant for European research funding policies (e.g. in the framework programmes). Being thoroughly anchored in European assessment communities’ understanding of challenges in EST assessment means that TranSTEP is tailor made for the European policy context.
4.3 ASSUMPTIONS ON THE SUCCESS OF THE PROJECT
The EST-Frame project has evaluated the way assessment of emerging science and technologies is carried out and provided at the end of the project period recommendations for new practices. We could not expect to see changes in assessment practices already at the end of the project period. Success will rather have to be measured in terms of the support expressed for the results of the project and the stimulation of follow on activities. As the design of the project has to a large extent been bottom-up, with extensive user involvement we anchored our integrated assessment approach with a number of relevant potential end users already as a part of the development process, securing a relatively sound basis of support.
Some quotes from the project’s last end user workshop (Oslo, October 2014) confirms this:
● “Is it revolutionary? Yes, perhaps, because the concept might consider different framings from different problem owners and therefore might be able to internalise plurality and different perspectives, which is powerful.”
● “The approach appears to be welcomed by potential end users, which is great.”
● “An approach like TranSTEP can help deliver transparency in the process of problem formulation, with the right set of people involved & a common understanding (if not agreement) can be achieved.”
The feedback on the need for an integrated approach such as the TranSTEP approach from the Integrated Assessment testing workshops also provides an indication of support:
The average score on the statement ‘There is a need for an integrative process such as the Integrated EST framework in this field’ was in the synthetic biology workshop 4,2 (on a scale from 1 to 5 where 1 was ‘completely disagree’ and 5 was ‘completely agree’). The average score on ‘There is a need for an integrative process such as the Integrated EST framework in other fields’ was in the same workshop 4,1. The scores were identical in the biofuels workshop. In the cloud computing workshop the scores were, respectively, 3,7 and 4,3. In the nano food workshop the average scores were, respectively, 4.3 and 4.1.
All in all, this allows us to claim that the participants that have been involved in the discussions and testing of the EST-Frame integrated approach seem to support this concept. The feedback we received at the project’s final conference also seemed to confirm broad support for the TranSTEP approach.
4.4 MAIN DISSEMINATION ACTIVITIES
The project has had the following targeted audiences:
A. Policy makers
B. Assessment commissioners
C. Assessment practitioners
D. Technology innovators
E. Stakeholders
F. Researchers (science, social science and humanities)
G. The general public
Policy makers, assessment commissioners, assessment practitioners, stakeholders and researchers have been extensively involved in the project through the project’s workshops and conference. The general public has been informed mostly through the public website and articles in the media.
The project’s workshops and conference
An important dissemination action has been all the workshops organised throughout the project.
- November 1st 2012: A Dutch expert workshop on assessment of nano food took place in Utrecht with six external assessment experts/stakeholders.
- November 9th 2012: A German expert workshop on assessment of synthetic biology with seven external assessment experts and stakeholders took place in Karlsruhe.
- November 14th 2012: A Danish video conference two external assessment experts and stakeholders took place.
- March 7th 2013: A British expert workshop on assessment of biofuels, with seven external assessment experts and stakeholders took place in London.
- April 16-17 2013: A project practitioner workshop with 32 participants (including EST-Frame researchers) was held in Copenhagen.
- February 19-20 2014 : “The Integrated EST Framework and Its Application to Synthetic Biology”, integrated assessment testing workshop with 11 international participants, was organised by Fraunhofer ISI, Berlin,
- March 10-11 2014 : “A transdisciplinary dialogue on the opportunities and challenges of cellulosic ethanol in the UK”, integrated assessment testing workshop with eleven external participants, organised by University of Nottingham, Manchester
- March 26-27 2014 : “Integrated appraisal of ‘nanofood’ in the Netherlands”, ”, integrated assessment testing workshop with 13 external participants was organised by LEI in Driebergen,
- March 28 2014 : “EST-Frame Workshop on Integrated assessment of emerging technologies”, organised by HiOA, Brussels,
- April 31-May 1 2014 : “Testing the integrated framework in the case of cloud computing”, integrated assessment testing workshop with eight external participants organised by the Danish Board of Technology, Copenhagen
- October 29-30 2014: The project’s second general practitioner workshop with 12 external participants, was organised in Oslo by HiOA
- December 2 2014: Final conference on integrated assessment of emerging science and technologies, with approximately 45 participants, Brussels.
Public deliverables
All public deliverables are accessible through www.estframe.net:
- EST-Frame deliverable 1.1 Frameworks for assessing societal aspects of emerging science and technologies.
- EST-Frame deliverable 1.2 The current and future context for EST analysis.
- EST-Frame deliverable 1.3 Criteria for an integrated analytical framework.
- EST-Frame deliverable 2.1 Nano food: technology, governance context and assessments
- EST-Frame deliverable 3.1 Synthetic biology: technology, governance context and assessments
- EST-Frame deliverable 4.1 Biofuels: technology, governance context and assessments
- EST-Frame deliverable 5.1 Security and emerging ICTs: technology, governance context and assessments
- EST-Frame deliverable 6.6 An integrated framework for assessing societal impacts of emerging science and technologies. Website: http://transtepapproach.wordpress.com/
- EST-Frame deliverable 6.7 An integrated framework for assessing societal impacts of emerging science and technologies.
Scientific articles
• Forsberg, E-M., Quaglio, G., O’Kane, H., Karapiperis, T., van Woensel, L. and Arnaldi, S. 2015. Assessment of science and technologies: Advising for and with responsibility. Technology in Society, 42: 21-27.
• Reiss, T. and Millar, K. 2014. Introduction to special section: Assessment of emerging science and technology: Integration opportunities and challenges. Science and Public Policy, 41: 269-271
Link: http://spp.oxfordjournals.org/content/41/3/269.abstract.html?etoc
• Van Doren, D. and Heyen, N. B. 2014. Synthetic biology: Too early for assessments? A review of synthetic biology assessments in Germany. Science and Public Policy, 41: 272-282
Link: http://spp.oxfordjournals.org/content/41/3/272.abstract.html?etoc
• Boucher, P., Smith, R. and Millar, K. 2014. Biofuels under the spotlight: The state of assessment and potential for integration. Science and Public Policy, 41: 283-293
Link: http://spp.oxfordjournals.org/content/41/3/317.abstract.html?etoc
• De Bakker, E., de Lauwere, C., Hoes, A-C. and Beekman, V. 2014. Responsible research and innovation in miniature: Information asymmetries hindering a more inclusive 'nanofood' development. Science and Public Policy, 41: 294-305
Link: http://spp.oxfordjournals.org/content/41/3/294.abstract.html?etoc
• Forsberg, E-M., Thorstensen, E., Nielsen, R. Ø. and de Bakker, E. 2014. Assessments of emerging science and technologies: Mapping the landscape. Science and Public Policy, 41: 306-316.
Link: http://spp.oxfordjournals.org/content/41/3/306.abstract.html?etoc
• Van Doren, D., Forsberg, E-M, and Lindner, R. 2014. Are assessments responding to a dynamic environment? Evidence from four emerging techno-scientific domains.
Science and Public Policy, 41: 317-331.
Link: http://spp.oxfordjournals.org/content/41/3/317.abstract.html?etoc
• Forsberg, E-M. and de Lauwere, C. 2013. Integration needs in assessments of nanotechnology in food and agriculture. In Etikk i Praksis. Nordic Journal of Applied Ethics, 7, 1: 38-54: http://tapir.pdc.no/index.php?el=Kapittel&p=EIP&seks_id=77394
• Beekman, V. and Dagevos, H. 2013. The knowledge society as pleonasm: towards mobilisation of social intelligence in the agricultural and food domain. In: H. Röcklinsberg and P. Sandin (eds.) The ethics of consumption: The Citizen, the market and the law, Wageningen Academic Publishers, 353-357
• E-M Forsberg. 2012. Integrated assessments of emerging food technologies - some options and challenges. In Potthast, T. and Meisch, S. Climate change and sustainable development. Ethical perspectives on land use and food production. Wageningen Academic Publishers.
Conference/seminar panels and presentations
- Erik de Bakker and Ellen-Marie Forsberg were part of a workshop on integrated assessment at the Snet 2012 conference, Enschede, 22-25 October 2012.
- Erik de Bakker, Volkert Beekman, Anne-Charlotte Hoes and Carolien de Lauwere gave a paper presentation on the topic Nanotechnology in food: public concerns, narratives and assessment needs at the Snet 2012 conference.
- Ellen-Marie Forsberg presented EST-Frame results at a workshop on ‘Changes in ethical, social and environmental assessments of science and technologies’ at the Nordic STS conference, organised by our ‘sister project’ EPINET, Trondheim, 24 April 2013: http://www.ntnu.no/documents/10265/18878035/sessions+on.pdf/e262a217-166b-4c64-a972-ce54b45e9ed7
- EST-Frame had a workshop at the PACITA conference on Technology Assessment, March 15th 2013, 09.45 Prague: Integrated assessment of emerging science and technologies. The workshop consisted of a presentation of results from the EST-Frame case studies, as well as a presentation of our preliminary approach to integrated assessment. The workshop was chaired by Professor Frans Brom from the Rathenau Institute. From EST-Frame participated Philip Boucher, Anne-Charlotte Hoes, Nils Heyen, Rasmus Øjvind Nielsen and Ellen-Marie Forsberg. See description: proposal_workshop_pacita_conference_est_frame.docx. See also http://pacita.strast.cz/en/conference/general-info.
- Volkert Beekman presented a paper (co-authored with H Dagevos), The knowledge society as pleonasm: towards mobilisation of social intelligence in the agricultural and food domain, at the 2013 Conference of the European Society for Agricultural and Food Ethics (EurSAFE), Uppsala, September 2013; Kate Millar was also in attendance in Uppsala
- Ellen-Marie Forsberg gave the presentation ‘Sustainability in Assessment of Emerging Science and Technologies’ at the 2013 AP Safe Conference, Bangkok, Thailand, 29.11.13
- Ellen-Marie Forsberg gave the presentation ‘The need for integration in assessing emerging science and technologies: Results from the EST-Frame project’ at the Workshop on Responsible Governance of Science and Technologies, at the European Parliament STOA panel, Brussels 19 March 2014
- Thomas Reiss gave the lecture “Integrierte Bewertung neuer Technologien am Beispiel der Synthetischen Biologie“ (integrated assessment of emerging technologies - the example of synthetic biology) at the University of Frankfurt, Germany, on July2nd, 2014
- Ellen-Marie Forsberg gave the presentation ‘The potential for an integrated assessment framework for new technologies’ at Europäische Akademie, Summer school ‘Analysing the Societal Dimensions of Synthetic Biology’, Berlin 19.09.14
- Ellen-Marie Forsberg gave the presentation ‘Responsible Research and Innovation and assessment of emerging science and technologies’ at the EU Spri 2014 Conference, Manchester, 20.06.14
- Erik de Bakker presented a paper (co-authored with Ellen-Marie Forsberg), ‘Responsible Innovation & Innovation Ethics: Advisory Domains and Circles of Legitimacy’, at the European Business Ethics Network (EBEN) Annual Conference, June 12th-14th 2014 in Berlin.
- Rob Smith presented at the Biennial Conference of the European Association for the Study of Science and Technology (EASST), 17-20 Oct 2012
- Rob Smith presented at the 4th Annual meeting of The Society for the Study of Nanoscience and Emerging Technologies (S.NET) 22-26 Oct 2012.
- Kate Millar and Ellen-Marie Forsberg attended and participated in the Rome SIS-RRI Conference: Science, Innovation and Society – Achieving Responsible Research and Innovation; 19-21 November 2014
- Barbara Ribeiro presented a paper for Horizon Digital Research Economy, University of Nottingham, Assessing the impacts of emerging technologies: Which way forward?, 10 April 2014
- Barbara Ribeiro presented a paper on ”Social sustainability of ethanol and responsible innovation” at the Department of Science and Technology policy, University of Campinas, Sao Paulo, Brazil, 15 August 2014.
- Barbara Ribeiro presented a paper co-authored with the Nottingham team (Ribeiro, B.E. Smith, R., Millar, K.) on “Governing and assessing the impacts of emerging technologies: Which way forward?”, 4S Conference, Buenos Aires, Argentina, 23 August 2014.
Internet and social media:
We have also disseminated the project in various media and arenas:
- The project has an updated internet site: www.estframe.net and facebook site: https://www.facebook.com/pages/EST-Frame-project/126774937460322
- The outcome of the project, the TranSTEP approach, is disseminated on a dedicated website, designed to enable broad uptake of the results: https://transtepapproach.wordpress.com/
- A flyer is developed describing TranSTEP in short: http://estframe.net/publications/content_1/text_721891ce-f43b-460e-80ed-339c02c7134d/1417094492092/transtep_in_a_nutshell.pdf
- A short ‘elevator pitch’ of TranSTEP is published on www.estframe.net: http://estframe.net/publications/content_1/text_721891ce-f43b-460e-80ed-339c02c7134d/1417095522034/transtep_elevator_pitch.pdf
- A flyer was developed at the start of the project. This has been handed out on several occasions and is often attached when contacting people about the project. See http://estframe.net/publications/content_1/text_721891ce-f43b-460e-80ed-339c02c7134d/1342029518319/estflyer_rgb_final_for_ordinary_printer.pdf
- The project was presented in EurSafe news, vol. 14, no. 1, April 2012, www.eursafe.org
Media presentations
- Article about the EST-Frame project at the HiOA wepages: http://www.hioa.no/Aktuelle-saker/Stor-interesse-for-HiOA-ledet-EU-prosjekt
- EST-Frame final conference was reported on the Bassetti Foundation website: http://www.fondazionebassetti.org/en/focus/2014/12/on_december_2_the_est-frame.html
- The Norwegian Government’s delegation to the European Union reported from the project’s final conference: https://www.regjeringen.no/nb/aktuelt/Anbefaler-mer-forskningsdialog/id2344459/
- Article on the STOA event at the HiOA webpages: http://www.hioa.no/News/AFI-forsker-informerer-EU-parlamentet
4.5 EXPLOITATION OF RESULTS
Further dissemination
We will continue to disseminate the results of the project even after its closure.
Scientific publications: The TranSTEP approach and outcomes from the empirical work will be disseminated to the wider academic public through the following planned articles:
• Ellen-Marie Forsberg, Volkert Beekman, Barbara Ribeiro, Nils Heyen, Rasmus Øjvind Nielsen, Lars Klüver, Thomas Reiss and Kate Millar. Needs for integration in assessments of emerging science and technologies. (Almost finalised) Science, Technology and Human Values (TBC).
• Ribeiro, B.E. Smith, R., Millar, K. Mapping and understanding Responsible Research and Innovation (Under development). Research Policy (TBC)
• Ribeiro, B.E. Smith, R., Millar, K. Interdisciplinary problem framing in the appraisal of emerging biofuel technologies. Technological Forecasting and Social Change (TBC)
Conference presentations: Pacita Conference 2015, Berlin, February 25 2015: Panel session: Integrated Approaches in Technology Assessment. Chairs: Nils Heyen and Rasmus Øjvind Nielsen. Presentations:
• J.C. Schmidt: Problem oriented interdisciplinarity and integration: Methodological reflections and considerations.
• S. Lingner: Interdisciplinary integration in technology assessment. A report from practice.
• E-M Forsberg: TranSTEP. Getting an integrated perspective on complex technology issues by teaming up across established assessment communities.
Other dissemination: EST-Frame is presented in the first edition of the newsletter of the SNet organisation, March 2015.
The DBT has implemented the EST-Frame criteria for integrated assessments in a method database for situation analysis, method selection and project self-evaluation. The database covers more than 55 methods, and for each method criteria are evaluated, such as the method’s ability to include multiple domain perspectives, to create collaboration across domains, or to integrate the work of multiple domains. The database includes method data from the foresight project ex ante / ex post evaluation tool DoingForesight.org and is expanded with methods from the Engage2020 project (engage2020.eu). The database will make up a major source for future tools for situation analysis, ex ante project evaluation, method selection, ex post evaluation and other function coupled to the project definition and setup phases.
Further exploitation of the results
We see two avenues for further exploitation of the results: continued research in the field of assessment of emerging science and technologies, and further development and application of the TranSTEP approach. We will pursue both strategies.
One action has already been implemented:
– Resiliency and adaptation to climate change in regional strategies: This is a bilateral Czech-Norwegian research project, funded by Norway Grants in the period of 2015-2016, where HiOA will apply TranSTEP on regional climate change adaptation strategies.
Partners in the EST-Frame consortium have also included versions of TranSTEP in the following research and development project proposals:
- ERASynBio 2nd Joint Call for Transnational Research Projects: Building Synthetic Biology Capacity Through Innovative Transnational Projects (July 3rd 2014): A proposal with the following title was submitted by a trinational consortium: BioMETCHEM Methane as a novel platform substrate. As a “strategic element” the TranSTEP approach was included, two partners of the EST-frame team (Fraunhofer ISI and University of Nottingham) were responsible for the strategic element. The proposal was unfortunately rejected.
- The Assisted Living project: In this project proposal, submitted to the Research Council of Norway by HiOA in February 2015, a TranSTEP group will follow a technology development process within welfare technologies throughout the whole project (2016 – 2020). The proposal is currently under evaluation.
- Atmospheric Carbon Trap: In this project proposal (coordinated by the University of Edinburgh) to the FET Open Programme a version of the TranSTEP approach was going to be used by HiOA to assess an Atmospheric Carbon Trap technology. The proposal was unfortunately rejected.
Other avenues for exploitation of the results
We will continue to promote the approach to relevant assessment commissioners and practitioners, through direct contact and through participation in relevant conferences (for instance conferences on risk assessment, integrated assessment, economic assessment, etc.).
Moreover, we will consider ways of further adapting the approach. It was suggested by Philippe Galiay at the project’s final conference that it would be of interest to the EC to further develop TranSTEP into a challenge based approach, rather than a technology based approach. This is duly noted and is a recommendation we will follow up on in the near future.
The project’s advisory committee as well as the end users involved in our workshops have been important for suggesting how we can exploit our results further. We have received the following advice:
- Make a practical guidance on how the different domains work and how information from the different domains can be balanced and integrated.
- Make a guidance on ethical review of nanotech and biotech projects in Horizon 2020.
- Contribute to OECD’s current work in developing impact assessment guidance.
- Contribute to developing guidelines for integrated sustainability impact assessment (SIA) for trade negotiations, such as The Trans-Atlantic trade negotiations.
- The UN international integrated assessment is another potential client. UN Institute for Crime (UNICRI) published in 2011 a report on biosecurity in NanoBio (incl. synthetic biology) where they ask for joint (i.e. inter-governmental) assessment methodologies. Joint assessment methodologies already exists for radiation in IAEA, but TranSTEP could be of use in other areas.
- An adapted TranSTEP approach could be of use to analyse what questions have been posed and answered in a research and innovation process.
- TranSTEP could function as a conflict resolution tool, where it is useful to connect to other people’s agendas.
The members of the EST-Frame consortium are determined to continue to exploit the results from EST-Frame, as a consortium and as individual partners.
List of Websites:
Project website: www.estframe.net
Oslo and Akershus University College, Research Group on Responsible Innovation, Ellen-Marie Forsberg, Ellenmarie.Forsberg@hioa.no
University of Nottingham, Centre for Applied Bioethics, Kate Millar, kate.millar@nottingham.ac.uk
The Danish Board of Technology Foundation, Lars Klüver, LK@tekno.dk
Wageningen Agricultural Research Institute – LEI, Erik de Bakker, erik.debakker@wur.nl
Fraunhofer Institute for Systems and Innovation Research ISI, Thomas Reiß, thomas.reiss@isi.fraunhofer.de
Exploring the needs for integration in assessment of emerging science and technologies (EST) was a central aim of the FP7 project Integrated EST Framework (EST-Frame). The EU, national governments, and a whole range of national and international institutions produce assessments of emerging science and technologies. These assessments take different disciplinary perspectives and may point in different directions for responsible governance of technologies, and there is currently limited integration of such advice, opinions and appraisals. The EST-Frame consortium, consisting of researchers from the Danish Board of Technology, Fraunhofer ISI (Germany), the Wageningen LEI Institute (the Netherlands), University of Nottingham (UK) and Oslo and Akershus University College (Norway, coordinator), responded to a call from the EU, through the FP7 Science in Society Programme for making recommendations for more integrated EST assessment.
The EST-Frame project established a threefold workplan for the task: 1) we studied how emerging science and technologies were assessed and governed in four case studies; 2) we studied the main methods and traditions for technology appraisals – and this we referred to as “assessment domains”; and 3) we identified major societal trends affecting assessment practices. Furthermore, we aimed at a differentiated analysis of what integration means at different levels and for different purposes.
The consortium engaged a range of stakeholders and assessment practitioners in order to discuss and review integration through a series of workshops. The results from the case studies, the study of the assessment domains, the trends analysis and the workshops are published in a special issue of the journal Science and Public Policy, June 2014. References to these articles, as well as the project’s full publication list, can be found at www.estframe.net.
In December 2014, the EST-Frame project presented the outcomes of the work to a policy-oriented audience. One of the notable conclusions was that there is a lack of dialogue between assessment communities on basic assumptions for assessment, and that assessment communities struggle with similar challenges specifically related to uncertainty, complexity and controversy around facts and values related to new technologies. The resulting recommendation is a process and learning approach named TranSTEP (TranS-domain Technology Evaluation Process). The central elements of a TranSTEP process include – through continual use of facilitated dialogue - to conduct a transdisciplinary situation analysis and transparent problem framing before setting out with assessments, to reflect commonly on method choices in current and potential new assessments to be integrated, and to integrate the results in a transparent and reflective way.
Although some integrated approaches already exist, such approaches are still not mainstream in technology appraisal. In the project’s final conference all practitioners and researchers linked and directly involved in EST assessment were invited to join in a longer term quest for achieving in practice more balanced and integrated assessment for responsible technology governance.
A full description of the TranSTEP is published on https://transtepapproach.wordpress.com
Project Context and Objectives:
The significant investment in science and the high levels of novel technological development and innovation currently witnessed globally raises questions about and constantly challenges the way in which technology governance is structured and conducted. Embedded within current EU policy goals, technology and innovation policy is intended to ensure that technology is developed in line with societal needs and is responsive to public concerns; at times responding to notable levels of public anxiety. Several forms of new technology development can be characterised as scientifically complex and novel innovations, presenting significant uncertainties related to the application of the technology. This means that policy makers and the wider public are largely dependent on expert interpretations of what risks and consequences the technologies may present to society or specific affected parties. These expert interpretations are provided in terms of risk assessments, impact assessments, ethical reviews, economic analysis, etc. In addition, different forms of public or stakeholder deliberation frameworks are applied to inform policy makers and build better understanding between societal groups regarding technology development.
However, individual analysis and assessment frameworks can often reveal only a partial presentation and assessment of social issues and hence different frameworks sometimes render different (and potentially conflicting) findings and conclusions. The selection of assessment framework may itself be a ‘value-based’ choice, as different frameworks rest on different premises, traditions of thought and ethics. The assessments, which can be conducted through the application of the different frameworks, may not converge into a stable and robust understanding of the social impact of the technology, but may point in different directions when it comes to responsible governance of technology. It is therefore important that the nature and choice of frameworks for technology assessments are developed to be more transparent, revealing the premises, potential biases, and strengths and weaknesses of the different models. On this background the European Commission, in the 2011 Science-in-Society programme, called for a more integrated approach to assessment and governance of emerging science and technologies (EST), to which the EST-Frame project is a response.
The notion of integrated assessment is not new. European Impact Assessments follow an integrated approach where economic, environmental and societal issues should be considered in assessments of European actions. Integrated Sustainability Assessment is a well-established approach on the side of environmental management. Integrated assessment of EST is therefore well placed in a landscape of integrated approaches. However, it is not immediately clear what an integrated EST assessment would consist in, although the subject matter indicates that scientific and social uncertainties and controversies would need to be handled by such an approach. This is the background for the EST-Frame project, which has carried out broad empirical studies in order to develop an integrated approach for assessment of emerging science and technologies.
In accordance with the overall concept presented above, the project has had the following aims and objectives:
Aim: To assist in making holistic and integrated assessments of emerging science and technologies resulting in socially robust technology development conducive to societal progress and increased quality of life.
In order to achieve the aim of the project we specified a number of objectives.
i. To undertake a study of four cases (nanotechnologies, synthetic biology, biofuels and security & emerging ICTs) to analyse the nature of the frameworks that are currently being used to assess the technology development and how the different assessments can be compared.
ii. To analyse the assumptions and results from the assessments of the four cases. Identify strengths and weaknesses of the frameworks.
iii. To identify different advisory contexts and in what contexts there is a need for integrated frameworks, and when more focused frameworks may be sufficient.
iv. To determine the role for an Integrated Assessment Framework in a policy context characterised by internationalisation of markets and technologies, global liberalist politics, and new forms of public-private partnerships.
v. To identify existing frameworks of a comprehensive or integrated nature.
vi. To develop an integrated assessment framework based on the analysis of the four cases and currently used frameworks.
vii. To apply the integrated assessment framework on the four cases and analyse how such an assessment differs from the earlier assessments. This includes organising four deliberative events where users, policy makers and stakeholders reflect on the totality of existing assessments in light of the integrated framework.
viii. To ensure end user relevance of the results by presenting them to a group of policy makers, advisers and key stakeholders, and revise the model based on their input.
ix. To disseminate the final results to end users and the academic community by a final project conference consisting of an expanded end user group, as well as communicating the results through the project web site, relevant other web sites, scientific journals and the media.
Project Results:
The project conducted four case studies of assessment and governance of emerging science and technologies, a mapping study of assessment frameworks currently in use, a study of policy trends affecting assessment and an analysis of integration needs in current EST assessment. We will here present the most important results from each of these research tasks.
3.1 THE CASE STUDIES
The four case studies followed the same protocol, structured into the following main sections:
• Description of the technology
• Description of the governance context. This context would be national (related to the responsible partner’s national context) as well as on the EU level.
• Identification/screening of assessments. A comprehensive search for relevant assessments in the selected country and the EU level.
• Selection of 8-12 assessments for closer review. These selected assessment should be policy relevant and represent different assessment perspectives
• Analysis of these assessments using the project’ analytic protocol; the process characterisation table and the purpose analysis table (see tables 1 and 2 in the pdf version of the final report), as well as a more general description of their context and significance.
• Qualitative comparison of the assessments and evaluation of the assessments situation revealed in the case study. This included the organisation of an expert workshop discussing the case study findings.
• Judgement on the needs for integration revealed in the case study (summarised in table 3 in the pdf version of the final report).
3.2. CASE STUDY ON NANOTECHNOLOGY IN FOOD AND AGRICULTURE
Nanotechnology has many different manifestations – in structures, as particles, and in degradable or non-degradable forms – and is surrounded by many uncertainties; nanomaterials exhibit properties not found at the macro-scale which may result in unpredictable safety problems and risks. As nanotechnology increasingly is being used in consumer products this will in time affect large number of consumers, workers and citizens in general, in addition to potentially impacting on the environment. Many therefore claim that nanotechnology cannot be widely implemented into society without public engagement, call for an inventory of nanofood and contact materials, and urge industry to communicate more with the broader public about the research and development they are doing in this field. Moreover, because of the potentially big gains to be harvested from nanotechnology, along with potential significant risks, there is a perceived need to carry out different kinds of risk assessments, impact assessments, economic assessments, ethics assessments, and so on. In this case study we studied a number of assessments with the field of nanotechnology in the food and agriculture sector, leading to a number of findings related to assessment and governance of nanotechnologies.
When selecting assessments for review we found that most nanotechnology assessments apply to nanotechnologies in general, and not specifically to nanotechnology in food (or agriculture). The assessments chosen for further analysis in this case study had primarily a focus on the Netherlands and Europe. We tried to encompass the perspectives of all relevant stakeholders and the multiple dimensions that can be impacted by the development of nanofood (economy, environment, safety, security, society and ethics). In total five assessments in the Netherlands and eleven assessments in other countries were selected and reviewed on several dimensions. For getting a more detailed picture of the assessments and the issues at stake in the debate on ‘nanofood’, interviews with twelve key stakeholders in the Dutch nanofood network were executed.
The study found that there is an equal assessment trend internationally (in the EU) as in the Netherlands. The first years of nanotechnology assessment have been dominated by deliberative efforts, while these kinds of approaches have not been so prominent in the latter years. Deliberative projects (like Nanologue, NEG, Nanodialogues, FramingNano etc.) started out with great ambitions, and achieved much, but on an overall level many felt that nanotechnology development and policy largely went on irrespectively of these activities. A vigorous public debate, outside of the organised exercises, was never achieved. Later, voluntary codes of conduct emerged, where industries (and researchers) would commit themselves to among other things be transparent and communicate uncertainty (EC 2008, Responsible NanoCode etc). Towards the end of the period we see more of the international institutions entering the stage and a shift towards risk assessments as well as guidelines for these. There is still a lack of methods and standards for risk research, and hence a lack of knowledge for assessing nanotechnology in a traditional scientific sense. Most of the evaluations acknowledge the lack of basic risk research as one of the major concerns.
The discussion of the findings in the case study highlighted that although it is understandable that environmental and human safety aspects are the most prominent concerns in debates on nanotechnology, this should not be the only focus in assessments. We believe that there needs to be an increased interconnecting of assessments of environmental and human risks, of economic costs and benefits and of other ethical and social issues. This is not only important for public opinion formation on nanofoods, but also for increasing sensitivity and trustworthiness of (scientific) risk assessments in the context of the uncertainties that seem inevitable with this new science and technology. The keyword in this context is learning, not least about the different perspectives and fundamental values that may be hidden behind concrete opinions on the health, environment and safety issues of nanotechnology. Looking at the results of our analysis of assessments and frameworks, a recommendation is to create and support networks on national levels that can overcome the organisational and institutional constraints that are an important cause for the weak links between the different kinds of assessments (scientific risk assessments, industrial foresights and economic analysis, public deliberation processes).
This study resulted in the following recommendations to policy makers:
1. By shaping a safe consortium or platform in which industry and other stakeholders can exchange views and information, or encouraging existing consortia and platforms to pay more attention to this kind of interaction and communication, a more mature ‘learning relationship’ with the (inevitable) ambivalence of citizen-consumers could be started off. Moreover, networks should be created on national levels that can overcome the organisational and institutional constraints that are an important cause for the weak links between the different social spheres in which nanotechnology assessments are taking place. Assessments should be seen not only as a way to inform decision makers, but as learning processes in itself. As such, they should be set up to also involve those involved in risk assessments, economic assessments and impact assessments, so that learning is facilitated across disciplinary boundaries. In order to be trustworthy risk and impact assessments need to be informed about the concerns of citizens and consumers.
2. There is a need for more application specific assessments. The large diversity of nanotechnology applications implies that specific assessments must be carried out for nanotechnology in food, packaging and agriculture. But even within this sector the diversity of applications is great. Assessments need to be tailored to specific issues within this broad field, and the participation and design of the assessments need to be adapted to the problem. This might involve setting up ad hoc assessment events, or encourage existing institutions to tailor assessments according to the specifics required by the topic. Such a problem/topic based approach will also by design avoid falling into the science trap identified by a number of authors; namely that assessments of emerging science and technologies become preoccupied with speculating on spectacular issues rather than addressing the most immediately relevant, but not so fascinating, applications.
3. There is a significant variation of the extent to which assessments address important contextual trends. This might be explained by the technology focus of most such assessments; the primary interest lies in describing the potential benefits and risks of the technology, as well as prescribing appropriate government responses to such. However, core contextual trends will impact on both the technology development, the magnitude and distribution of benefits and risks, and the ability to control these. Therefore we recommend that such trends are taken into account by assessors, and that decision makers request such contextual analysis when commissioning assessments.
3.3 CASE STUDY ON SYNTHETIC BIOLOGY
Synthetic biology is an emerging sub-discipline within the field of biotechnology, focussing on introducing engineering at various levels of biological systems. Although synthetic biology is still primarily a scientific endeavour, the huge potential and impact it could have on the economy, environment and society as a whole emphasises the importance of sound assessments. A review of a selection of current synthetic biology assessments on a number of aspects was elaborated, including issues related to purpose, applied methodological approach and created impact on policy making processes. Based on the identification of both strong and weak points within these assessments, some recommendations are suggested for future assessments of synthetic biology.
Of the in total 91 identified synthetic biology assessments, 11 assessments were selected for review and comparative analysis (4 German assessments, 7 European/international assessments). Criteria used for selecting assessments included geographical origin, methodological approach applied and analysed impacts. The reviewed assessments were conducted by various organisation types including research institutes and consortia, science and technology advisory organisations, NGOs, economic interest groups and funding organisations.
The assessments were systematically reviewed based on a developed analytical framework. Based on the assessment reviews, we would like to emphasise the following issues:
Concerning the aim and content of the reviewed assessments:
• The majority of assessments seem to be self-initiated; there is limited evidence concerning external commissioning. A main driver for many assessments is a need to position the respective organisation within the synthetic biology debate, or to react to active discussions within this field.
• The transparency and impartiality is in general good. Nevertheless, the reviewed assessments conducted by both NGOs and industry representatives seem to be less transparent and impartial compared to assessments conducted by other actor types.
• Considered impacts within synthetic biology seem to focus mainly on safety and security issues. However, issues related to economy, environment, society and health are also being discussed to a certain extent within most of the reviewed assessments. The topic of sustainable development does not seem to be very present in synthetic biology assessment.
Concerning the methodological approach of the reviewed assessments:
• Due to the emerging nature of synthetic biology, the majority of assessments are anticipatory. Considered time-scales within the assessments range from short term (<5 year) to long term (>15 year); however, none of the analysed assessments have applied structured tools for anticipation.
• Assessments are very much orientated towards scientific expertise. There is limited involvement of industry representatives, lay-people and policy makers in constructing synthetic biology assessments.
• The methodological approach of synthetic biology assessments is likely to be driven by the emergent nature of synthetic biology. The number of tools used within assessments, as well as methodological reflections, is limited. The majority of reviewed assessments rely to a large extent on literature review. In addition, some assessments use information derived from workshops, the majority of them being driven by expert judgement.
• Synthetic biology is seen as an international R&D quest, and is largely funded under existing biotechnology programmes.
Concerning the addressing of synthetic biology governance within the reviewed assessments:
• Although the majority of assessments state that synthetic biology governance could depart from currently active regulatory structures regarding genetic modification, it is often not clear how these structures are limited with respect to future synthetic biology activities and product development.
• The limited amount of observed regulatory activity might also relate towards the unclear situation regarding the need for market regulation versus liberalisation, as identified within the reviewed assessments.
• There is limited attention within the assessments concerning how obtained results have been, or could be, integrated within the overarching field of biotechnology governance. This situation may correlate with the limited policy impact of the reviewed assessments.
Although past assessments might have proven very useful and valuable at the time and within the context of their execution, the fast changing and dynamic field of synthetic biology might require different approaches for future technology assessments. At this point, there seems to be a need for improved dialogue between assessors and policy makers to clarify specific assessment needs that could improve policy relevance and policy impact. Our analysis implies that future assessments might need to consider more structured designs for anticipation, more inclusive participation of relevant stakeholders in methodological approaches and more systematic analysis of contextual variables and relevant trends.
Based on the synthetic biology case study results, a number of recommendations for policy makers are suggested:
• Be more pro-active in commissioning synthetic biology assessments: Since the majority of synthetic biology assessments seems reactive, in order to position an organisation into the synthetic biology debate, the link with and impact on policy making processes is not always evident. Governments could increase the effectiveness of assessments through communicating to assessing institutes policy maker needs with respect to synthetic biology governance, including the need to address relevant political, social, economic and scientific trends.
• Understand the time-scale of future synthetic biology developments: In order to communicate the needs for synthetic biology governance, policy makers need to have a clear understanding to what extent current governance structures apply to both current and future synthetic biology developments. A roadmap could be an instrument to anticipate decision making with respect to short- and long term synthetic biology developments.
• Stress the importance of public dialogue in assessing synthetic biology: Although a number of Science-in-Society research projects have been initiated by the EC to stimulate public dialogue concerning synthetic biology, the limited inclusion of lay people in assessing synthetic biology seems to limit the scope and applicability of such assessments.
• Synthetic biology is global: The international character of biotechnology needs to be appreciated in policy making. Activities with respect to governance outside the borders of Europe need to be closely monitored, in combination with the effect of the introduction of new governance approaches on the development of synthetic biology.
3.4 CASE STUDY ON BIOFUELS
Biofuel research and technology development has seen notable investment in the past decade at a European level. A proliferation of assessments has accompanied the intensification of research, although the body of assessments is very diverse and, often, poorly coordinated. A set of UK, European and international reports were analysed in a desk-study, interviews and workshop in order to examine the state of biofuel assessment and the potential for the application of integrated approaches.
One of the significant aspects of this case study is that it represents one of the few studies that has attempted to clearly characterise biofuel assessment approaches. Biofuels has developed significantly in the past decade and a proliferation of assessments has accompanied the intensification of research, although the body of assessments is very diverse and, often, poorly coordinated. Within the case study a set of UK, European and international reports were identified and analysed.
In terms of the assessment landscape, the assessments tend to focus upon environmental and, to a lesser extent, economic impacts with social impacts often overlooked. Most assessments position biofuel research and technology development in an ’opening’ stage and demand more knowledge and broader understandings of potential impacts. The assessments exhibited high scientific literacy and impartiality. Transparency of reporting varied amongst assessments and both participation and self-reflection are low. The body of biofuel assessments is diverse and the embedded assumptions often appear incommensurable. This indicated that it would be difficult to integrate findings.
The workshop findings confirmed that many practitioners consider the embedded and implicit assumptions in assessments to be a major problem for the body of biofuel assessments. The findings of the workshop suggest that a group could be established to deliver a report summarising the state of current knowledge about biofuels with particular reference to the embedded assumptions and methods. Participants suggested that the group should be independent but state-sanctioned, preferably at a European level. The procedural design and transparency of such a group would be a crucial factor in its legitimacy and success.
Based upon the findings of the analysis, recommendations were made for (i) assessment practitioners, (ii) research councils and (iii) policymakers. These primarily respond to the following problems identified in the body of biofuel assessments:
• Lack of transparency, affecting the utility of assessments for policymakers or further studies and also making the integration of findings difficult
• Insufficient attention paid to societal aspects
• Focusing upon specific technologies rather than specific problems
• Limited participation, particularly of non-experts, which may affect assessments’ sensitivity to different perspectives on technical impacts.
Recommendations for Assessment Practitioners:
• The desk study found that biofuel assessments are weak in the areas of transparency and self-reflection. The workshop findings confirmed that this failure to highlight and justify approaches, methods and boundaries negatively affects the utility of assessments, regardless of their scientific integrity (which is considered strong), and presents a serious barrier to the integration of knowledge. As such, we recommend that assessment producers consider the following practical points to improve the transparency and, ultimately, the utility of their work:
• Explicitly state the methods applied, justify their selection and examine their limitations. The assumptions embedded within these methods should be considered and described.
• Include as much data as possible (using annexes or on-line resources where necessary), stating any assumptions and clearly describing the analytical steps taken to produce the results.
• Explicitly state the boundaries of analysis. Consider the geographic scope, specific combinations of feedstocks and process technologies, and different types of impacts (e.g. GHG balances or societal aspects) to which the study is limited. Justifications should be provided where results are extrapolated beyond these boundaries.
• Consider the extent to which results depend upon the context of assessment, and how the results would be affected by, for example, new targets. While such changes are difficult to predict, clearly describing the data and methods applied could prolong the utility of an assessment which may otherwise be made obsolete as contextual features change.
• Reflect upon the limitations of the study and the conditions under which results remain valid.
• Participation, particularly of non-experts, is low in biofuel assessments. Although there are many forms of participation and participation per se may not appropriate for every study, increased participation may improve the legitimacy of assessments by incorporating the insights of publics and other stakeholders. This may also improve assessments’ reflexivity and strengthen their attention to societal aspects, each of which is also considered low.
• Consider how policymakers engage the academic literature and develop strategies to improve visibility, for example publishing targeted policy briefs on open access on-line sites.
Recommendations for Research Councils:
• The desk study, workshop and interviews reveal that assessments are insufficiently sensitive to social impacts. This is a difficult problem and is probably not limited to the study of biofuels. The situation may be improved by greater cross-domain or cross-funding council activity relating to biofuels, including grants for the examination of social impacts of biofuel development and support for dialogue across disciplinary boundaries. Some such activities are underway, cross-cutting events organised by the UK bioscience research council, BBSRC.
• Workshop participants find that the focus upon specific technologies such as biofuels, rather than specific problems such as land-use, makes it difficult to produce meaningful assessments which can adequately respond to contemporary challenges. This is exacerbated by the widely reported difficulties of publishing and building careers outside of disciplinary silos. The provision of funding to research the impacts of specific technological developments could discourage assessment practitioners from considering problems holistically. Research councils could respond by providing funding structures which encourage broad research into potential responses to challenges such as land-use change.
Recommendations for Policymakers
• The body of biofuel assessments is diverse and the embedded assumptions often appear incommensurable. This means it is difficult to integrate findings. The workshop participants suggested that a committee should be established to deliver a report summarising the state of current knowledge about biofuels with particular reference to the embedded assumptions and methods. It was suggested that the committee should be independent but state-sanctioned, preferably at a European level. The procedural design and transparency of such a committee would be a crucial factor in its legitimacy and success.
• Assessments are framed with reference to policy discourse. As climate change increasingly dominates policy discourse, GHG emissions increasingly dominate assessment spaces. Maintaining alternative discursive arenas such as rural development, energy security and land use may inspire assessments to consider a broader range of impacts. Even indicating a desire to consider more social perspectives on biofuel development, for example, may inspire activity amongst social scientists who are not otherwise not engaged with these issues.
• Policymakers could engage the academic community more systematically. Ad hoc approaches to the literature are not transparent and are unlikely to yield the best results. A closer relationship could be forged, for example, through partnerships with the teams of multi-disciplinary researchers working across topics or directly with research councils. The research community could improve their penetration by releasing policy briefs alongside their academic contributions or by running short policy-focused dissemination events.
3.5 CASE STUDY ON CLOUD COMPUTING
This case study described assessments in the field of cloud computing as well as their interrelations and their links with policy processes. The case study focused specifically on assessments influencing the European Commission's cloud strategy presented in 2012. The study's aim was to understand which kinds of assessments have played which role in the evolution of the field of assessments and to seek indications as to the formative effect – if any – the production of the cloud strategy has had on the assessment field.
As part of a broader investigation of “integration” in assessments and the possible need thereof, the case study focused especially on integrative functions in the field of assessment. Looking beyond methodology, the case study attempted to establish the formative effects of the cloud strategy production process on the assessment field as a whole as an “integrative” effect. The purpose was to lay the ground for a better understanding of the strategic situation facing any project attempting (normatively) to promote integration in assessments.
The main part of the case study took the shape of an outline of the field of assessments carried out under discourse-analytical inspiration. This part of the study shows that cloud computing assessments have historically grouped around seven “problem perspectives”, some of which are integrated under the strategic perspective of the cloud strategy. Importantly, efforts to promote and assess productive development paths alternative to those promoted by major industry actors are largely ignored. Instead, “conventional” perspectives such as security, privacy and legality are the main perspectives opposing industry hype, which are integrated in/by the political process. The main concerns driving integration of opposing evaluations of cloud computing, however, remains strategic and economical ones. “Integrative” perspectives in the broader sense of ethical/societal evaluations of the technology only emerge late in the process in reaction to political initiatives.
The study supplemented this assessment field analysis with analysis of process characteristics and purposes of individual assessments along with interrelations between assessments and relevant political documents.
In the course of the case study, seven consecutively emerging “problem perspectives” were identified from which cloud computing has been assessed since its emergence in the market in 2006. These were:
1. Individual strategy assessment (in reaction to hype)
2. Co-creative promotion of alternate development paths
3. Assessment of the technology's ontology, risk and reward
4. Security and privacy in a globalized world
5. Growth in a time of crisis
6. Societal strategy with regard to cloud computing
7. Integrative responsibility (in response to political actions)
One important example of such emergent relations between problem perspectives is that of “trust” emerging as a term bridging and combining the concerns of security and privacy assessments on the one hand and economic concerns on the other. “Trust” is neither technical, legal nor economic, but bridges all of these conceptual areas and provides a target for strategic coordination of perspectives, which might otherwise remain locked in opposition. As something of a dialectical creation, this concept acts therefore as a specific key to conceptual integration at a level, we might call “politico-epistemological”, i.e. it provides a simple framing (building trust in the cloud) for the complex of strategic problems facing decision-makers (balancing data security, privacy rights and carbon footprints against the primary and secondary economic benefits cloud computing seems to entail).
With regard to the question of integration, we observe an evolutionary maturation of the field of assessments as a whole and a continual widening of the range of topics dealt with in assessments. This is not so much due to systematic attempts at “integrative assessment”, but more due to mutual learning in the field and the fact that assessors read and digest already completed assessments. As such, the field of assessment has something of a closed circuit about it, with analysis points traveling from one assessment to the next.
To be sure, many possible points and perspectives are never represented or mostly ignored in assessments. We have mentioned for instance the absence of the perspectives of lay people and the marked silence in the field about alternative development paths such as open-source. Four interviewees stated explicit needs for further integration. Two of these expressed a need in the field for more stable platforms or methods for dialogue between stakeholders, where the power effects of major corporate actors shaping the dialogue could be neutralized. Two other interviewees expressed a need for better translation of “ethical” issues (a term they found more confusing than enlightening) into operable terms directly relevant to the topics under discussion in the wider process of societal appraisal of something like cloud computing.
Our reading of the field shows that it is possible to continually widen the range of topics being assessed. Such widening, however, takes place mainly by the promotion of issues through hype; through the forcing of issues through advocacy; or through the dialectical integration of opposing viewpoints. The only explicit attempts at systematically integrating a comprehensive spectrum of assessment topics in a single assessment we find in one ethical assessment and one technology assessment.
The main conclusions about the field of cloud computing assessments are:
1) The assessment field is dominated by assessments concerned with ontological and risk issues followed closely by strategic and economic assessments.
2) Assessments of privacy issues act as the main counterweight to these perspectives while ecological sustainability plays a minor role. Broader assessments of social sustainability and value discussions largely do not exist in the field.
3) Political action plans have a major formative effect on the assessment field.
4) Very few attempts exist at “integrative” assessment.
The main recommendations to policy makers consist in
• implementing deconstruction of framings in policy preparation, especially with regard to identifying business interests versus general societal interests
• seeking out visions alternative to those presented by business interests
• explicating methods for balancing / resolving conflicting viewpoints on new technology
• actively including citizens’ perspectives
3.6 FINDINGS ON INTEGRATION NEEDS ACROSS CASE STUDIES
See table 4 in the pdf version of the report.
This analysis importantly informed the process of identifying the criteria for our integrated assessment approach (presented in section 3.9 below).
3.7 FRAMEWORKS FOR ASSESSING SOCIETAL IMPACTS OF EMERGING SCIENCE AND TECHNOLOGIES
In this part of the work we collated and compared the work in the case studies and enriched this material with additional analysis of six established advisory domains: risk analysis, impact assessment, economic assessment, foresight, technology assessment (TA) and ethical assessment. In the case study and domain work all in all 1506 assessment reports were screened and 101 reports were reviewed according to the protocol. We also organised a practitioner workshop where we discussed with professionals involved in assessment about challenges and best practices of assessment. We documented that emerging science and technologies are assessed with a great width of methods and approaches. Several advisory domains are involved in producing such assessments and many assessments or reports are produced also outside the established advisory domains. Advisory domain assessments follow the conventions of their domains, and are imprinted with the cultures and traditions of their respective domains. Theses cultural assumptions influence the framing of the assessments, the choice of methods and how the assessments are reported.
Through the project’s analytic instruments we documented that the manifold of assessments have different intended purposes and functions. They frame their topics in different ways depending on their mandate, the traditions in the domains, the impact they intend to have, the participants included, etc. This means that there is a wide range of assessments answering different questions. We also documented that the advisory domains generally do not make use of systematic tools for situation analysis and method choice, though some such tools may exist. The domains also generally lack standards for transparent reporting of situation analysis, including framing assumptions, method choice and the unfolding of the assessment process.
We found that all domains have a focus on methodological development, and try to tackle the complexities and uncertainties implied by emerging science and technologies. However, they do not often discuss such challenges with practitioners from other fields. There are sometimes collaborative assessment efforts across the domains, for instance between TA offices and ethics committees. However, most often the domains do not collaborate to solve methodological challenges. Even if there are many overlaps between the domains in terms of both topics and methods, there is a lack of communication between the domains. There seems especially to be a communication gap between ethicists and economists in the context of emerging science and technologies. From the project practitioner workshop we learned that such cross-domain learning was appreciated.
Assessments from outside the established advisory domains may be just as important for policy as those from inside. However, we argued that there is a risk that these have less institutionalised mechanisms for methodological reflection and learning. Legitimacy of the input, methods and output is important for assessments coming from both inside and outside the established advisory domains. In this study we found no institutionalised mechanisms for judging the quality and legitimacy of assessments that influence policy. Moreover, there was very limited transparency with regard to the impact of the different assessments on policy.
Several integrated methods exist, but from the 101 assessments reviewed in this project, very few could be characterised as integrated on several dimensions. Moreover, for those assessments that scored high on several dimensions relevant for integration, it was generally hard (though with some notable exceptions) to trace this back to systematic methodologies. From the project practitioner workshop and from the literature studies we learned that there is no established understanding of what the right approach to achieving integration is. However, de facto integration happens when translating lessons from the assessments into practical EST policy. There is evidence from the project’s case studies and discussions with end users that this translation process is not entirely transparent.
In the analytic work of the project we found that there is no comprehensive acknowledgement and transparency of the value-laden assumptions underlying EST assessments. In many cases of EST there is no ‘innocent’ starting point for assessment; a starting point that is not contested. This applies to risk assessment, to economic assessment, and to ethical assessment alike, as well as to all other assessments.
There are many EST assessments focusing on specific EST dimensions in a general way, such as health, safety and environmental concerns (HSE) of nano particles, security concerns of ICTs, ethical concerns of synthetic biology, economic projections of bioenergy, etc. These are important for an initial mapping of issues. We identified fewer assessments trying to tackle concrete EST related policy problems, such as specific applications in a specific geographical context. With increased practical problem orientation comes increased complexity in variables and a need to consider all the above dimensions at once, with corresponding problems of delimiting the adequate scope of assessments. These are topics that need to be addressed in order to facilitate responsible technology governance in practice.
The work generated a number of recommendations to policy makers and assessment professionals:
1 Assessments should be transparent in their framing of the topic, situation analysis, method choice and practical process development. Only in this way is it possible to assess the quality and legitimacy of the assessments in their function as providing an evidence base for policy. Policy makers thus need to request such transparency. Transparency guidelines should be developed for advisory reports in general.
2 There should be increased interaction between the advisory domains in order to enhance learning and facilitate extended peer review. Such interaction should be facilitated by institutionalised instruments.
3 There should be increased focus on developing methods for tackling EST issues that have become practical policy issues. This inherently involves interdisciplinarity and broader involvement of stakeholders and/or the public. Policy makers should request such problem oriented assessment before the problems become acute.
4 In most cases, before a field of assesment reports can come to function as an evidence base for policy, some form of integration of the main lessons from these reports will be necessary. The legitimacy of this evidence base will increase when such integration is done in dialogical and problem-oriented interdisciplinary processes. Policy makers should work to institutionalise such forms of integrated assessment.
3.8 THE CURRENT AND FUTURE CONTEXT FOR EST ANALYSIS
Technology advising takes place in larger societal contexts and the study of these contextual developments were considered important in EST-Frame. These contexts can be understood as influenced by some significant mega/macro trends, as well as a variety of trends and developments at a meso and micro level. Assessment of emerging science and technologies must respond to such trends in order to be well targeted and policy relevant.
In this work macro trends were identified and analysed, with the most important aspects examined in more detail in order to consider the implications for technology advisory practices. The trends analysed were selected based on a study of reports from seven larger trends studies. The trends and contextual factors that were identified as the most significant were those that are:
a) Important to consider when designing, carrying out and evaluating assessments; and
b) Important for developing a framework for integrated assessment that is capable of being responsive to such developments.
The trends were profiled as:
1. Liberalisation and globalisation
2. New governance networks
3. Public-private partnerships (PPPs)
4. Citizen empowerment and public deliberation
5. Rapid technological change
6. Focus on sustainability and climate change
7. Economic change
8. Quantification
9. Policy integration as a response to the identified trends
In particular our analysis has examined whether or how the identified assessments (in the case and domain studies):
a) discuss the effect of the trends on technology governance
b) adapt their methods to the trends in order to have more impact on technology governance
c) are transparent about their assumptions regarding the future and how the trends influence any of the topics they address ( for example, ethical issues, economic projections, technology development, etc.)
The main lessons and recommendations for policy makers that emerged from this work are:
Greater levels of market liberalisation and public-private partnerships (PPP) would appear to indicate increased importance of economic assessments in decision-making. However, economic assessments are not dominating any of the chosen case studies, and PPPs are hardly mentioned. If it is the case that liberalisation is an important contextual factor then the effects of market liberalisation on responsible technology governance should be discussed more widely. Moreover, the economic assessments that underlie policy on PPPs should become more transparent and scrutinised as part of the assessments in EST fields. As liberalisation potentially affects the boundaries of technology governance assessments should discuss this reflectively to a larger extent.
New governance networks, especially on an international level, are often considered in assessments. However, assessments are rarely conducted at a global level. Because of the global character of the current grand societal challenges, there should be increased infrastructure and competence building for assessment at a global level.
Citizen empowerment is an important trend and this has typically been an important element within technology assessment (TA), but also impact assessments include some wider involvement. However, systematic infrastructure or instruments for the involvement of lay people and a broader range of stakeholders in all assessments seem to be lacking, as does funding instrument and capacity building initiatives for enabling the participation of potentially marginal or socio-economic weak groups (for instance consumer and societal organisations without much financial resources). Standards for transparency in such involvement processes, in order to be able to assess the nature and quality of the involvement, would be useful.
Rapid technological change is a trend that implies that significant uncertainties may result, both with regard to environmental and human health risks, and with regard to societal and economic impacts. These uncertainties must be characterised and applying the precautionary principle must be considered. Moreover, such rapid change also affects important societal values. These must be considered and appropriately addressed in an anticipatory way, so that societal, value based technology governance can be carried out at an appropriate early stage.
Increased focus on sustainability is seen in some technology fields, such as bioenergy. However, sustainability – and especially its social pillar - can be better operationalised. Sustainability needs to be addressed as an integrative concept, and not split into separate ‘silo’ assessments that are not integrated. If sustainability is to be an important policy goal, better tools for integration of knowledge need to be developed. Even at a time of economic crises, the balance of ecological, economic and social concerns need to be considered. In such balancing, quantification has a place, but it must be scrutinised and placed within a wider perspective.
Policy integration is a response to trends (such as liberalisation) that appear to reduce the opportunities for state steering. Integration in assessment can facilitate policy integration and, to the extent that assessments influence policy, transparency about assumptions, methods and the practical aspects of the assessment process is crucial for the legitimacy of the policy.
Overall, we have found that the policy trends may have significant impact both on the development of EST, the possibilities for responsible governance of EST and the setups and practices of EST assessment. This merits devoting much more attention to the trends in EST assessment in general and especially in assessment processes that intend to integrate the assessment evidence base in a field into practical, multi-dimensional policy recommendations.
3.9 CRITERIA FOR AN INTEGRATED FRAMEWORK
From the case and domain studies, dialogues with assessment professionals and stakeholders, and the analysis of the integration dimensions, the EST-Frame project was able to better diagnose the needs for integration currently unmet in the EST field. These basic needs were described as criteria for an integrated framework and were subsequently used for developing the integrated approach in the project.
The three basic criteria for an integrated framework were identified as:
a) It needs to be focused on governance problems. The project’s case studies and domain studies showed that rich, problem-oriented assessments were scarce. This problem does not necessarily need to be a topic on the political agenda. It can well a topic that important stakeholders believe should be subject to policy making.
b) It needs to give guidance on how assessments in a larger body of assessments relevant for a particular problem, can be integrated. It is noted that integration of existing assessments may be facilitated by convening an assessment team consisting of assessment practitioners from the different assessment traditions.
c) It needs to provide guidance on how to ensure sufficient transparency in assessments for such problem oriented integration can take place. If problem-orientation, trans-domain interaction and integrating lessons from existing assessments are to be done, the assumptions of the different domain representatives and assessments need to be transparent. Assessments with incompatible assumptions may not be possible to integrate. Moreover, the situation analysis and method choice of the integrated process must be explicit and reflective.
These criteria were used to develop the integrated approach that was tested in the project, the Integrated EST Framework, ultimately finalised as the Trans-domain Technology Evaluation Process, TranSTEP.
From the work on the criteria for an integrated assessment framework we developed the following recommendations to policy makers:
1. In order to facilitate responsible research and innovation emerging science and technologies must be assessed in their practical contexts of use, taking into account the richness of impacts that appear in such concrete situations. Trans-disciplinary and trans-domain assessments must be carried out in order not to fragment complex real-life situations into generalised, abstract reductions.
2. Transparency of all assessments – also disciplinary assessments - is necessary for their inclusion into the evidence base for technology policy. In order to know whether an existing assessment can provide valid and relevant knowledge for solving the governance problem at hand the situation analysis and method choices must be justified and transparent. The EST-Frame project recommends that all assessments of new technology clearly show their situation analysis and method choices.
3. Assessment institution directors and managers should increase their strategic focus on the development of "home-grown" approaches to problem-oriented transdisciplinary research, to develop transdisciplinary competences, to foster connections and interaction with other assessment domains, and to secure transparency in assessments with regard to situation analysis, dialogue and method choice.
4. Assessment commissioners, for example in European DGs and members state ministries and agencies, should help to foster problem-oriented transdisciplinary assessments by implementing an approach such as the Integrated EST Framework as a way of securing transparency with regard to situation analysis, dialogue and method choice in assessments and assessment-based policy-development.
5. Policy developers in European DGs and member state ministries and agencies should work to secure transparency in the use of assessments in policy-development through clearer presentation of the interpretations made of assessments and the conclusions drawn. Policy makers must ensure that the evidence base for EST related policy-making is integrated in a transparent and balanced way, taking into account the different framings, methods and approaches of the assessments making up the evidence base.
6. European and member state policy makers should work to secure the implementation of responsible research and innovation in the Horizon 2020 program. Such requirements would place demands on assessment researchers, encouraging that they apply the quality criteria of problem-orientation and transdisciplinarity and that assessment research is carried out in ways which secure transparency with regard to situation analysis, dialogue and method choice.
3.10 THE TRANSTEP APPROACH - TRANS-DOMAIN TECHNOLOGY EVALUATION PROCESS
The main outcome of the EST-Frame project is the TranSTEP approach. This was thoroughly tested and discussed with assessment practitioners, commissioners and stakeholders in altogether six workshops. The motto for this approach is: Get an integrated perspective on complex technology issues by teaming up across established assessment communities. The approach is described in the following way:
TranSTEP is a conceptual assessment approach targeted towards policy makers commissioning assessments and professionals conducting assessments.
TranSTEP is an approach to the assessment of technologies or technological applications that present challenges related to complexity, uncertainty and controversy over facts and values. In such situations the legitimacy of any assessment may be challenged with respect to who participates, how the assessment is conducted and the quality and meaning of the results. TranSTEP offers an approach where the range of participants is widened, the assessment process itself is made transparent and the output has been subject to broad review. In this respect TranSTEP aims to confront the difficult discussions as an integrated part of the assessment, conducive to better robustness and legitimacy of its output.
TranSTEP is a conceptual guide for practical work. It gives commissioners and facilitators the necessary leeway to blend tools and methods from different assessment traditions in a flexible manner without sacrificing transparency and accountability in the process.
TranSTEP focuses on the enhancement of communication and interdisciplinary learning between different domains of expertise, because fragmentation of expertise is one of the main barriers to integrate factual evidence, values and normative perspectives across these domains.
Using the TranSTEP approach involves initiating and facilitating assessment groups composed of people from different assessment backgrounds to integrate assessment perspectives on complex technology issues. Participants in such processes can be assessment practitioners from domains such as economics, risk assessment, ethics, foresight, impact assessment or technology assessment, or from outside these domains. This is why TranSTEP is called a trans-domain approach. The various perspectives to be integrated are found in the participants’ professional background. In order to ensure that all relevant perspectives are brought in, other actors should also be involved, such as representatives from industry and public research, private sector stakeholders, public sector decision-makers or administrators, NGOs or the public.
The breadth of possible participation, and the focus on learning, requires a high degree of process design reflexivity amongst the assessment initiators and participants. TranSTEP involves collaborative situation analysis, transparent method reflection, and a continual use of facilitated dialogue.
An important part of the process is to assess existing knowledge and identify gaps where the knowledge base may be improved. The work may also involve initiating and carrying out new activities in order to strengthen this base. The purpose is to integrate existing knowledge that is compatible with the group’s problem formulation into an evidence base (where evidence must be understood in a wide sense, also potentially including input from the public) for responding to this problem. The approach ultimately aims at the production of an integrated assessment by the assessment group with a firm rooting in science, society and policy.
In order to disseminate this approach in a more user-friendly way than a report we created a designated website: http://transtepapproach.wordpress.com/. Please consult this website to find the details of the approach.
We are dedicated to disseminate this approach widely in order to increase its impact. Please see the next chapter for details on the further exploitation of these results.
Potential Impact:
4.1 POTENTIAL IMPACT OF THE PROJECT
As shown above the project has had outputs of different character; new knowledge, analytic instruments and a practical approach for integrated assessment of emerging science and technologies. The potential socio-economic impact of these activities are hard to estimate, let alone quantify. Here we will instead qualitatively describe the potential impacts of the project’s output and the measures we have taken to increase them.
A. The impact of an analytic framework to evaluate assessments and provide internal quality control
The comparative assessment in WP1 contributed to a better understanding of the respective roles of the various technology assessment frameworks. This work resulted in an analytic tool (the process characterisation table) for better appraising the assessments of emerging sciences and technologies and for more reflected use of assessment frameworks (see table 1 above). This tool was used as an analytic framework in the project and can also be used in future research and to improve the quality of assessments. In order to increase the impact of this tool we have disseminated it widely, both in Deliverable 1.1 and in Forsberg et al. 2014 (see list of the project’s publications below).
B. Better understanding of the assessment of governance of the selected case technologies
The case studies and the processes testing the integrated framework on the technology cases has provided more holistic understanding of these respective technologies; the main issues to consider, further assessment needs, and governance needs. The impact of these case studies have been strengthened by the inclusion of key stakeholders in two workshops in each case study (confer sections 3.2 to 3.5 for the outcomes of these studies). The results of these studies have been disseminated in Deliverables 2.1 3.1 4.1 and 5.1 and in De Bakker, De Lauwere, Hoes and Beekman 2014, Van Doren and Heyen 2014 and Boucher, Smith and Millar 2014.
C. Better understanding of how EST assessments are conducted and how they respond to a dynamic societal context
The systematic comparison of different advisory domains and assessment frameworks, as well as the cross-domain dialogues organised in the project, have facilitated a better understanding of the common challenges facing assessment practices and practitioners. A potential longer-term impact of taking such a comprehensive perspective may be to professionalise assessment practices across domains. The trends study allows such practices to better respond to a dynamic environment. These cross-domain analytic frameworks may be important for further quality control of EST assessment practices. In order to increase this potential impact we have disseminated this work in Deliverables 1.1 and 1.2 as well as in Forsberg, Thorstensen, Nielsen and De Bakker 2014 and Van Doren, Forsberg and Lindner 2014.
D. Integrated assessment of four technology cases
During the work in the project we organised an integrated assessment workshop for each case study. These workshops were not full implementation of TranSTEP, but workshops testing the initial phases of an integrated assessment process. These workshops pointed to specific actions that need to be conducted in the future.
The workshop on integrated assessment of cellulosic ethanol pointed to the need to take further actions on the following issues:
• Actions must be taken to reduce scientific uncertainty over crucial sustainability issues of biofuels and uncertainty over long-term performance and cost of advanced biofuels.
• Advanced biofuels, such as cellulosic ethanol, need to be considered against different alternatives and competing uses of biomass need to be further explored.
• Methodological challenges involved in the assessment of the impacts of first-generation ethanol seem to remain valid for the case of cellulosic ethanol. A transdisciplinary system-wide approach is needed.
The possibilities for following-up on these action points are being explored by the University of Nottingham team.
The integrated assessment workshop on nano food ended up in recommendations for the follow-up actions to be taken by the Dutch consortium NanoNextNL. NanoNextNL is a consortium of more than one hundred companies, universities, knowledge institutes and university medical centres, aimed at research into micro and nanotechnology. Noting the problem of the ‘waiting game’ (an unresolved situation where no one wants to be the first mover) the consortium was advised to organise a ‘ societal incubator’ where nano food industry could discuss with key societal stakeholders in a protected sphere. This is now being implemented. Inspired by the activities of EST-Frame NanoNextNL has asked the Rathenau Institute (a Dutch Technology Assessment agency) to write a proposal for the development a ‘societal incubator’ that could explore and support a broader dialogue with societal stakeholders.
The integrated assessment workshop in synthetic biology ended up focusing on a specific case example, genetically modified/engineered mosquitoes and flies using DNA-synthesis. This was given a definite problem framing and a list of questions the envisioned integrated assessment should tackle. The assessment was not completed, but the design for follow-up actions were made and the participants appeared willing to continue to engage in this process.
The integrated assessment workshop on the European cloud strategy concluded with two recommendations:
- An integrated assessment process concerning the European Cloud could pave the way for the development of a set of guidelines to defined groups of stakeholders to apply cloud-based solutions for certain sets of services. If such development was undertaken, it should be matched by strategies for the continued development of certification schemes based on open standards.
- Europe’s recent experiences in connection with data protection and privacy and the difficulty of governing these in a digital society have brought attention back to a long standing need to develop strategies and capacities for the adequately flexible development of regulation. An integrated assessment process could explore ways of developing practical capabilities for more responsive forms of ongoing societal regulation of data use.
Follow-up actions on the integrated assessment of the European cloud strategy were desired by the participants and are planned by the Danish Board of Technology.
E. An integrated approach to EST assessment: TranSTEP
The development work in WP6 has provided an integrated framework, TranSTEP, conducive to a better and more balanced assessment of emerging sciences and technologies.
The potential impacts of using a TranSTEP approach for EST assessment are:
- Better understanding of complex technology issues: The TranSTEP approach helps to ensure that the rich nature of complex technology issues are captured.
- Better understanding of one’s own and others’ assumptions, view and values: Through the systematic situation analysis and method reflection undertaken in a TranSTEP assessment, public and private decision makers and stakeholders will better understand other actors’ assumptions, views and values. The learning that results from actually working together in the TranSTEP group will likely influence the participants’ own understanding of the issue and of their own assumptions and default working methods.
- Better legitimacy and trust for advice and decisions: The nature of complex technology policy issues means that facts and values often intertwine and will frequently be contested by experts, stakeholders or the public. If such contestation is invited into the assessment group from the start, the assumptions of the assessments are likely to be much more broadly deliberated and thus more robust when presented to a wider audience.
- Increased transparency in assessments: TranSTEP provides explicit situation analysis and method reflection, which will make it easier for those outside the TranSTEP group to evaluate the assessment and the integration of results from TranSTEP. Explicit situation analysis, method reflection and assessment design are important quality criteria for assessments in general and enhance the ability of decision makers, stakeholders and the public to evaluate whether they agree with the assumptions of the assessment and corresponding results, and will therefore facilitate integration of assessments in the future.
- That the most urgent issues are addressed: Addressing a broad range of perspectives BEFORE designing assessment strategies may help to tailor assessments that target the most urgent issues, and not only the issues that established domains most easily can address. This may increase the efficiency of assessment efforts, by avoiding spending time and money on resolving issues that are not the real problem.
The project has taken the following measures to promote the use of this integrated framework model by advisors and policy makers when making assessment decisions:
- It was presented to European policy makers, stakeholders and researchers at the project’s final conference in Brussels in December 2nd 2014
- We have built up awareness of our approach by engaging end users, researchers and stakeholders in workshops throughout the project, informing on the websites and presenting preliminary results in conferences (see the dissemination section below)
- We sent out an email informing about our results to all the end users, stakeholders and researchers we have been in touch with in the project, approximately 120 persons (December 2014)
So far (February 24th 2015) the TranSTEP website has had 3 500 views, from 277 individual visitors.
4.2 A EUROPEAN APPROACH
This project required a European approach because technology developments in the EST field are global and are primarily treated strategically at a European level. It would be hard for a European country to develop a national strategy on, for instance, synthetic biology, in isolation of the developments in Europe because the technology developers are globally connected and must be met with harmonised international governance approaches. The European Commission funds developments in these fields and the assessments coming out of the kind of integrated approaches proposed here will be directly relevant for European research funding policies (e.g. in the framework programmes). Being thoroughly anchored in European assessment communities’ understanding of challenges in EST assessment means that TranSTEP is tailor made for the European policy context.
4.3 ASSUMPTIONS ON THE SUCCESS OF THE PROJECT
The EST-Frame project has evaluated the way assessment of emerging science and technologies is carried out and provided at the end of the project period recommendations for new practices. We could not expect to see changes in assessment practices already at the end of the project period. Success will rather have to be measured in terms of the support expressed for the results of the project and the stimulation of follow on activities. As the design of the project has to a large extent been bottom-up, with extensive user involvement we anchored our integrated assessment approach with a number of relevant potential end users already as a part of the development process, securing a relatively sound basis of support.
Some quotes from the project’s last end user workshop (Oslo, October 2014) confirms this:
● “Is it revolutionary? Yes, perhaps, because the concept might consider different framings from different problem owners and therefore might be able to internalise plurality and different perspectives, which is powerful.”
● “The approach appears to be welcomed by potential end users, which is great.”
● “An approach like TranSTEP can help deliver transparency in the process of problem formulation, with the right set of people involved & a common understanding (if not agreement) can be achieved.”
The feedback on the need for an integrated approach such as the TranSTEP approach from the Integrated Assessment testing workshops also provides an indication of support:
The average score on the statement ‘There is a need for an integrative process such as the Integrated EST framework in this field’ was in the synthetic biology workshop 4,2 (on a scale from 1 to 5 where 1 was ‘completely disagree’ and 5 was ‘completely agree’). The average score on ‘There is a need for an integrative process such as the Integrated EST framework in other fields’ was in the same workshop 4,1. The scores were identical in the biofuels workshop. In the cloud computing workshop the scores were, respectively, 3,7 and 4,3. In the nano food workshop the average scores were, respectively, 4.3 and 4.1.
All in all, this allows us to claim that the participants that have been involved in the discussions and testing of the EST-Frame integrated approach seem to support this concept. The feedback we received at the project’s final conference also seemed to confirm broad support for the TranSTEP approach.
4.4 MAIN DISSEMINATION ACTIVITIES
The project has had the following targeted audiences:
A. Policy makers
B. Assessment commissioners
C. Assessment practitioners
D. Technology innovators
E. Stakeholders
F. Researchers (science, social science and humanities)
G. The general public
Policy makers, assessment commissioners, assessment practitioners, stakeholders and researchers have been extensively involved in the project through the project’s workshops and conference. The general public has been informed mostly through the public website and articles in the media.
The project’s workshops and conference
An important dissemination action has been all the workshops organised throughout the project.
- November 1st 2012: A Dutch expert workshop on assessment of nano food took place in Utrecht with six external assessment experts/stakeholders.
- November 9th 2012: A German expert workshop on assessment of synthetic biology with seven external assessment experts and stakeholders took place in Karlsruhe.
- November 14th 2012: A Danish video conference two external assessment experts and stakeholders took place.
- March 7th 2013: A British expert workshop on assessment of biofuels, with seven external assessment experts and stakeholders took place in London.
- April 16-17 2013: A project practitioner workshop with 32 participants (including EST-Frame researchers) was held in Copenhagen.
- February 19-20 2014 : “The Integrated EST Framework and Its Application to Synthetic Biology”, integrated assessment testing workshop with 11 international participants, was organised by Fraunhofer ISI, Berlin,
- March 10-11 2014 : “A transdisciplinary dialogue on the opportunities and challenges of cellulosic ethanol in the UK”, integrated assessment testing workshop with eleven external participants, organised by University of Nottingham, Manchester
- March 26-27 2014 : “Integrated appraisal of ‘nanofood’ in the Netherlands”, ”, integrated assessment testing workshop with 13 external participants was organised by LEI in Driebergen,
- March 28 2014 : “EST-Frame Workshop on Integrated assessment of emerging technologies”, organised by HiOA, Brussels,
- April 31-May 1 2014 : “Testing the integrated framework in the case of cloud computing”, integrated assessment testing workshop with eight external participants organised by the Danish Board of Technology, Copenhagen
- October 29-30 2014: The project’s second general practitioner workshop with 12 external participants, was organised in Oslo by HiOA
- December 2 2014: Final conference on integrated assessment of emerging science and technologies, with approximately 45 participants, Brussels.
Public deliverables
All public deliverables are accessible through www.estframe.net:
- EST-Frame deliverable 1.1 Frameworks for assessing societal aspects of emerging science and technologies.
- EST-Frame deliverable 1.2 The current and future context for EST analysis.
- EST-Frame deliverable 1.3 Criteria for an integrated analytical framework.
- EST-Frame deliverable 2.1 Nano food: technology, governance context and assessments
- EST-Frame deliverable 3.1 Synthetic biology: technology, governance context and assessments
- EST-Frame deliverable 4.1 Biofuels: technology, governance context and assessments
- EST-Frame deliverable 5.1 Security and emerging ICTs: technology, governance context and assessments
- EST-Frame deliverable 6.6 An integrated framework for assessing societal impacts of emerging science and technologies. Website: http://transtepapproach.wordpress.com/
- EST-Frame deliverable 6.7 An integrated framework for assessing societal impacts of emerging science and technologies.
Scientific articles
• Forsberg, E-M., Quaglio, G., O’Kane, H., Karapiperis, T., van Woensel, L. and Arnaldi, S. 2015. Assessment of science and technologies: Advising for and with responsibility. Technology in Society, 42: 21-27.
• Reiss, T. and Millar, K. 2014. Introduction to special section: Assessment of emerging science and technology: Integration opportunities and challenges. Science and Public Policy, 41: 269-271
Link: http://spp.oxfordjournals.org/content/41/3/269.abstract.html?etoc
• Van Doren, D. and Heyen, N. B. 2014. Synthetic biology: Too early for assessments? A review of synthetic biology assessments in Germany. Science and Public Policy, 41: 272-282
Link: http://spp.oxfordjournals.org/content/41/3/272.abstract.html?etoc
• Boucher, P., Smith, R. and Millar, K. 2014. Biofuels under the spotlight: The state of assessment and potential for integration. Science and Public Policy, 41: 283-293
Link: http://spp.oxfordjournals.org/content/41/3/317.abstract.html?etoc
• De Bakker, E., de Lauwere, C., Hoes, A-C. and Beekman, V. 2014. Responsible research and innovation in miniature: Information asymmetries hindering a more inclusive 'nanofood' development. Science and Public Policy, 41: 294-305
Link: http://spp.oxfordjournals.org/content/41/3/294.abstract.html?etoc
• Forsberg, E-M., Thorstensen, E., Nielsen, R. Ø. and de Bakker, E. 2014. Assessments of emerging science and technologies: Mapping the landscape. Science and Public Policy, 41: 306-316.
Link: http://spp.oxfordjournals.org/content/41/3/306.abstract.html?etoc
• Van Doren, D., Forsberg, E-M, and Lindner, R. 2014. Are assessments responding to a dynamic environment? Evidence from four emerging techno-scientific domains.
Science and Public Policy, 41: 317-331.
Link: http://spp.oxfordjournals.org/content/41/3/317.abstract.html?etoc
• Forsberg, E-M. and de Lauwere, C. 2013. Integration needs in assessments of nanotechnology in food and agriculture. In Etikk i Praksis. Nordic Journal of Applied Ethics, 7, 1: 38-54: http://tapir.pdc.no/index.php?el=Kapittel&p=EIP&seks_id=77394
• Beekman, V. and Dagevos, H. 2013. The knowledge society as pleonasm: towards mobilisation of social intelligence in the agricultural and food domain. In: H. Röcklinsberg and P. Sandin (eds.) The ethics of consumption: The Citizen, the market and the law, Wageningen Academic Publishers, 353-357
• E-M Forsberg. 2012. Integrated assessments of emerging food technologies - some options and challenges. In Potthast, T. and Meisch, S. Climate change and sustainable development. Ethical perspectives on land use and food production. Wageningen Academic Publishers.
Conference/seminar panels and presentations
- Erik de Bakker and Ellen-Marie Forsberg were part of a workshop on integrated assessment at the Snet 2012 conference, Enschede, 22-25 October 2012.
- Erik de Bakker, Volkert Beekman, Anne-Charlotte Hoes and Carolien de Lauwere gave a paper presentation on the topic Nanotechnology in food: public concerns, narratives and assessment needs at the Snet 2012 conference.
- Ellen-Marie Forsberg presented EST-Frame results at a workshop on ‘Changes in ethical, social and environmental assessments of science and technologies’ at the Nordic STS conference, organised by our ‘sister project’ EPINET, Trondheim, 24 April 2013: http://www.ntnu.no/documents/10265/18878035/sessions+on.pdf/e262a217-166b-4c64-a972-ce54b45e9ed7
- EST-Frame had a workshop at the PACITA conference on Technology Assessment, March 15th 2013, 09.45 Prague: Integrated assessment of emerging science and technologies. The workshop consisted of a presentation of results from the EST-Frame case studies, as well as a presentation of our preliminary approach to integrated assessment. The workshop was chaired by Professor Frans Brom from the Rathenau Institute. From EST-Frame participated Philip Boucher, Anne-Charlotte Hoes, Nils Heyen, Rasmus Øjvind Nielsen and Ellen-Marie Forsberg. See description: proposal_workshop_pacita_conference_est_frame.docx. See also http://pacita.strast.cz/en/conference/general-info.
- Volkert Beekman presented a paper (co-authored with H Dagevos), The knowledge society as pleonasm: towards mobilisation of social intelligence in the agricultural and food domain, at the 2013 Conference of the European Society for Agricultural and Food Ethics (EurSAFE), Uppsala, September 2013; Kate Millar was also in attendance in Uppsala
- Ellen-Marie Forsberg gave the presentation ‘Sustainability in Assessment of Emerging Science and Technologies’ at the 2013 AP Safe Conference, Bangkok, Thailand, 29.11.13
- Ellen-Marie Forsberg gave the presentation ‘The need for integration in assessing emerging science and technologies: Results from the EST-Frame project’ at the Workshop on Responsible Governance of Science and Technologies, at the European Parliament STOA panel, Brussels 19 March 2014
- Thomas Reiss gave the lecture “Integrierte Bewertung neuer Technologien am Beispiel der Synthetischen Biologie“ (integrated assessment of emerging technologies - the example of synthetic biology) at the University of Frankfurt, Germany, on July2nd, 2014
- Ellen-Marie Forsberg gave the presentation ‘The potential for an integrated assessment framework for new technologies’ at Europäische Akademie, Summer school ‘Analysing the Societal Dimensions of Synthetic Biology’, Berlin 19.09.14
- Ellen-Marie Forsberg gave the presentation ‘Responsible Research and Innovation and assessment of emerging science and technologies’ at the EU Spri 2014 Conference, Manchester, 20.06.14
- Erik de Bakker presented a paper (co-authored with Ellen-Marie Forsberg), ‘Responsible Innovation & Innovation Ethics: Advisory Domains and Circles of Legitimacy’, at the European Business Ethics Network (EBEN) Annual Conference, June 12th-14th 2014 in Berlin.
- Rob Smith presented at the Biennial Conference of the European Association for the Study of Science and Technology (EASST), 17-20 Oct 2012
- Rob Smith presented at the 4th Annual meeting of The Society for the Study of Nanoscience and Emerging Technologies (S.NET) 22-26 Oct 2012.
- Kate Millar and Ellen-Marie Forsberg attended and participated in the Rome SIS-RRI Conference: Science, Innovation and Society – Achieving Responsible Research and Innovation; 19-21 November 2014
- Barbara Ribeiro presented a paper for Horizon Digital Research Economy, University of Nottingham, Assessing the impacts of emerging technologies: Which way forward?, 10 April 2014
- Barbara Ribeiro presented a paper on ”Social sustainability of ethanol and responsible innovation” at the Department of Science and Technology policy, University of Campinas, Sao Paulo, Brazil, 15 August 2014.
- Barbara Ribeiro presented a paper co-authored with the Nottingham team (Ribeiro, B.E. Smith, R., Millar, K.) on “Governing and assessing the impacts of emerging technologies: Which way forward?”, 4S Conference, Buenos Aires, Argentina, 23 August 2014.
Internet and social media:
We have also disseminated the project in various media and arenas:
- The project has an updated internet site: www.estframe.net and facebook site: https://www.facebook.com/pages/EST-Frame-project/126774937460322
- The outcome of the project, the TranSTEP approach, is disseminated on a dedicated website, designed to enable broad uptake of the results: https://transtepapproach.wordpress.com/
- A flyer is developed describing TranSTEP in short: http://estframe.net/publications/content_1/text_721891ce-f43b-460e-80ed-339c02c7134d/1417094492092/transtep_in_a_nutshell.pdf
- A short ‘elevator pitch’ of TranSTEP is published on www.estframe.net: http://estframe.net/publications/content_1/text_721891ce-f43b-460e-80ed-339c02c7134d/1417095522034/transtep_elevator_pitch.pdf
- A flyer was developed at the start of the project. This has been handed out on several occasions and is often attached when contacting people about the project. See http://estframe.net/publications/content_1/text_721891ce-f43b-460e-80ed-339c02c7134d/1342029518319/estflyer_rgb_final_for_ordinary_printer.pdf
- The project was presented in EurSafe news, vol. 14, no. 1, April 2012, www.eursafe.org
Media presentations
- Article about the EST-Frame project at the HiOA wepages: http://www.hioa.no/Aktuelle-saker/Stor-interesse-for-HiOA-ledet-EU-prosjekt
- EST-Frame final conference was reported on the Bassetti Foundation website: http://www.fondazionebassetti.org/en/focus/2014/12/on_december_2_the_est-frame.html
- The Norwegian Government’s delegation to the European Union reported from the project’s final conference: https://www.regjeringen.no/nb/aktuelt/Anbefaler-mer-forskningsdialog/id2344459/
- Article on the STOA event at the HiOA webpages: http://www.hioa.no/News/AFI-forsker-informerer-EU-parlamentet
4.5 EXPLOITATION OF RESULTS
Further dissemination
We will continue to disseminate the results of the project even after its closure.
Scientific publications: The TranSTEP approach and outcomes from the empirical work will be disseminated to the wider academic public through the following planned articles:
• Ellen-Marie Forsberg, Volkert Beekman, Barbara Ribeiro, Nils Heyen, Rasmus Øjvind Nielsen, Lars Klüver, Thomas Reiss and Kate Millar. Needs for integration in assessments of emerging science and technologies. (Almost finalised) Science, Technology and Human Values (TBC).
• Ribeiro, B.E. Smith, R., Millar, K. Mapping and understanding Responsible Research and Innovation (Under development). Research Policy (TBC)
• Ribeiro, B.E. Smith, R., Millar, K. Interdisciplinary problem framing in the appraisal of emerging biofuel technologies. Technological Forecasting and Social Change (TBC)
Conference presentations: Pacita Conference 2015, Berlin, February 25 2015: Panel session: Integrated Approaches in Technology Assessment. Chairs: Nils Heyen and Rasmus Øjvind Nielsen. Presentations:
• J.C. Schmidt: Problem oriented interdisciplinarity and integration: Methodological reflections and considerations.
• S. Lingner: Interdisciplinary integration in technology assessment. A report from practice.
• E-M Forsberg: TranSTEP. Getting an integrated perspective on complex technology issues by teaming up across established assessment communities.
Other dissemination: EST-Frame is presented in the first edition of the newsletter of the SNet organisation, March 2015.
The DBT has implemented the EST-Frame criteria for integrated assessments in a method database for situation analysis, method selection and project self-evaluation. The database covers more than 55 methods, and for each method criteria are evaluated, such as the method’s ability to include multiple domain perspectives, to create collaboration across domains, or to integrate the work of multiple domains. The database includes method data from the foresight project ex ante / ex post evaluation tool DoingForesight.org and is expanded with methods from the Engage2020 project (engage2020.eu). The database will make up a major source for future tools for situation analysis, ex ante project evaluation, method selection, ex post evaluation and other function coupled to the project definition and setup phases.
Further exploitation of the results
We see two avenues for further exploitation of the results: continued research in the field of assessment of emerging science and technologies, and further development and application of the TranSTEP approach. We will pursue both strategies.
One action has already been implemented:
– Resiliency and adaptation to climate change in regional strategies: This is a bilateral Czech-Norwegian research project, funded by Norway Grants in the period of 2015-2016, where HiOA will apply TranSTEP on regional climate change adaptation strategies.
Partners in the EST-Frame consortium have also included versions of TranSTEP in the following research and development project proposals:
- ERASynBio 2nd Joint Call for Transnational Research Projects: Building Synthetic Biology Capacity Through Innovative Transnational Projects (July 3rd 2014): A proposal with the following title was submitted by a trinational consortium: BioMETCHEM Methane as a novel platform substrate. As a “strategic element” the TranSTEP approach was included, two partners of the EST-frame team (Fraunhofer ISI and University of Nottingham) were responsible for the strategic element. The proposal was unfortunately rejected.
- The Assisted Living project: In this project proposal, submitted to the Research Council of Norway by HiOA in February 2015, a TranSTEP group will follow a technology development process within welfare technologies throughout the whole project (2016 – 2020). The proposal is currently under evaluation.
- Atmospheric Carbon Trap: In this project proposal (coordinated by the University of Edinburgh) to the FET Open Programme a version of the TranSTEP approach was going to be used by HiOA to assess an Atmospheric Carbon Trap technology. The proposal was unfortunately rejected.
Other avenues for exploitation of the results
We will continue to promote the approach to relevant assessment commissioners and practitioners, through direct contact and through participation in relevant conferences (for instance conferences on risk assessment, integrated assessment, economic assessment, etc.).
Moreover, we will consider ways of further adapting the approach. It was suggested by Philippe Galiay at the project’s final conference that it would be of interest to the EC to further develop TranSTEP into a challenge based approach, rather than a technology based approach. This is duly noted and is a recommendation we will follow up on in the near future.
The project’s advisory committee as well as the end users involved in our workshops have been important for suggesting how we can exploit our results further. We have received the following advice:
- Make a practical guidance on how the different domains work and how information from the different domains can be balanced and integrated.
- Make a guidance on ethical review of nanotech and biotech projects in Horizon 2020.
- Contribute to OECD’s current work in developing impact assessment guidance.
- Contribute to developing guidelines for integrated sustainability impact assessment (SIA) for trade negotiations, such as The Trans-Atlantic trade negotiations.
- The UN international integrated assessment is another potential client. UN Institute for Crime (UNICRI) published in 2011 a report on biosecurity in NanoBio (incl. synthetic biology) where they ask for joint (i.e. inter-governmental) assessment methodologies. Joint assessment methodologies already exists for radiation in IAEA, but TranSTEP could be of use in other areas.
- An adapted TranSTEP approach could be of use to analyse what questions have been posed and answered in a research and innovation process.
- TranSTEP could function as a conflict resolution tool, where it is useful to connect to other people’s agendas.
The members of the EST-Frame consortium are determined to continue to exploit the results from EST-Frame, as a consortium and as individual partners.
List of Websites:
Project website: www.estframe.net
Oslo and Akershus University College, Research Group on Responsible Innovation, Ellen-Marie Forsberg, Ellenmarie.Forsberg@hioa.no
University of Nottingham, Centre for Applied Bioethics, Kate Millar, kate.millar@nottingham.ac.uk
The Danish Board of Technology Foundation, Lars Klüver, LK@tekno.dk
Wageningen Agricultural Research Institute – LEI, Erik de Bakker, erik.debakker@wur.nl
Fraunhofer Institute for Systems and Innovation Research ISI, Thomas Reiß, thomas.reiss@isi.fraunhofer.de
