Periodic Reporting for period 3 - FluidKnowledge (How evaluation shapes ocean science. A multi-scale ethnography of fluid knowledge.)
Okres sprawozdawczy: 2022-03-01 do 2023-08-31
New forms of evaluation are reconfiguring science in ways we are only beginning to understand. Through the rich case of ocean science, this project addresses a key challenge in social-scientific research regarding how evaluations are implicated in scientific understandings of the world. Ocean science is increasingly multivalent. Not only is it expected to contribute to a more systemic understanding of the ocean as an ecosystem, it is also called on to analyze environmental effects of climate change, and help fight effects of intensified exploitation. At the same time, it operates in a highly research-focused and efficiency-oriented academic system whose norms partly work against societal relevance. The ambition of FluidKnowledge is to 1) investigate how research agendas are shaped in ocean scientific research; 2) analyze how the value of ocean science is enacted in European and national science policy contexts; 3) develop concepts on the basis of the outcomes of 1) and 2), to theoretically grasp how research evaluation shapes knowledge making. Ocean science provides a planet-critical research site in which to analyze how steering efforts toward interdisciplinary engagement and societal relevance relate to other norms and criteria of scientific quality (e.g. excellence) in actual practice. This project creates a new interface between longitudinal scientometric analysis and rich ethnographic studies. This paves the way for a new interdisciplinary field. A second contribution is conceptual. The project will build theory that encourages a more comprehensive understanding of how evolving evaluation and knowledge production are mutually implicated. A third novelty is the focus on ocean science. Systematic analysis of its workings and policy implications is crucial for understanding a world in which trust in scientific knowledge is no longer obvious.
In the first half of the FluidKnowledge project reporting period, significant work has been performed towards the project’s main objectives. Ethical approval has been secured, research protocols have been developed, and the PhD candidates have written their PhD proposals. Key documents were collected on global and EU ocean science policy, and the team organised a conceptual workshop with the entire scientific advisory board in March 2020. The first key project output was published in June of that year (Marres & de Rijcke, 2020). The PI co-authored this article with a member of the project’s scientific advisory board. It represents an important innovation for mixed methods research on emerging interdisciplinary research communities.
FluidKnowledge investigates the past, present and future of evaluating ocean science. Between April and August 2020, the team has studied three marine institutions in three different national contexts using digital mixed methodologies. The initial findings of this investigation were presented at the largest conference in the field of Science and Technology Studies (STS), 4S-EASST, held in August 2020. Since then, members of the team have continued to perform large-scale quantitative analyses on how research priorities in ocean science have evolved historically, and up until now. Which lines of inquiry became hot topics, and which died out? Who became global players, who ended up in the periphery? Who published on what, in which interdisciplinary collaborations? Novel methods have been developed which combine descriptive statistics and network analyses. This has enabled a detailed exploration of the epistemological and institutional history of global and European ocean science (Varga & de Rijcke, 2021).
Regarding the present, the FluidKnowledge project asks what currently matters in the field. For that purpose, the team is analysing the knowledge cultures in different European marine institutes. This is where research evaluation and the daily practice of research come together. PhD students are studying how researchers do their work: in labs, on ships, during team meetings, and in supervisory meetings. As part of this work, the project includes a cross-field comparison between the ocean science fields of paleoceanography and deep-sea microbiology. In addition, the project focuses on small-scale and coastal ocean modelling in ocean science, and on the work of incorporating ecosystem concerns in European fisheries modelling.
Regarding the future, the project studies priority setting in science policy. Members of the team analyse how expectations about the role of ocean science are formulated in European policies. Which values and priorities will ocean science be held accountable to? How do policy makers and ocean scientists work together? How and when does this fail? As part of this work, the team now has field access to the largest European marine research policy body. FluidKnowledge team members are collaborating with its leadership, secretariat, and relevant working groups to better understand how scientists intervene in policy, by studying the development of new policy documents.
FluidKnowledge investigates the past, present and future of evaluating ocean science. Between April and August 2020, the team has studied three marine institutions in three different national contexts using digital mixed methodologies. The initial findings of this investigation were presented at the largest conference in the field of Science and Technology Studies (STS), 4S-EASST, held in August 2020. Since then, members of the team have continued to perform large-scale quantitative analyses on how research priorities in ocean science have evolved historically, and up until now. Which lines of inquiry became hot topics, and which died out? Who became global players, who ended up in the periphery? Who published on what, in which interdisciplinary collaborations? Novel methods have been developed which combine descriptive statistics and network analyses. This has enabled a detailed exploration of the epistemological and institutional history of global and European ocean science (Varga & de Rijcke, 2021).
Regarding the present, the FluidKnowledge project asks what currently matters in the field. For that purpose, the team is analysing the knowledge cultures in different European marine institutes. This is where research evaluation and the daily practice of research come together. PhD students are studying how researchers do their work: in labs, on ships, during team meetings, and in supervisory meetings. As part of this work, the project includes a cross-field comparison between the ocean science fields of paleoceanography and deep-sea microbiology. In addition, the project focuses on small-scale and coastal ocean modelling in ocean science, and on the work of incorporating ecosystem concerns in European fisheries modelling.
Regarding the future, the project studies priority setting in science policy. Members of the team analyse how expectations about the role of ocean science are formulated in European policies. Which values and priorities will ocean science be held accountable to? How do policy makers and ocean scientists work together? How and when does this fail? As part of this work, the team now has field access to the largest European marine research policy body. FluidKnowledge team members are collaborating with its leadership, secretariat, and relevant working groups to better understand how scientists intervene in policy, by studying the development of new policy documents.
The FluidKnowledge project moves beyond the state of the art in the following ways. It moves beyond merely providing methodological critique of evaluation. Instead, the project understands research evaluation as a thoroughly social process. It also moves beyond focusing on formal evaluation systems. Instead, the project studies how evaluation plays out on the ground, in real organisational settings. Finally, the project moves beyond the idea of research institutes as homogeneous. Instead, the project approaches research as placed between many different demands. Think of scientific values versus market logics, or of science that scores versus science that matters.
Upon completion, the project will have collected hard, empirical evidence on how evaluations affect actual scientific conduct in ocean science, by combining scientometrics with ethnography. The project will also have advanced conceptual research about combining quantitative science studies and science and technology studies. Combining these two fields and methodological traditions – which are rarely combined due to institutional and methodological reasons – is highly novel. It is also essential for developing a new, interdisciplinary field at the intersection of Evaluation Studies and Science and Technology Studies (one of the main goals of FluidKnowledge). Thirdly, the project will have developed a new theory of evaluation based on this evidence: a theory for understanding interactions between research conduct and research evaluation.
Upon completion, the project will have collected hard, empirical evidence on how evaluations affect actual scientific conduct in ocean science, by combining scientometrics with ethnography. The project will also have advanced conceptual research about combining quantitative science studies and science and technology studies. Combining these two fields and methodological traditions – which are rarely combined due to institutional and methodological reasons – is highly novel. It is also essential for developing a new, interdisciplinary field at the intersection of Evaluation Studies and Science and Technology Studies (one of the main goals of FluidKnowledge). Thirdly, the project will have developed a new theory of evaluation based on this evidence: a theory for understanding interactions between research conduct and research evaluation.