Periodic Reporting for period 2 - NanoBubbles (Nano bubbles: how, when and why does science fail to correct itself?)
Periodo di rendicontazione: 2022-12-01 al 2024-05-31
Science relies on the correction of errors to advance, yet in practice scientists find it difficult to erase erroneous, exaggerated, or even fraudulent claims from the scientific record. In recent years, both the scientific and popular press have declared a “replication crisis” casting doubt on many scientific findings – doubt that is amplified both by the actions parties with an ideological or commercial interested in undermining specific sciences (anti-vaxxers, climate denialists, covid skeptics, etc.) as well as by the discovery of actual fraud in social psychology, behavioral economics, semiconductor physics, and other fields. Unfortunately, we still know little about how non-replicated or even fraudulent claims can be removed from the scientific record; nor do we fully understand the systemic conditions that give rise to fraudsters and denialists; and equally importantly there is almost no non-anecdotal evidence regarding the actors (e.g. journal editors) and mechanisms that hinder the correction of fraud and error. This project combines approaches from the natural, engineering, and social sciences and the humanities to understand how error and fraud in science arise and are maintained and propagated, and how error correction works and what obstacles it faces.
The project’s main focus is nanobiology, a highly interdisciplinary field founded around the year 2000 that has already seen multiple episodes of overpromising and promotion of erroneous claims. We aim to examine three such “bubbles”: the claim that nanoparticles can cross the blood-brain barrier; that nanoparticles can penetrate the cell membrane; and the promotion of the “protein corona” concept to describe ordinary adsorption of proteins on nanoparticles. Since findings based on error (non)correction in nanobiology should be generalizable to other new, highly interdisciplinary fields such as synthetic biology and artificial intelligence we complement our study of nanoscience with cases drawn from other fields that have experienced similar problems.
We trace claims and corrections in various channels of scientific communication (journals, social media, advertisements, conference programs, etc.) via innovative digital methods. We examine error (non)correction practices in scientific conferences via ethnographic participant-observation. We follow the history of conferences, journals, and other sites of error (non)correction from the 1970s (before nanobio per se existed) to the present. And we attempt to replicate nanobiological claims and, in case of non-replication, document obstacles to correcting those claims. Finally, we will spark a dialogue within the nanobiology community by organizing workshops and events at conferences for practitioners. Through the study and practice of nanobiology, we will analyse how, when and why science fails to correct itself, and explore ways to improve the reliability and efficiency of the scientific process.
The project’s main focus is nanobiology, a highly interdisciplinary field founded around the year 2000 that has already seen multiple episodes of overpromising and promotion of erroneous claims. We aim to examine three such “bubbles”: the claim that nanoparticles can cross the blood-brain barrier; that nanoparticles can penetrate the cell membrane; and the promotion of the “protein corona” concept to describe ordinary adsorption of proteins on nanoparticles. Since findings based on error (non)correction in nanobiology should be generalizable to other new, highly interdisciplinary fields such as synthetic biology and artificial intelligence we complement our study of nanoscience with cases drawn from other fields that have experienced similar problems.
We trace claims and corrections in various channels of scientific communication (journals, social media, advertisements, conference programs, etc.) via innovative digital methods. We examine error (non)correction practices in scientific conferences via ethnographic participant-observation. We follow the history of conferences, journals, and other sites of error (non)correction from the 1970s (before nanobio per se existed) to the present. And we attempt to replicate nanobiological claims and, in case of non-replication, document obstacles to correcting those claims. Finally, we will spark a dialogue within the nanobiology community by organizing workshops and events at conferences for practitioners. Through the study and practice of nanobiology, we will analyse how, when and why science fails to correct itself, and explore ways to improve the reliability and efficiency of the scientific process.
The first half of the project has, naturally, focused extensively on setting up procedures and infrastructure; publications and other outputs are therefore mostly in-progress rather than delivered. However, we do have several significant publications listed later in this report. In terms of procedures and infrastructure, we have established a secure site for storing and sharing project data and common Slack and Zotero sites for project communications. All project personnel have been hired into positions that match those in our project proposal (with minor adjustments in a small number of cases). All PhD and postdoctoral positions also have an interdisciplinary supervision team in place. The project as a whole has consistently submitted our required ethics and data management deliverables on time, and each individual site has received the required local clearances for members’ ethics protocols and data management plans.
The full project team meets on-line every other week, and in-person roughly every six months (each time hosted by one of the co-PIs’ sites) for training, team-building, presentation of interim results, feedback, planning, discussion of ethics, and collaboration. Project members have also organized events at which they and project colleagues have exchanged results with researchers external to the project – e.g. at conferences in Paris in November 2022 and 2023 which about ten project members attended (each time), and another conference planned for Paris in 2024 with similar participation. Bi- and multi-lateral exchanges among project members are common, including visits to each others’ sites, attendance at the same conferences, and organization of workshops and conference panels together. Project members cooperate regularly, for example in the construction of corpora that can be queried with natural language processing tools.
Finally, the proposed projects are well underway at all project sites. All the methods described in the project proposal (computational tools for assembling, scraping, and/or querying corpora and detecting claims, arguments, citation types, etc.; ethnographic interviews and participant observation; post-publication peer review; attempted replication of four influential nanobiology papers; social scientific surveys; archival research; conceptual analysis) are being conducted by at least one member of the project, usually in collaboration with other project members. We have also added several smaller sub-projects designed to elicit aspects of error (non)correction that will complement our findings for the three nanobiology “bubbles” described in the proposal; these include studies of hype and narratives as contributions to bubbles in nuclear energy and artificial intelligence; studies of resources that inflate bubbles in nanoelectronics or in areas related to the oil industry; a study of ghostwriting and the propagation of erroneous claims about opioids; and studies of “tortured phrases,” “sneaked references,” and other techniques used by paper mills. Several project sub-studies have already received significant exposure in the scientific and even popular press and have been featured in influential events in the metascience, science reform, research integrity, and open science communities.
The full project team meets on-line every other week, and in-person roughly every six months (each time hosted by one of the co-PIs’ sites) for training, team-building, presentation of interim results, feedback, planning, discussion of ethics, and collaboration. Project members have also organized events at which they and project colleagues have exchanged results with researchers external to the project – e.g. at conferences in Paris in November 2022 and 2023 which about ten project members attended (each time), and another conference planned for Paris in 2024 with similar participation. Bi- and multi-lateral exchanges among project members are common, including visits to each others’ sites, attendance at the same conferences, and organization of workshops and conference panels together. Project members cooperate regularly, for example in the construction of corpora that can be queried with natural language processing tools.
Finally, the proposed projects are well underway at all project sites. All the methods described in the project proposal (computational tools for assembling, scraping, and/or querying corpora and detecting claims, arguments, citation types, etc.; ethnographic interviews and participant observation; post-publication peer review; attempted replication of four influential nanobiology papers; social scientific surveys; archival research; conceptual analysis) are being conducted by at least one member of the project, usually in collaboration with other project members. We have also added several smaller sub-projects designed to elicit aspects of error (non)correction that will complement our findings for the three nanobiology “bubbles” described in the proposal; these include studies of hype and narratives as contributions to bubbles in nuclear energy and artificial intelligence; studies of resources that inflate bubbles in nanoelectronics or in areas related to the oil industry; a study of ghostwriting and the propagation of erroneous claims about opioids; and studies of “tortured phrases,” “sneaked references,” and other techniques used by paper mills. Several project sub-studies have already received significant exposure in the scientific and even popular press and have been featured in influential events in the metascience, science reform, research integrity, and open science communities.
Our progress to this point and beyond can best be summarized by breaking our larger research question into its constituent parts:
1. What circumstances encourage scientists to make sloppy, exaggerated, hyped, erroneous, fraudulent, or nonsensical claims?
- We have gone beyond the state of the art in understanding the contribution of resource providers (particularly in the pharmaceutical, oil, and microelectronics industries) in encouraging hype and fraud in nanobiology, biotechnology, artificial intelligence, and other fields.
2. What techniques do fraudsters and hypesters use to put forward and to propagate erroneous claims?
- We have gone beyond the state of the art in analyzing the grammar of scientific promises, and are making headway in developing digital tools for identifying promises in large corpora.
- We are making headway in developing philosophical methods for identifying claims and tracing the uptake of new concepts and vocabulary, and have made progress in translating those methods into digital tools that can be applied to large corpora.
- We have developed several cutting-edge digital tools for discovering and cataloguing new forms of scientific misconduct, especially misconduct involving generation of nonsensical papers or skewing of citation statistics.
- We have assembled detailed timelines of specific cases of exaggeration and distortion, in particular in the creation of scientific articles crafted to support a pharmaceutical company’s opioid marketing; and also in the coordination of articles, patents, contracts, investment pitches, and other media to support commercialization of spherical nucleic acids.
3. How, when, and why do scientists attempt to correct the scientific record?
- We have conducted a survey of practicing scientists soliciting their answers to these questions.
- We have conducted ethnographic interviews with scientists who were in a position to offer corrections to the record, some of whom have and some haven’t.
- We have conducted participant observation with scientists (some working within our own project) who have attempted/are attempting to offer corrections.
- We have compiled and begun to analyze corpora of “retractions” (and related terms) in the scientific press to understand the circumstances under which retractions were demanded and/or corrections were offered.
- We have identified articles that contributed to belief in a controversial claim in nanobiology, have offered annotations of those articles in post-publication peer review, and have begun an attempted replication of those papers in an attempt to compare our results to the originals.
4. What happens when scientists attempt to correct the record?
- We have conducted survey, ethnographic, digital, and historical research tracing past attempts at correction at conferences and in journals and the reactions to those attempts by editors, conference attendees/organizers, the targets of correction, and bystanders.
- We have attempted our own corrections of errors and fraud via our home institutions, journals, social media, the popular press, national research integrity offices, post-publication peer review, and other channels. We have documented those attempts and also studied them via auto-ethnography and participant observation.
- We have organized a conference on replication both to better understand the phenomenon of replication (and its relation to error correction) and to better understand the phenomenon of scientific conferencing (and its relation to error correction).
1. What circumstances encourage scientists to make sloppy, exaggerated, hyped, erroneous, fraudulent, or nonsensical claims?
- We have gone beyond the state of the art in understanding the contribution of resource providers (particularly in the pharmaceutical, oil, and microelectronics industries) in encouraging hype and fraud in nanobiology, biotechnology, artificial intelligence, and other fields.
2. What techniques do fraudsters and hypesters use to put forward and to propagate erroneous claims?
- We have gone beyond the state of the art in analyzing the grammar of scientific promises, and are making headway in developing digital tools for identifying promises in large corpora.
- We are making headway in developing philosophical methods for identifying claims and tracing the uptake of new concepts and vocabulary, and have made progress in translating those methods into digital tools that can be applied to large corpora.
- We have developed several cutting-edge digital tools for discovering and cataloguing new forms of scientific misconduct, especially misconduct involving generation of nonsensical papers or skewing of citation statistics.
- We have assembled detailed timelines of specific cases of exaggeration and distortion, in particular in the creation of scientific articles crafted to support a pharmaceutical company’s opioid marketing; and also in the coordination of articles, patents, contracts, investment pitches, and other media to support commercialization of spherical nucleic acids.
3. How, when, and why do scientists attempt to correct the scientific record?
- We have conducted a survey of practicing scientists soliciting their answers to these questions.
- We have conducted ethnographic interviews with scientists who were in a position to offer corrections to the record, some of whom have and some haven’t.
- We have conducted participant observation with scientists (some working within our own project) who have attempted/are attempting to offer corrections.
- We have compiled and begun to analyze corpora of “retractions” (and related terms) in the scientific press to understand the circumstances under which retractions were demanded and/or corrections were offered.
- We have identified articles that contributed to belief in a controversial claim in nanobiology, have offered annotations of those articles in post-publication peer review, and have begun an attempted replication of those papers in an attempt to compare our results to the originals.
4. What happens when scientists attempt to correct the record?
- We have conducted survey, ethnographic, digital, and historical research tracing past attempts at correction at conferences and in journals and the reactions to those attempts by editors, conference attendees/organizers, the targets of correction, and bystanders.
- We have attempted our own corrections of errors and fraud via our home institutions, journals, social media, the popular press, national research integrity offices, post-publication peer review, and other channels. We have documented those attempts and also studied them via auto-ethnography and participant observation.
- We have organized a conference on replication both to better understand the phenomenon of replication (and its relation to error correction) and to better understand the phenomenon of scientific conferencing (and its relation to error correction).