Periodic Reporting for period 1 - PRODEMINFO (Protecting the Democratic Information Space in Europe)
Periodo di rendicontazione: 2021-10-01 al 2023-03-31
PRODEMINFO (https://sks.to/prodeminfo) aims to gain a greater understanding of people’s perception of truth in an age rife with “fake news”, disinformation, and conspiracy theories.
The main objectives of PRODEMINFO are:
(1) to understand contemporary misinformation in Europe and how it connects to people's tacit personal and subjective ontology of truth;
(2) to develop countermessages based on inoculation theory that are sensitive to people's different ontologies (e.g. addressing authenticity rather than accuracy of political speech); and
(3) to test such countermessages at scale on social media.
PRODEMINFO pursues these objectives by combining controlled behavioural and cognitive experimentation with "big data'' analyses of social media, simulation modelling of behaviour online, and text modelling of large corpora.
To achieve these three objectives, the project is organised into three main work packages:
WP1 explores the question “what does truth actually mean and to whom” using a combination of computational text modeling, social-media analysis, and behavioral experimentation to explore the changing and diverse ontology of truth.
WP2 explores the question “how do we reach people who do not place much weight on factual accuracy", combining corpus-curation and behavioural experimentation to find the reflection of the ontology of truth in misinformation and to create ontologically-aligned countermessages.
WP3 explores the question “can we detect misinformation before it goes viral and inhibit its spread online" by combining computational social media analysis, simulation modeling, and field-experimentation on social media to roll out countermeasures to misinformation online.
The full version of our 'Summary for publication' is available from the PRODEMINFO website: https://tinyurl.com/3nnhh92a.
The main objectives of PRODEMINFO are:
(1) to understand contemporary misinformation in Europe and how it connects to people's tacit personal and subjective ontology of truth;
(2) to develop countermessages based on inoculation theory that are sensitive to people's different ontologies (e.g. addressing authenticity rather than accuracy of political speech); and
(3) to test such countermessages at scale on social media.
PRODEMINFO pursues these objectives by combining controlled behavioural and cognitive experimentation with "big data'' analyses of social media, simulation modelling of behaviour online, and text modelling of large corpora.
To achieve these three objectives, the project is organised into three main work packages:
WP1 explores the question “what does truth actually mean and to whom” using a combination of computational text modeling, social-media analysis, and behavioral experimentation to explore the changing and diverse ontology of truth.
WP2 explores the question “how do we reach people who do not place much weight on factual accuracy", combining corpus-curation and behavioural experimentation to find the reflection of the ontology of truth in misinformation and to create ontologically-aligned countermessages.
WP3 explores the question “can we detect misinformation before it goes viral and inhibit its spread online" by combining computational social media analysis, simulation modeling, and field-experimentation on social media to roll out countermeasures to misinformation online.
The full version of our 'Summary for publication' is available from the PRODEMINFO website: https://tinyurl.com/3nnhh92a.
Six activities were scheduled to be undertaken or commence during the first reporting period of the project [M 1 to 18]:
Analysis 1.1. [originally M 1 to 12; revised to M 1 to 24]
Progress on this activity is ongoing. In retrospect, given the inevitable lag time in setting up a team and hiring personnel, the projected 12 months completion time was too short. To date, this analysis has yielded one publication in PNAS Nexus (Lasser et al., 2022), one invited revision to Nature Human Behaviour (Lasser et al., 2023), and at least two additional manuscripts that are in preparation and expected to be submitted by the end of 2023.
Analysis 1.2. [originally M 10 to 16; revised to M 12 to M 24]
This analysis tests the outcomes of Analysis 1.1 by leveraging major historical events, whose link to putative shifts in conceptions of honesty and truth are well established, to probe shifts in ontology. Specifically, this analysis was to focus on the rise and fall of fascism in Spain and Germany to examine linguistic changes in the conception of truth during those tumultuous times.
Analysis 1.3. [originally M 9 to M 24; revised to M 14 to M 30]
This analysis relates the linguistic markers of truth and evidence from Analysis 1.1 and Analysis 1.2 to the longitudinal and cross-sectional data from three leading democracy indices (e.g. V-Dem), and whether these linguistic markers can provide an early warning system when democracy is retreating.
Study 1.4. [originally M 4 to 12; revised to M 4 to M 20]
The study aimed to recruit representative participants in five target countries from online panel providers, in order to probe attitudes towards democracy, allowing extraction of a subset of items that can form an efficient assay of personal ontologies of truth. The first arm of the study has been completed using a UK sample, which has been sufficient to select items for further studies as originally planned. The items have also already been translated into German, Italian, and Hungarian, and we are poised to collect data in those other countries in the very near future.
Study 1.5. [originally M 10 to 18; revised to M 10 to M 24]
This study is similar to Study 1.4 except that participants are Twitter users and analysis of their historical Twitter discourse will be linked to their survey responses to examine whether individual ontologies of honesty or truth are detectable in spontaneous social media activity.
Analysis 2.1. A European misinformation corpus [M 2 to 26]
This analysis is dedicated to curating a European misinformation corpus through a search of fact-checking archives in the five target countries to select items – from both trustworthy and untrustworthy sources – for potential use as experimental stimuli. Together with collaborator Alessandro Miani, we have collated an Italian-language corpus containing >600,000 articles from >50 websites from 2004 to present. The corpus has been presented at the European Chapter of the Association for Computational Linguistics (EACL) in 2023, and it is a valuable resource for both quantitative and qualitative misinformation studies.
Analysis 1.1. [originally M 1 to 12; revised to M 1 to 24]
Progress on this activity is ongoing. In retrospect, given the inevitable lag time in setting up a team and hiring personnel, the projected 12 months completion time was too short. To date, this analysis has yielded one publication in PNAS Nexus (Lasser et al., 2022), one invited revision to Nature Human Behaviour (Lasser et al., 2023), and at least two additional manuscripts that are in preparation and expected to be submitted by the end of 2023.
Analysis 1.2. [originally M 10 to 16; revised to M 12 to M 24]
This analysis tests the outcomes of Analysis 1.1 by leveraging major historical events, whose link to putative shifts in conceptions of honesty and truth are well established, to probe shifts in ontology. Specifically, this analysis was to focus on the rise and fall of fascism in Spain and Germany to examine linguistic changes in the conception of truth during those tumultuous times.
Analysis 1.3. [originally M 9 to M 24; revised to M 14 to M 30]
This analysis relates the linguistic markers of truth and evidence from Analysis 1.1 and Analysis 1.2 to the longitudinal and cross-sectional data from three leading democracy indices (e.g. V-Dem), and whether these linguistic markers can provide an early warning system when democracy is retreating.
Study 1.4. [originally M 4 to 12; revised to M 4 to M 20]
The study aimed to recruit representative participants in five target countries from online panel providers, in order to probe attitudes towards democracy, allowing extraction of a subset of items that can form an efficient assay of personal ontologies of truth. The first arm of the study has been completed using a UK sample, which has been sufficient to select items for further studies as originally planned. The items have also already been translated into German, Italian, and Hungarian, and we are poised to collect data in those other countries in the very near future.
Study 1.5. [originally M 10 to 18; revised to M 10 to M 24]
This study is similar to Study 1.4 except that participants are Twitter users and analysis of their historical Twitter discourse will be linked to their survey responses to examine whether individual ontologies of honesty or truth are detectable in spontaneous social media activity.
Analysis 2.1. A European misinformation corpus [M 2 to 26]
This analysis is dedicated to curating a European misinformation corpus through a search of fact-checking archives in the five target countries to select items – from both trustworthy and untrustworthy sources – for potential use as experimental stimuli. Together with collaborator Alessandro Miani, we have collated an Italian-language corpus containing >600,000 articles from >50 websites from 2004 to present. The corpus has been presented at the European Chapter of the Association for Computational Linguistics (EACL) in 2023, and it is a valuable resource for both quantitative and qualitative misinformation studies.
PRODEMINFO is situated in one of the fastest-moving research contexts in the social sciences. Advances in AI, such as the recent release of chatGPT, and changes in policy – not only by governments but also private corporations – can drastically alter the landscape literally overnight. The PRODEMINFO team is at the leading edge of those developments and has a finger on the policy pulse at the European level, mainly through the PI’s extensive contacts within and collaborations with the European Commission in Brussels. This helps ensure that the outcomes of the project will feed into policy wherever possible.
At the same time, the intensely dynamic nature of the research landscape can throw up unexpected problems as well: the takeover of Twitter by Elon Musk in late 2022 has led to the revocation of academic access to the API (in April 2023), which will require considerable redesign of the remaining planned work packages. At the time of this writing, it is entirely unclear what, if anything, will replace the academic API. For the moment this does not have any impact on the project because we have access to large Twitter corpora that were downloaded in anticipation of the API closing down. However, in the long run, the project may consider other avenues to conduct the Twitter studies planned for WP3.
PRODEMINFO is progressing well overall and in accordance with the original plans (albeit with some modified timelines). The project has given rise to some unexpected positive synergies which will amplify its reach and output.
PRODEMINFO has already yielded several extensions of the inoculation approach along the lines foreshadowed in the proposal. For example, we recently analyzed more than 160,000 news articles (in English) and found a negative relationship between the extent of polarizing language (quantified by computational text analysis) and the trustworthiness of news sources (indicated by the NewsGuard score).
At the same time, the intensely dynamic nature of the research landscape can throw up unexpected problems as well: the takeover of Twitter by Elon Musk in late 2022 has led to the revocation of academic access to the API (in April 2023), which will require considerable redesign of the remaining planned work packages. At the time of this writing, it is entirely unclear what, if anything, will replace the academic API. For the moment this does not have any impact on the project because we have access to large Twitter corpora that were downloaded in anticipation of the API closing down. However, in the long run, the project may consider other avenues to conduct the Twitter studies planned for WP3.
PRODEMINFO is progressing well overall and in accordance with the original plans (albeit with some modified timelines). The project has given rise to some unexpected positive synergies which will amplify its reach and output.
PRODEMINFO has already yielded several extensions of the inoculation approach along the lines foreshadowed in the proposal. For example, we recently analyzed more than 160,000 news articles (in English) and found a negative relationship between the extent of polarizing language (quantified by computational text analysis) and the trustworthiness of news sources (indicated by the NewsGuard score).