The spread of disinformation is a serious problem that impacts social structure and threatens democracies worldwide. Citizens increasingly encounter (dis)information available online, either somewhat passively, through social media feeds, or actively, following search engines’ recommendations and/or by visiting specific websites. In both scenarios, algorithms filter and select displayed information, often according to the users’ past choices. Therefore, if users have a history of consuming even if a little misinformation, there is a real risk that algorithms might reinforce the user’s preferences by offering less divergent views, or even help create (mis)information bubbles, by systematically directing them to low-credibility content. For these reasons, serious efforts have been made to identify and remove “fake-news” websites and minimize the spread of disinformation on social media. However, we have not witnessed equivalent attempts to understand and curtail the potential role(s) of search engines in promoting low credibility information or in increasing polarization. As the recommendation algorithms are typically proprietary, it is not possible to directly evaluate them and we can only infer decisions from the search results.
Thus, the main aim of FARE_AUDIT was to address this imbalance through the development of an unbiased tool to audit search engines, particularly around situations of conflict (political or military), when stakes are high and disinformation rampant. The rationale was to create a system of bots (web crawlers) and incrementally change their features, controlling for factors known to impact search engine results. The bots, made to resemble users from different countries and speaking different languages, visited different websites (including those known to share disinformation) to mimic human online behavior. Through their websurfing, they collected cookies and other “fingerprints”, becoming “profiled”. These profiled bots were then directed to different search engines and instructed to perform the exact same search. By comparing the search engine recommendations, it should be possible to “reverse engineer” the recommendation systems and better understand how browsing history influences those results, particularly the likelihood of being directed to disinformation.
More specifically, FARE_AUDIT’s main goals were to:
1. Develop and implement an unbiased bot-based audit tool;
2. Systematically identify how browsing history influences search-engine results using this system of “web crawlers” that mimics different user profiles;
3. Create and test an online interface that that allows NGO’s, journalists, and interested users to scrutinize search-engine platforms and understand how different profiles access information differently;
4. Extend this concept to novel tracking or search methodologies.
Overall, we expected this tool to have meaningful social impact at at least three different levels: by increasing our knowledge on search-engine personalization, by raising public awareness of the role(s) of search engines on polarization and disinformation spread, and by better equipping civil society organizations with a tool to detect and monitor different ongoing narratives, in close to real-time. Moreover, by relying on web crawlers, our tool is privacy-protecting and does not require any real user data, paving the way to other unbiased audits. In fact, our tool is currently being adapted to include Large Language Models (LLM)-based chatbots (ChatGPT, Gemini, Llama), particularly when integrated with traditional search engines.
