Safety Mechanisms for Artificial General Intelligence (AGI)

Description du projet

Des mécanismes de sécurité pionniers pour l’intelligence artificielle générale

Bien que l’intelligence artificielle générale (IAG) ait le potentiel de transformer les industries futures, elle pourrait causer des dommages involontaires, être exploitée délibérément ou se comporter de manière imprévisible si des mesures de protection appropriées ne sont pas mises en place. Le projet AGI-Safety, financé par le CER, vise à développer des mécanismes de sécurité IAG novateurs basés sur les principes de la cybersécurité. Grâce à une approche par couches, deux niveaux de protection supplémentaires seront ajoutés aux mécanismes de sécurité proactifs actuels en matière d’IA, qui resteront la première ligne de défense. La deuxième couche comprend des mécanismes de sécurité actifs, tels que des sécurités contre les défaillances, qui reconnaissent et corrigent en temps réel les pensées négatives de l’IAG afin de garantir la poursuite de l’opération en toute sécurité. La dernière couche comprend des mécanismes de sécurité réactifs, tels que des commandes d’arrêt, pour inhiber ou mettre fin à l’IAG en dernier recours.

Objectif

Artificial General Intelligence (AGI) represents AI systems with human-level cognitive abilities, capable of understanding, learning, and applying knowledge across a wide range of tasks and domains. While AGI holds immense potential to revolutionize industries, its imminent arrival also poses significant threats to society. Without proper safety mechanisms, AGI could cause unintended harm, be misused by malicious actors, or act autonomously in unpredictable and dangerous ways.

Our ambitious goal is to pioneer AGI safety by introducing a new paradigm grounded in cybersecurity principles. Current safety mechanisms—such as safeguards and alignment training—are proactive, serve only as the first line of defense, and are insufficient for the complex, autonomous nature of AGI. Stronger, more explicit mechanisms are essential to handle AGI use cases and mitigate their inherent risks.

The new paradigm employs a layered approach: beyond proactive safety, we propose adding two additional protective layers. These layers form the novel domains of active and reactive safety, both built upon a foundation of adversarial robustness. Active safety mechanisms, such as fail safes, enable us to detect and correct harmful thoughts made by the AGI in real time and explicitly, ensuring continuous and safe operation while enabling us to perform auditing when necessary. Reactive safety mechanisms, such as kill switches, serve as a last line of defense to contain or neutralize an AGI when all other measures fail. We also propose research into making these safety mechanisms immutable, preventing adversarial bypass.

Our preliminary data shows that these mechanisms are feasible and have high potential to outperform existing AI safety approaches. By fundamentally rethinking AI safety for the AGI era, this research aims to ensure we have robust safety mechanisms in place before AGI becomes a reality, while also enhancing the security and reliability of current AI systems in the interim.