Periodic Reporting for period 2 - ADIX (Argumentation-based Deep Interactive EXplanations(ADIX))
Période du rapport: 2023-04-01 au 2024-09-30
Today’s AI landscape is permeated by data and data-centric methods (e.g. deep learning) but suffers from widespread problems: albeit powerful, data-centric AI methods are often opaque and their outputs (e.g. classifications) predominantly lack human-understandable explanations that can give confidence in the outputs’ validity and ensure trust in the AI systems.
As a result, extensive research efforts are being devoted towards a plethora of explanation methods. The potential role of explanation in AI goes well beyond informing how and why a particular output of an automated decision-making system is produced. Indeed, it can also provide means to contest and rectify the output if undesired, e.g. because resulting from a bias in the underlying data, thus empowering humans and facilitating take-up. Overall, explainability is crucial to guarantee comprehensibility to support collaboration and communication between AI-powered machines and humans and ultimately allowing AI to benefit society.
However, most existing efforts in explainable AI (XAI) focus on shallow explanations of outputs in terms of inputs alone, disregarding the inner workings of the underlying AI systems and thus limiting transparency. Moreover, existing methods focus on generating static explanations that cannot account for and benefit from human input. ADIX strives to address at the same time the challenges of deep explainability and interactability of data-driven AI, crucial for AI to pave the way towards tomorrow’s human-centred but AI-supported society, where AI benefits humans. ADIX aims at meeting these challenges in symphony, by developing a novel scientific paradigm of argumentation-based deep interactive explanations (DIX) as a catalyst for AI systems where data-centric and symbolic techniques, amenable to human consumption, work together and in synergy with humans, while keeping humans fully in control. Specifically, ADIX’ overall objectives are:
Objective 1. To define an overarching, general-purpose formal theory of argumentation-based DIX for explainable and interactable AI.
Objective 2. To deploy argumentation-based DIX within a variety of settings, representative of today’s data-centric AI landscape, in order to evaluate empirically and inform the development of the new paradigm.
Objective 3. To demonstrate applicability of the novel paradigm to real-life settings of societal importance, in particular in healthcare and law.
As a result, extensive research efforts are being devoted towards a plethora of explanation methods. The potential role of explanation in AI goes well beyond informing how and why a particular output of an automated decision-making system is produced. Indeed, it can also provide means to contest and rectify the output if undesired, e.g. because resulting from a bias in the underlying data, thus empowering humans and facilitating take-up. Overall, explainability is crucial to guarantee comprehensibility to support collaboration and communication between AI-powered machines and humans and ultimately allowing AI to benefit society.
However, most existing efforts in explainable AI (XAI) focus on shallow explanations of outputs in terms of inputs alone, disregarding the inner workings of the underlying AI systems and thus limiting transparency. Moreover, existing methods focus on generating static explanations that cannot account for and benefit from human input. ADIX strives to address at the same time the challenges of deep explainability and interactability of data-driven AI, crucial for AI to pave the way towards tomorrow’s human-centred but AI-supported society, where AI benefits humans. ADIX aims at meeting these challenges in symphony, by developing a novel scientific paradigm of argumentation-based deep interactive explanations (DIX) as a catalyst for AI systems where data-centric and symbolic techniques, amenable to human consumption, work together and in synergy with humans, while keeping humans fully in control. Specifically, ADIX’ overall objectives are:
Objective 1. To define an overarching, general-purpose formal theory of argumentation-based DIX for explainable and interactable AI.
Objective 2. To deploy argumentation-based DIX within a variety of settings, representative of today’s data-centric AI landscape, in order to evaluate empirically and inform the development of the new paradigm.
Objective 3. To demonstrate applicability of the novel paradigm to real-life settings of societal importance, in particular in healthcare and law.
Up until now, in line with the workplan, work in ADIX has focused on three main dimensions of the envisaged formal theory of argumentation-based DIX for explainable and interactable AI: i) identifying faithful argumentative abstractions of a variety of data-centric AI methods; ii) extracting diverse types of explanations from these abstractions, and evaluating them in terms of several desirable properties (notably faithfulness to the explained model); and iii) supporting interactions amongst stakeholders in XAI based on the argumentative abstractions and explanations drawn from them. We have explored these dimensions in three settings: 1) data-centric AI with labelled data (including supervised machine learning methods); 2) data-centric AI with text (including generative AI for classification); and 3) interpretable AI classification models drawn from labelled data (including tabular data and images). Some highlights of our work to date follow.
We have developed a novel method (SpArX) to draw bipolar argumentation frameworks from artificial neural networks (first setting), whereby arguments are clusters of neurons and these can attack and support other clusters. We use clusters rather than directly neurons in order to control the size of the argumentation frameworks, which can serve directly as visual explanations for the mechanics of the artificial neural networks. We have extended this method to obtain the novel interpretable ProtoArgNet method for image classification (third setting). Further, we have developed methods to perform relation-based argument-mining with text (second setting), based on generative AI.
We have grounded our work, again in line with the workplan, in healthcare and legal use cases, in collaboration with our external partners. Specifically in the healthcare space, we have conducted a user study with explanations for the QCancer tool. We have also considered and evaluated explanations for models built from a number of tabular datasets, including: a breast cancer database, data from the BrainWear clinical trial for patients with brain cancer, and the (brain cancer) GlioCova dataset. In addition, we have worked with a number of image datasets in the healthcare space, including radiographs for detecting pneumonia and for grading osteoarthritis. In the legal space, we have considered legal cases, in natural (legal) language, from contract law and tabular data on recidivism (drawn from COMPAS) and welfare benefits.
We have developed a novel method (SpArX) to draw bipolar argumentation frameworks from artificial neural networks (first setting), whereby arguments are clusters of neurons and these can attack and support other clusters. We use clusters rather than directly neurons in order to control the size of the argumentation frameworks, which can serve directly as visual explanations for the mechanics of the artificial neural networks. We have extended this method to obtain the novel interpretable ProtoArgNet method for image classification (third setting). Further, we have developed methods to perform relation-based argument-mining with text (second setting), based on generative AI.
We have grounded our work, again in line with the workplan, in healthcare and legal use cases, in collaboration with our external partners. Specifically in the healthcare space, we have conducted a user study with explanations for the QCancer tool. We have also considered and evaluated explanations for models built from a number of tabular datasets, including: a breast cancer database, data from the BrainWear clinical trial for patients with brain cancer, and the (brain cancer) GlioCova dataset. In addition, we have worked with a number of image datasets in the healthcare space, including radiographs for detecting pneumonia and for grading osteoarthritis. In the legal space, we have considered legal cases, in natural (legal) language, from contract law and tabular data on recidivism (drawn from COMPAS) and welfare benefits.
We are taking a fresh look at explainable AI, pushing the line that human understanding of AI models is crucial to their fruitful deployment while revisiting commonly held assumptions as to which types of explanations are useful to humans. Rather than shallow explanations, which explain outputs in terms of input features alone without any indication of how they are generated, we are developing deep explanations, faithful to the explained method both in terms of input/output behaviour (as for shallow explanations) and structurally. These deep explanations are drawn on a variety of argumentative abstractions and are the basis for interactions, which we have defined in terms of novel notions of argumentative exchanges between stakeholders in explanatory settings (standardly a machine and a human, however our methods could lend themselves to several machines and humans being involved in the same explanatory interactions).
While to date we have focused on the argumentative abstractions, explanations therefrom and skeleton of interactions based on these, we are currently working on the final piece of the groundwork towards our envisaged formal theory of argumentation-based DIX for explainable and interactable AI, namely methods to accommodate human feedback to improve future outputs and contest the explained (underlying) models. Feedback and contestation may be needed, for instance, because the model builds upon artifacts or biases. We envisage that these methods will rely upon wrappers of the explained models, built upon the argumentative abstractions. These wrappers may also inform data analysts as to how to modify the underlying models, in addition to informing any future predictions (and possibly explanations, if the wrapper modifies the argumentative abstraction) until the model is kept unchanged. An important reason for not modifying the models directly is that the user feedback may in turn show biases and errors, and data analysts may need to intervene on the wrappers to mitigate the effect of feedback.
We are also working on accommodating all dimensions of our explorations to date (on argumentative abstractions, explanation types, interactions and wrappers) within a cohesive theory of argumentation-based DIX, focusing on properties, including robustness and consensus-related properties. Finally, we are working, with our external partners, towards case studies in healthcare and law showcasing all components of our formal theory in practice.
While to date we have focused on the argumentative abstractions, explanations therefrom and skeleton of interactions based on these, we are currently working on the final piece of the groundwork towards our envisaged formal theory of argumentation-based DIX for explainable and interactable AI, namely methods to accommodate human feedback to improve future outputs and contest the explained (underlying) models. Feedback and contestation may be needed, for instance, because the model builds upon artifacts or biases. We envisage that these methods will rely upon wrappers of the explained models, built upon the argumentative abstractions. These wrappers may also inform data analysts as to how to modify the underlying models, in addition to informing any future predictions (and possibly explanations, if the wrapper modifies the argumentative abstraction) until the model is kept unchanged. An important reason for not modifying the models directly is that the user feedback may in turn show biases and errors, and data analysts may need to intervene on the wrappers to mitigate the effect of feedback.
We are also working on accommodating all dimensions of our explorations to date (on argumentative abstractions, explanation types, interactions and wrappers) within a cohesive theory of argumentation-based DIX, focusing on properties, including robustness and consensus-related properties. Finally, we are working, with our external partners, towards case studies in healthcare and law showcasing all components of our formal theory in practice.