Periodic Reporting for period 2 - XAI (Science and technology for the explanation of AI decision making)
Okres sprawozdawczy: 2021-04-01 do 2021-10-31
Black box AI systems for automated decision making, often based on ML over (big) data, map a user’s features into a class or a score without exposing the reasons why. This is problematic both for the lack of transparency and also for possible biases inherited by the algorithms from prejudices and collection artifacts hidden in the training data, which may lead to unfair or wrong decisions. The future of AI lies in enabling people to collaborate with machines, requiring good communication, trust, clarity, and understanding.
The XAI project, focuses on the challenge of how to construct meaningful explanations of opaque AI/ML systems, aiming at empowering individuals against undesired effects of automated decision making, implementing the “right of explanation”, and helping people to make better decisions preserving (and expand) human autonomy.
Trust is crucial in the adoption of AI/ML technologies, due to perceived challenges to human autonomy, and the lack of knowledge about the assumptions, limitations, and capabilities of AI assistants. Building trust in AI models relies on their capacity to reveal their logic in a comprehensible, human-accessible format, allowing them to understand and validate their decision rationale and highlighting possible biases learned by the model.
Understandable explanations impact not only information ethics but they are a cornerstone for accountability, safety, and industrial liability. Therefore, explainability is fundamental for boosting a vivid market of AI-based services in particular in safety-critical industries.
Likewise, the use of AI systems in scientific research requires explanations not only for trust and acceptance but also for the openness and progress of scientific discovery.
The XAI project faces the challenge of requiring AI to be explainable and understandable in human terms and articulates its research along 5 Research Activities (RA): RA1) algorithms to infer local explanations and their generalization to global ones (post-hoc) and algorithms that are transparent by-design; RA2) languages for expressing explanations in terms of logic rules, with statistical and causal interpretation; RA3) XAI watchdog platform for sharing experimental dataset and explanation algorithms; RA4) a repertoire of case studies aimed at in involving also final users; RA5) a framework to study the interplay between XAI and ethical and legal dimensions.
The XAI project, focuses on the challenge of how to construct meaningful explanations of opaque AI/ML systems, aiming at empowering individuals against undesired effects of automated decision making, implementing the “right of explanation”, and helping people to make better decisions preserving (and expand) human autonomy.
Trust is crucial in the adoption of AI/ML technologies, due to perceived challenges to human autonomy, and the lack of knowledge about the assumptions, limitations, and capabilities of AI assistants. Building trust in AI models relies on their capacity to reveal their logic in a comprehensible, human-accessible format, allowing them to understand and validate their decision rationale and highlighting possible biases learned by the model.
Understandable explanations impact not only information ethics but they are a cornerstone for accountability, safety, and industrial liability. Therefore, explainability is fundamental for boosting a vivid market of AI-based services in particular in safety-critical industries.
Likewise, the use of AI systems in scientific research requires explanations not only for trust and acceptance but also for the openness and progress of scientific discovery.
The XAI project faces the challenge of requiring AI to be explainable and understandable in human terms and articulates its research along 5 Research Activities (RA): RA1) algorithms to infer local explanations and their generalization to global ones (post-hoc) and algorithms that are transparent by-design; RA2) languages for expressing explanations in terms of logic rules, with statistical and causal interpretation; RA3) XAI watchdog platform for sharing experimental dataset and explanation algorithms; RA4) a repertoire of case studies aimed at in involving also final users; RA5) a framework to study the interplay between XAI and ethical and legal dimensions.
The work is organized into the five interleaved research activities. RA1 focused on the design and implementation of explanation algorithm for Machine Learning methods: post-hoc and by-design. Six novel algorithms have been realized: LORE produces local explanations for tabular data in the form of logic-based factual and counterfactual rules. Exploiting a latent space projection, LORE applies also to other forms of data: images with ABELE, time series with LASTS, and text with LORE-T. Exploiting a health ontology, MARLENA builds on top of LORE to realize the DrXAI an explanator for an existing clinical predictor. GLOCALX, is a Local2Global approach (L2G) that combines the local logic-based explanations provided by LORE to describe the global behavior of the model. Logic expressions and predicates in RA2 habilitate reasoning and causality. An example of this is TriplEx, a local explanation method for text classification, that returns explanation in the form of factual triples representing relevant words.
The XAI library (github.com/kdd-lab/XAI-Lib) is the basic element of the Watchdog platform of RA3, making accessible the project’s algorithms and others from the literature. This provides a benchmarking workspace for XAI algorithms and their explanations through a set of quantitative evaluation measures.
In RA4, three use cases in healthcare have been developed and qualitative validation involving end-users has been pursued with the approval of the CNR ethical board: more than 200 health professionals have been involved to test the impact of explanations in their decision-making process, using the judge-advisor system (JAS; Sniezek & Buckley, 1995) to evaluate participants’ trust [18 with the Honorable Mention Award].
RA5 realized and published two new algorithms to combine fairness, privacy, and explanation: Fairlens[19] and EXPERT [4].
To summarize in this period, the XAI team has realized 8 novel algorithms and published 36 international papers, with the contribution of 11 PhD students. XAI project has fostered collaboration with other H2020 projects (AI4EU, Humane-Ai-Net, Tailor, and SoBigData++). XAI team has organized 8 workshops, 8 tutorials at international conferences and 4 PhD schools, and 20 invited speeches by the PI, co-PI, and senior staff. We also launched a series of “XAI distinguished seminar (DS)” xai-project.eu/dist-seminars.html discussing important scientific topics of XAI with international researchers, the attendance was for each seminar over 130 people.
The XAI library (github.com/kdd-lab/XAI-Lib) is the basic element of the Watchdog platform of RA3, making accessible the project’s algorithms and others from the literature. This provides a benchmarking workspace for XAI algorithms and their explanations through a set of quantitative evaluation measures.
In RA4, three use cases in healthcare have been developed and qualitative validation involving end-users has been pursued with the approval of the CNR ethical board: more than 200 health professionals have been involved to test the impact of explanations in their decision-making process, using the judge-advisor system (JAS; Sniezek & Buckley, 1995) to evaluate participants’ trust [18 with the Honorable Mention Award].
RA5 realized and published two new algorithms to combine fairness, privacy, and explanation: Fairlens[19] and EXPERT [4].
To summarize in this period, the XAI team has realized 8 novel algorithms and published 36 international papers, with the contribution of 11 PhD students. XAI project has fostered collaboration with other H2020 projects (AI4EU, Humane-Ai-Net, Tailor, and SoBigData++). XAI team has organized 8 workshops, 8 tutorials at international conferences and 4 PhD schools, and 20 invited speeches by the PI, co-PI, and senior staff. We also launched a series of “XAI distinguished seminar (DS)” xai-project.eu/dist-seminars.html discussing important scientific topics of XAI with international researchers, the attendance was for each seminar over 130 people.
In this period we concentrated on the algorithms for producing explanations out of trained black box and the core of these is LORE aimed at enabling why, why-not, and what-if reasoning. The three strengths, which make LORE a great advance over the state of the art are: i) the genetic algorithm for neighborhood generation, ii) the construction of factual and counterfactual logical rules, and iii) the combination with latent space. These elements are like lego pieces that allow adaptability to different forms of data and provide the basis for stability, robustness, and actionability. LORE outperforms state of art methods, both those focusing on feature relevance (Lime and Shap) and the more recent ones on the factual and counterfactual scene. Fundamental has been the XAI library, a workbench offering a variety of metrics to evaluate explanators and explanations achieving the goal to make order in an impressively growing field. LORE is the core of almost all other algorithms (and relative papers) developed so far and it has been validated in various domains (health, finance) and new demonstrations on real cases are ongoing.
L2G paradigm introduced with GLOCALX is a distinctive contribution, unique in its proposal. It leverages the efficiency of local explanations to build a global transparent surrogate model that enables logic and causal reasoning. We plan to extend the L2G paradigm with parametric explanations and new types of data.
XAI has now all the ingredients, albeit still evolving, to go to “step two” of the project, towards “explanation as synergistic human-machine collaboration” with two directions of work: i) (human-in-the-loop) an adaptive interface layer to habilitate a conversational interaction combined with new ML methods capable to defer decisions and putting the user in full control: I know “when you succeed”, “when you fail”, “when to trust you”, “why you erred”. ii) incorporating the cognitive elements of the user decisional process into the above interactions, validated through a series of experiments involving end-users.
RA1 will also investigate how to design ML methods that are transparent by design in the following directions: i) by exploiting the compression on latent space; ii) using knowledge graphs and iii) by following a mechanistic approach in the direction of the Hopfield model.
RA5 plans to define an assessment framework enabling the systematic evaluation of the privacy and unfairness risk of the different types of explainers.
L2G paradigm introduced with GLOCALX is a distinctive contribution, unique in its proposal. It leverages the efficiency of local explanations to build a global transparent surrogate model that enables logic and causal reasoning. We plan to extend the L2G paradigm with parametric explanations and new types of data.
XAI has now all the ingredients, albeit still evolving, to go to “step two” of the project, towards “explanation as synergistic human-machine collaboration” with two directions of work: i) (human-in-the-loop) an adaptive interface layer to habilitate a conversational interaction combined with new ML methods capable to defer decisions and putting the user in full control: I know “when you succeed”, “when you fail”, “when to trust you”, “why you erred”. ii) incorporating the cognitive elements of the user decisional process into the above interactions, validated through a series of experiments involving end-users.
RA1 will also investigate how to design ML methods that are transparent by design in the following directions: i) by exploiting the compression on latent space; ii) using knowledge graphs and iii) by following a mechanistic approach in the direction of the Hopfield model.
RA5 plans to define an assessment framework enabling the systematic evaluation of the privacy and unfairness risk of the different types of explainers.