Periodic Reporting for period 1 - ExplainYourself (Explainable and Robust Automatic Fact Checking)
Reporting period: 2023-09-01 to 2026-02-28
ExplainYourself proposes to study explainable automatic fact checking, the task of automatically predicting the veracity of textual claims using machine learning (ML) methods, while also producing explanations about how the model arrived at the prediction.
Automatic fact checking methods often use opaque deep neural network models, whose inner workings cannot easily be explained. Especially for complex tasks such as automatic fact checking, this hinders greater adoption, as it is unclear to users when the models’ predictions can be trusted. Existing explainable ML methods partly overcome this by reducing the task of explanation generation to highlighting the right rationale. While a good first step, this does not fully explain how a ML model arrived at a prediction. For knowledge intensive natural language understanding (NLU) tasks such as fact checking, a ML model needs to learn complex relationships between the claim, multiple evidence documents, and common sense knowledge in addition to retrieving the right evidence. There is currently no explainability method that succeeds in illuminating this highly complex process. In addition, existing approaches are unable to produce diverse explanations, geared towards users with different information needs.
ExplainYourself radically departs from existing work in proposing methods for explainable fact check- ing that more accurately reflect how fact checking models make decisions, and are useful to diverse groups of end users. It is expected that these innovations will apply to explanation generation for other knowledge-intensive NLU tasks, such as question answering or entity linking.
To achieve this, ExplainYourself builds on my pioneering work on explainable fact checking as well as my interdisciplinary expertise.
Automatic fact checking methods often use opaque deep neural network models, whose inner workings cannot easily be explained. Especially for complex tasks such as automatic fact checking, this hinders greater adoption, as it is unclear to users when the models’ predictions can be trusted. Existing explainable ML methods partly overcome this by reducing the task of explanation generation to highlighting the right rationale. While a good first step, this does not fully explain how a ML model arrived at a prediction. For knowledge intensive natural language understanding (NLU) tasks such as fact checking, a ML model needs to learn complex relationships between the claim, multiple evidence documents, and common sense knowledge in addition to retrieving the right evidence. There is currently no explainability method that succeeds in illuminating this highly complex process. In addition, existing approaches are unable to produce diverse explanations, geared towards users with different information needs.
ExplainYourself radically departs from existing work in proposing methods for explainable fact check- ing that more accurately reflect how fact checking models make decisions, and are useful to diverse groups of end users. It is expected that these innovations will apply to explanation generation for other knowledge-intensive NLU tasks, such as question answering or entity linking.
To achieve this, ExplainYourself builds on my pioneering work on explainable fact checking as well as my interdisciplinary expertise.
We have developed novel methods to learn complex relationships between the claim and multiple evidence documents. We have further developed methods for measuring the faithfulness of the resulting explanations, i.e. to what degree they align with the models' internal reasoning processes.
A core challenge identified is that language models struggle when there are knowledge conflicts between evidence documents and the models' internal knowledge, such as in the case of temporally changing facts or disputable facts. We have collected new datasets to benchmark such knowledge conflicts.
Lastly, we have studied the user needs of professional fact checkers for explainable automatic fact checking. Our findings show that they work with a variety of existing AI tools, whose main gap is that they do not provide explanations of their internal reasoning processes, including how certain they are about the different parts of the reasoning process and why.
A core challenge identified is that language models struggle when there are knowledge conflicts between evidence documents and the models' internal knowledge, such as in the case of temporally changing facts or disputable facts. We have collected new datasets to benchmark such knowledge conflicts.
Lastly, we have studied the user needs of professional fact checkers for explainable automatic fact checking. Our findings show that they work with a variety of existing AI tools, whose main gap is that they do not provide explanations of their internal reasoning processes, including how certain they are about the different parts of the reasoning process and why.
Future work on developing methods for ensuring that Large Language Models use the right kind of knowledge (intrinsic vs. extrinsic) for different scenarios is needed.
Moreover, more research on understanding the user needs for a variety of different users beyond professional fact checkers is needed. There is also a need for faithful methods which can provide explanations of their certainty related to different parts of the reasoning process.
Moreover, more research on understanding the user needs for a variety of different users beyond professional fact checkers is needed. There is also a need for faithful methods which can provide explanations of their certainty related to different parts of the reasoning process.