Periodic Reporting for period 1 - PEER (The hyPEr ExpeRt collaborative AI assistant)
Période du rapport: 2023-10-01 au 2024-09-30
PEER project addresses the barriers to accepting AI in sequential decision-making problems, such as route planning, by systematically putting the user at the center of the entire AI design, development, deployment, and evaluation pipeline. To do this, PEER takes an interdisciplinary approach that considers both social sciences and artificial intelligence to facilitate a two-way communication flow to enhance user trust. PEER also develops an AI acceptance index for the evaluation of AI systems from a human-centric perspective and conducts integration and evaluation of these novel approaches in 4 real-world use cases.
PEER seeks to address the challenges of AI acceptance by systematically designing, realizing, and evaluating human-centric AI for sequential decision-making settings. The main objectives include: 1) To realize AI via design prototypes and to make the capabilities and limitations of AI known to stakeholders and users before the development process is carried out; 2) To improve feedback loops and enable a two-way communication flow between AI and human users to ensure better engagement in collaborative human-AI teamwork; 3) To measure progress towards truly mixed and trustworthy AI by defining qualitative and quantitative, transparent, reliable measurement scales and metrics for interactivity, acceptance, explainability and trustworthiness as well as perceived trust and fairness; 4) To integrate and evaluate the system taking into account a human-centric perspective, relying on the proposed indexes, next to more traditional performance indicators.
Besides, the PEER project takes a holistic approach involving technical sciences (AI), social sciences, and humanities (SSH) which will lead to novel ways of end-user engagement with AI and AI design. The project will focus on the end-user, and by design will consider intersectional factors (gender, ethnicity, age, socioeconomic status, disability) that could directly impact acceptance and trust in line with the EU's top agenda to achieve trustworthy AI. Furthermore, the efforts undertaken in the PEER initiative, addressing both social and technical dimensions, will serve as a reference for creating novel human-in-the-loop AI frameworks in Europe.
PEER seeks to address the challenges of AI acceptance by systematically designing, realizing, and evaluating human-centric AI for sequential decision-making settings. The main objectives include: 1) To realize AI via design prototypes and to make the capabilities and limitations of AI known to stakeholders and users before the development process is carried out; 2) To improve feedback loops and enable a two-way communication flow between AI and human users to ensure better engagement in collaborative human-AI teamwork; 3) To measure progress towards truly mixed and trustworthy AI by defining qualitative and quantitative, transparent, reliable measurement scales and metrics for interactivity, acceptance, explainability and trustworthiness as well as perceived trust and fairness; 4) To integrate and evaluate the system taking into account a human-centric perspective, relying on the proposed indexes, next to more traditional performance indicators.
Besides, the PEER project takes a holistic approach involving technical sciences (AI), social sciences, and humanities (SSH) which will lead to novel ways of end-user engagement with AI and AI design. The project will focus on the end-user, and by design will consider intersectional factors (gender, ethnicity, age, socioeconomic status, disability) that could directly impact acceptance and trust in line with the EU's top agenda to achieve trustworthy AI. Furthermore, the efforts undertaken in the PEER initiative, addressing both social and technical dimensions, will serve as a reference for creating novel human-in-the-loop AI frameworks in Europe.
During the first year of the project, we made significant progress in both foundational and technical areas, setting the stage for successful future phases. We established robust management structures and laid the groundwork for advanced AI systems, focusing on operational excellence and technical innovation.
In the first year, we focused on developing a methodology to gather and analyze sociotechnical requirements from end users. These needs were translated into technical requirements to ensure our AI systems align with user values and practical needs, fostering acceptance and trust.
Our analysis revealed that AI systems should act as assistive tools for decision-making, route planning, and navigation, while allowing users to retain control. This balance is crucial for building trust and ensuring transparency.
We created “tech cards” to help AI developers understand user expectations and prioritize interactions. These cards were used in workshops to refine our understanding of how AI can best serve users.
We identified preliminary usage scenarios and validation plans for each PEER use case, laying the groundwork for technical integration and real-world testing. This ensures our AI systems are practical, effective, and aligned with user expectations.
Additionally, we identified the first Minimum Viable Product (MVP) functionalities for the PEER AI assistant, providing a prototype for further development and testing.
In the first year, we focused on developing a methodology to gather and analyze sociotechnical requirements from end users. These needs were translated into technical requirements to ensure our AI systems align with user values and practical needs, fostering acceptance and trust.
Our analysis revealed that AI systems should act as assistive tools for decision-making, route planning, and navigation, while allowing users to retain control. This balance is crucial for building trust and ensuring transparency.
We created “tech cards” to help AI developers understand user expectations and prioritize interactions. These cards were used in workshops to refine our understanding of how AI can best serve users.
We identified preliminary usage scenarios and validation plans for each PEER use case, laying the groundwork for technical integration and real-world testing. This ensures our AI systems are practical, effective, and aligned with user expectations.
Additionally, we identified the first Minimum Viable Product (MVP) functionalities for the PEER AI assistant, providing a prototype for further development and testing.
Several innovative developments have pushed our work beyond the current state of the art, particularly in the fields of Human-AI interaction and user-centric AI design.
Innovative representation of Human-AI interactions: A key advancement has been the creation of technical cards that represent different forms of human-AI interactions. These cards, which were used in co-creation workshops to facilitate discussions, are a novel method for exploring and understanding collaborative needs between humans and AI. This tool not only aids in refining AI system requirements but also offers a new methodology for co-designing AI systems with stakeholders. The tech cards go beyond current practices by promoting a deeper integration of user input during the design phase of AI algorithms, which is not commonly seen in many AI development processes.
User-centric AI design and control: Based on the insights of the co-creation workshops the specifications of the first versions of the MVPs have been defined. The MVPs are currently being developed primarily based on state-of-the-art approaches and will serve as baselines for future iterations. In parallel, novel approaches are being developed, allowing users to understand the inherent complexity of AI algorithms for sequential decision making. We have developed an explainable Monte Carlo Tree Search as one of the search algorithms for sequential planning tasks that incorporates user-defined requirements and logic verification for factual and contrastive queries. Our results indicated significant improvement in user satisfaction compared to other baselines. In the context of AI-driven policies for professional in-store pickers in large retailers such as our industrial partner, we have investigated how Reinforcement Learning (RL) can be applied to learning efficient and customer-friendly picking strategies. The policies learned using the proposed AI approach reduced the number of customer encounters by up to 50%, compared to policies solely focused on picking orders. Since most of the problems we tackle in PEER are inherently multi-objective, we studied AI agents operating in multi-objective problems while optimizing for base utility functions. For instance, we studied how to extend meta-heuristic algorithms such ant colonies. The approach allows to vary the utility and provide routes building upon base routes for the different objectives.
Reliable and comprehensive trustworthy measurement scales: We have implemented the first version of transparent and reliable measurement scales (the AI acceptance index) to evaluate different AI transparency dimensions, setting a new benchmark for how AI systems should be assessed in terms of user trust and AI transparency. The AI acceptance index is currently also being adapted to sequential decision setting. First focusing on how the users experience the support of choosing between alternatives, however keeping in mind that later on we will also evaluate the acceptance during the actual execution of the sequential plan.
Innovative representation of Human-AI interactions: A key advancement has been the creation of technical cards that represent different forms of human-AI interactions. These cards, which were used in co-creation workshops to facilitate discussions, are a novel method for exploring and understanding collaborative needs between humans and AI. This tool not only aids in refining AI system requirements but also offers a new methodology for co-designing AI systems with stakeholders. The tech cards go beyond current practices by promoting a deeper integration of user input during the design phase of AI algorithms, which is not commonly seen in many AI development processes.
User-centric AI design and control: Based on the insights of the co-creation workshops the specifications of the first versions of the MVPs have been defined. The MVPs are currently being developed primarily based on state-of-the-art approaches and will serve as baselines for future iterations. In parallel, novel approaches are being developed, allowing users to understand the inherent complexity of AI algorithms for sequential decision making. We have developed an explainable Monte Carlo Tree Search as one of the search algorithms for sequential planning tasks that incorporates user-defined requirements and logic verification for factual and contrastive queries. Our results indicated significant improvement in user satisfaction compared to other baselines. In the context of AI-driven policies for professional in-store pickers in large retailers such as our industrial partner, we have investigated how Reinforcement Learning (RL) can be applied to learning efficient and customer-friendly picking strategies. The policies learned using the proposed AI approach reduced the number of customer encounters by up to 50%, compared to policies solely focused on picking orders. Since most of the problems we tackle in PEER are inherently multi-objective, we studied AI agents operating in multi-objective problems while optimizing for base utility functions. For instance, we studied how to extend meta-heuristic algorithms such ant colonies. The approach allows to vary the utility and provide routes building upon base routes for the different objectives.
Reliable and comprehensive trustworthy measurement scales: We have implemented the first version of transparent and reliable measurement scales (the AI acceptance index) to evaluate different AI transparency dimensions, setting a new benchmark for how AI systems should be assessed in terms of user trust and AI transparency. The AI acceptance index is currently also being adapted to sequential decision setting. First focusing on how the users experience the support of choosing between alternatives, however keeping in mind that later on we will also evaluate the acceptance during the actual execution of the sequential plan.