Periodic Reporting for period 1 - MULTIDOOR (Diagnostic Screening Platform to Facilitate Conflict Resolution)
Reporting period: 2022-08-01 to 2024-08-31
MultiDoor was planned as a digital platform based on conflict resolution and machine learning expertise to address the comprehensive needs of litigants and recommend their best way forward to resolve their disputes. At present, litigants attempting to navigate through the civil justice system end up drifting through an incoherent, opaque process generally resulting in some form of reluctant compromise. While court systems worldwide are investing much effort to increase efficiency, a human-centred approach, which takes into account litigants' needs, interests and emotions, is lacking. MultiDoor employs an innovative intake screening recommendation system to integrate each litigant's (or potential litigant's) specific needs, interests and emotions, the features of the case, and the predicted case trajectory in the legal system, resulting in a diagnostic recommendation (e.g. mediation, arbitration, adjudication, out-of-the-box solutions). We have developed a crowdsourcing experiment to accumulate data on users’ satisfaction with conflict resolution-oriented processing of their disputes; developing forecasting models for user satisfaction; and developing a machine-learning based recommendation system.
Our vision of the The MultiDoor Recommendation system was of Transitioning from a default process to a customized and informed recommendation; creating a “triage” platform that integrates each party’s specific needs, interests, and emotions, as well as the features of the case; Broadening options for litigants and providing them with coaching in conflict resolution; Creating an empowering process for the users and building a process of informed decision-making.
We successfully recruited 621 unique participants for our study. Each participant was presented with three distinct case studies and was asked to indicate, for each case study, the resolution method they deemed most suitable. On average, we were able to elicit 37 responses for each case study. A response refers to the conflict resolution method that a participant identified as the most suitable for resolving the conflict presented in the case study.
Our experiment proved less successful in terms of predicting the choice of future litigants. Our results reflect the performance of three advanced prediction algorithms. Despite the sophistication of these algorithms, the maximum accuracy achieved was 55%, a significant improvement over random guessing but potentially insufficient for a commercial product. Since the models’ accuracy is not sufficient for commercial use, it made us rethink the next steps of our endeavour. Considering the significant developments in the realm of GenAI, we plan to develop a persuasion bott based on conflict resolution foundations in order to help possible litigants to realise the advantages of mediation.
Our vision of the The MultiDoor Recommendation system was of Transitioning from a default process to a customized and informed recommendation; creating a “triage” platform that integrates each party’s specific needs, interests, and emotions, as well as the features of the case; Broadening options for litigants and providing them with coaching in conflict resolution; Creating an empowering process for the users and building a process of informed decision-making.
We successfully recruited 621 unique participants for our study. Each participant was presented with three distinct case studies and was asked to indicate, for each case study, the resolution method they deemed most suitable. On average, we were able to elicit 37 responses for each case study. A response refers to the conflict resolution method that a participant identified as the most suitable for resolving the conflict presented in the case study.
Our experiment proved less successful in terms of predicting the choice of future litigants. Our results reflect the performance of three advanced prediction algorithms. Despite the sophistication of these algorithms, the maximum accuracy achieved was 55%, a significant improvement over random guessing but potentially insufficient for a commercial product. Since the models’ accuracy is not sufficient for commercial use, it made us rethink the next steps of our endeavour. Considering the significant developments in the realm of GenAI, we plan to develop a persuasion bott based on conflict resolution foundations in order to help possible litigants to realise the advantages of mediation.
WP1 The conflict stories based on interviews with real life disputants were conducted and the questionnaire for the experiment was built. The intake framework was built and the intake instrument is used as a deliverable to educate students and professionals in various courses and training.
WP2 The scenarios (case studies) for the experiment were built and enriched. They will be released to the public after the publications which emerge from the project will be accepted.
WP3 The Experiment was built and launched. To ensure participant engagement and motivation throughout the experiment, we took several measures to optimize the design and presentation of the study. In addition to providing monetary compensation, we consulted with a professional UI/UX company to develop an intuitive and user-friendly layout for the experiment, ensuring that each stage was clear and accessible.
The experiment was structured into multiple stages. The first stage involved collecting demographic information from the participants. In the second stage, participants completed a personality trait questionnaire based on the Big Five model. The third stage included a negotiation style assessment using the framework developed by Thomas and Kilmann.
In the fourth stage, participants were introduced to various conflict resolution methods. This phase included an explanation of the benefits and drawbacks of each method to provide participants with a comprehensive understanding. To enhance clarity, we produced a series of short videos illustrating the procedures associated with each resolution method. Following the videos, participants completed a brief quiz to ensure they understood the strengths and limitations of each approach. Only participants who successfully passed the quiz proceeded to the subsequent stages of the experiment.
In the fifth stage, participants were presented with three case studies describing interpersonal conflicts. They were instructed to immerse themselves in the scenarios, imagining themselves as part of the conflict. After reading each story, participants were asked to reflect on and identify their needs and feelings in the context of the conflict. Finally, they were presented with several conflict resolution options, including mediation, arbitration, adjudication, letting go, and public complaint. For each story, participants selected the resolution method they considered most suitable.
We successfully recruited 621 unique participants for our study. Each participant was presented with three distinct case studies and was asked to indicate, for each case study, the resolution method they deemed most suitable. On average, we were able to elicit 37 responses for each case study. A response refers to the conflict resolution method that a participant identified as the most suitable for resolving the conflict presented in the case study.
WP4 We have completed the experiment phase of the project by using the Prolific platform. Our experiment proved less successful in terms of predicting the choice of future litigants. Our results reflect the performance of three advanced prediction algorithms: XGBoost, Neural Networks, and Support Vector Machines (SVM). Each of these algorithms offers unique strengths and was chosen for its potential to accurately predict legal solutions by matching case characteristics and the individuals involved to one of six possible outcomes. Despite the sophistication of these algorithms, the maximum accuracy achieved was 55%, a significant improvement over random guessing but potentially insufficient for a commercial product.
WP2 The scenarios (case studies) for the experiment were built and enriched. They will be released to the public after the publications which emerge from the project will be accepted.
WP3 The Experiment was built and launched. To ensure participant engagement and motivation throughout the experiment, we took several measures to optimize the design and presentation of the study. In addition to providing monetary compensation, we consulted with a professional UI/UX company to develop an intuitive and user-friendly layout for the experiment, ensuring that each stage was clear and accessible.
The experiment was structured into multiple stages. The first stage involved collecting demographic information from the participants. In the second stage, participants completed a personality trait questionnaire based on the Big Five model. The third stage included a negotiation style assessment using the framework developed by Thomas and Kilmann.
In the fourth stage, participants were introduced to various conflict resolution methods. This phase included an explanation of the benefits and drawbacks of each method to provide participants with a comprehensive understanding. To enhance clarity, we produced a series of short videos illustrating the procedures associated with each resolution method. Following the videos, participants completed a brief quiz to ensure they understood the strengths and limitations of each approach. Only participants who successfully passed the quiz proceeded to the subsequent stages of the experiment.
In the fifth stage, participants were presented with three case studies describing interpersonal conflicts. They were instructed to immerse themselves in the scenarios, imagining themselves as part of the conflict. After reading each story, participants were asked to reflect on and identify their needs and feelings in the context of the conflict. Finally, they were presented with several conflict resolution options, including mediation, arbitration, adjudication, letting go, and public complaint. For each story, participants selected the resolution method they considered most suitable.
We successfully recruited 621 unique participants for our study. Each participant was presented with three distinct case studies and was asked to indicate, for each case study, the resolution method they deemed most suitable. On average, we were able to elicit 37 responses for each case study. A response refers to the conflict resolution method that a participant identified as the most suitable for resolving the conflict presented in the case study.
WP4 We have completed the experiment phase of the project by using the Prolific platform. Our experiment proved less successful in terms of predicting the choice of future litigants. Our results reflect the performance of three advanced prediction algorithms: XGBoost, Neural Networks, and Support Vector Machines (SVM). Each of these algorithms offers unique strengths and was chosen for its potential to accurately predict legal solutions by matching case characteristics and the individuals involved to one of six possible outcomes. Despite the sophistication of these algorithms, the maximum accuracy achieved was 55%, a significant improvement over random guessing but potentially insufficient for a commercial product.
This project enabled unique collaboration among data scientists, mediators, lawyers, and court administration. For the first time, it created a data-driven approach to dispute resolution and shows strong potential to influence the development of online court intake mechanisms.
After decades of theoretical development in the field of conflict resolution, particularly Alternative Dispute Resolution (ADR), which relied on abstract assumptions about parties' behaviour and choices in conflict, we have successfully accumulated big data on real-life choices in various types of actual conflicts. Our findings are surprising and, while not conclusive for predicting litigants' choices, they provide valuable insights for future research and can be used to develop persuasive bots based on conflict resolution studies to de-escalate potential litigants.
After decades of theoretical development in the field of conflict resolution, particularly Alternative Dispute Resolution (ADR), which relied on abstract assumptions about parties' behaviour and choices in conflict, we have successfully accumulated big data on real-life choices in various types of actual conflicts. Our findings are surprising and, while not conclusive for predicting litigants' choices, they provide valuable insights for future research and can be used to develop persuasive bots based on conflict resolution studies to de-escalate potential litigants.