Objective 1: Society-Aware ML framework
We have proposed a paradigm shift to machine learning (ML) in the context of algorithmic decision making, where the utilities of the relevant stakeholders, namely, the decision maker, the decision subjects and/or the regulators have been defined and measured from data in both stationary (WP1.1) and dynamical scenarios (WP1.2,) which have led to three scientific articles. More in detail, in our two articles focusing on the stationary settings (WP.1.1) we have explicitly distinguished between individuals’ covariates, actions/decisions, and outcome variables to model the utilities of decision makers (e.g. profit), decision subjects (benefits) and regulators (fairness notions). For dynamical setting (WP1.2) we have relied on Markov decision processes (MDP) where the states of the MDP represent the distribution of the decision subjects’ covariates, which evolve over time as a result of the outcomes of the decisions made.
Objective 2: Consensual ML algorithms across relevant stakeholders.
Our achievements towards ML algorithms that are agreed upon by all stakeholders are threefold:
1. Measuring the effect of actions (e.g. decisions) by means of practical causal inference in both stationary (WP.2.1) and dynamical scenarios (WP.4.1). More in detail, we have first focused on measuring the causal effects of decisions through the different causal paths in stationary data generating processes by extending recent state-of-the-art (SOTA) on deep learning for causal inference: i) to enable path-specific counterfactual analysis, and ii) to account for hidden confounders, which hinder to accurately assess the effect of actions. We have also considered dynamical data generating processes, where both the distributions of both the individuals’ covariates and of the outcomes of actions change over time as we change the decision-making policy.
2. Novel Society-aware ML approaches that jointly account and optimize for all stakeholders’ utilities. In the stationary setting, we have relied on practical ML methods for causal inference to propose in-processing approaches for algorithmic decision-making that, by mitigating unfairness in the available data, result in an improvement of all stakeholders’ utilities (WP.2.1). For dynamical settings, we have proposed an optimization problem over and MDP to find a decision-making policy that guarantees convergence to a pre-defined targeted fair state of the system while maximizing the owner’s utility (WP.3).
3. A multi-objective framework for evaluation of ML algorithms (WP.2) that allows comparing, aggregating and, ultimately, trading-off stakeholders’ utilities, even if they are measured in different units or scales. With this framework, we have addressed one of the main challenges of Society-aware ML, namely, enabling an informed discussion among all stakeholders about the performance of algorithms in terms of all stakeholders’ utilities to ultimately agree upon on an algorithm to deploy.
Objective 3: Society-aware application of ML
The third and last objective of the SAML project is to apply the proposed SAML methodologies to find better solutions to the growing ethical concerns resulting from the deployment of ML in society. In this context, we have extensively worked on the fairness of algorithmic decision making as a focus for the first period of the project (WP.5.1). In this context, our main achievements are twofold. First, we have advanced the SOTA on long-term fairness by studying and learning ML algorithms that ensure long term impact of group fairness in dynamically systems. Second, and more importantly, we have proposed a paradigm shift to ensure fairness in non-binary treatment decisions as well as in applications of large language models.
