Periodic Reporting for period 1 - SAML (Society-Aware Machine Learning: The paradigm shift demanded by society to trust machine learning.)
Okres sprawozdawczy: 2023-02-01 do 2025-07-31
The ultimate goal of the SAML project is to find algorithmic solutions that are not only driven by economic interests of technology owners, but that all stakeholders agree on and, ultimately, that society trusts. To achieve this goal, I define the following three key project objectives:
Objective 1: Provide a society-aware ML framework that explicitly accounts for the goals of all relevant stakeholders.
Objective 2: Achieve consensual ML algorithms that make all stakeholders better off.
Objective 3: Find solutions to the growing ethical concerns, such as unfairness and societal polarization, resulting from the deployment of ML in society.
The proposed SAML approach requires changes to every stage of the ML development and will
thus involve the following methodological innovations:
■ Society-aware modeling frameworks (Objective 1) that allows the individual goals of the various
stakeholders to be explicitly defined and quantified, under different practical assumptions on the
feedback loop between the algorithm and the data collection process (WP 1).
■ Society-aware learning approaches (Objective 2) for the collaborative design of ML algorithms
among different stakeholders (WPs 2–4). I am contributing novel multi-party learning approaches
to handle the multi-objective nature (i.e. one objective per stakeholder) of the SAML algorithms.
The goal is to enable negotiation among stakeholders towards a consensual algorithmic solution by
exploring different compromises among their individual objectives.
■ Society-aware application of ML in society (Objective 3) to find algorithmic solutions to existing
ethical concerns, which make all stakeholders better off. Specifically, I work to evaluate the developed
SAML algorithms in two societal applications where ML is raising ethical concerns—i.e. fairness
in algorithmic decision-making processes (WP 5); and societal polarization resulting from
algorithmic recommendations in online social media platforms (WP 6).
Objective 1: Provide a society-aware ML framework that explicitly accounts for the goals of all relevant stakeholders.
Objective 2: Achieve consensual ML algorithms that make all stakeholders better off.
Objective 3: Find solutions to the growing ethical concerns, such as unfairness and societal polarization, resulting from the deployment of ML in society.
The proposed SAML approach requires changes to every stage of the ML development and will
thus involve the following methodological innovations:
■ Society-aware modeling frameworks (Objective 1) that allows the individual goals of the various
stakeholders to be explicitly defined and quantified, under different practical assumptions on the
feedback loop between the algorithm and the data collection process (WP 1).
■ Society-aware learning approaches (Objective 2) for the collaborative design of ML algorithms
among different stakeholders (WPs 2–4). I am contributing novel multi-party learning approaches
to handle the multi-objective nature (i.e. one objective per stakeholder) of the SAML algorithms.
The goal is to enable negotiation among stakeholders towards a consensual algorithmic solution by
exploring different compromises among their individual objectives.
■ Society-aware application of ML in society (Objective 3) to find algorithmic solutions to existing
ethical concerns, which make all stakeholders better off. Specifically, I work to evaluate the developed
SAML algorithms in two societal applications where ML is raising ethical concerns—i.e. fairness
in algorithmic decision-making processes (WP 5); and societal polarization resulting from
algorithmic recommendations in online social media platforms (WP 6).
Objective 1: Society-Aware ML framework
We have proposed a paradigm shift to machine learning (ML) in the context of algorithmic decision making, where the utilities of the relevant stakeholders, namely, the decision maker, the decision subjects and/or the regulators have been defined and measured from data in both stationary (WP1.1) and dynamical scenarios (WP1.2,) which have led to three scientific articles. More in detail, in our two articles focusing on the stationary settings (WP.1.1) we have explicitly distinguished between individuals’ covariates, actions/decisions, and outcome variables to model the utilities of decision makers (e.g. profit), decision subjects (benefits) and regulators (fairness notions). For dynamical setting (WP1.2) we have relied on Markov decision processes (MDP) where the states of the MDP represent the distribution of the decision subjects’ covariates, which evolve over time as a result of the outcomes of the decisions made.
Objective 2: Consensual ML algorithms across relevant stakeholders.
Our achievements towards ML algorithms that are agreed upon by all stakeholders are threefold:
1. Measuring the effect of actions (e.g. decisions) by means of practical causal inference in both stationary (WP.2.1) and dynamical scenarios (WP.4.1). More in detail, we have first focused on measuring the causal effects of decisions through the different causal paths in stationary data generating processes by extending recent state-of-the-art (SOTA) on deep learning for causal inference: i) to enable path-specific counterfactual analysis, and ii) to account for hidden confounders, which hinder to accurately assess the effect of actions. We have also considered dynamical data generating processes, where both the distributions of both the individuals’ covariates and of the outcomes of actions change over time as we change the decision-making policy.
2. Novel Society-aware ML approaches that jointly account and optimize for all stakeholders’ utilities. In the stationary setting, we have relied on practical ML methods for causal inference to propose in-processing approaches for algorithmic decision-making that, by mitigating unfairness in the available data, result in an improvement of all stakeholders’ utilities (WP.2.1). For dynamical settings, we have proposed an optimization problem over and MDP to find a decision-making policy that guarantees convergence to a pre-defined targeted fair state of the system while maximizing the owner’s utility (WP.3).
3. A multi-objective framework for evaluation of ML algorithms (WP.2) that allows comparing, aggregating and, ultimately, trading-off stakeholders’ utilities, even if they are measured in different units or scales. With this framework, we have addressed one of the main challenges of Society-aware ML, namely, enabling an informed discussion among all stakeholders about the performance of algorithms in terms of all stakeholders’ utilities to ultimately agree upon on an algorithm to deploy.
Objective 3: Society-aware application of ML
The third and last objective of the SAML project is to apply the proposed SAML methodologies to find better solutions to the growing ethical concerns resulting from the deployment of ML in society. In this context, we have extensively worked on the fairness of algorithmic decision making as a focus for the first period of the project (WP.5.1). In this context, our main achievements are twofold. First, we have advanced the SOTA on long-term fairness by studying and learning ML algorithms that ensure long term impact of group fairness in dynamically systems. Second, and more importantly, we have proposed a paradigm shift to ensure fairness in non-binary treatment decisions as well as in applications of large language models.
We have proposed a paradigm shift to machine learning (ML) in the context of algorithmic decision making, where the utilities of the relevant stakeholders, namely, the decision maker, the decision subjects and/or the regulators have been defined and measured from data in both stationary (WP1.1) and dynamical scenarios (WP1.2,) which have led to three scientific articles. More in detail, in our two articles focusing on the stationary settings (WP.1.1) we have explicitly distinguished between individuals’ covariates, actions/decisions, and outcome variables to model the utilities of decision makers (e.g. profit), decision subjects (benefits) and regulators (fairness notions). For dynamical setting (WP1.2) we have relied on Markov decision processes (MDP) where the states of the MDP represent the distribution of the decision subjects’ covariates, which evolve over time as a result of the outcomes of the decisions made.
Objective 2: Consensual ML algorithms across relevant stakeholders.
Our achievements towards ML algorithms that are agreed upon by all stakeholders are threefold:
1. Measuring the effect of actions (e.g. decisions) by means of practical causal inference in both stationary (WP.2.1) and dynamical scenarios (WP.4.1). More in detail, we have first focused on measuring the causal effects of decisions through the different causal paths in stationary data generating processes by extending recent state-of-the-art (SOTA) on deep learning for causal inference: i) to enable path-specific counterfactual analysis, and ii) to account for hidden confounders, which hinder to accurately assess the effect of actions. We have also considered dynamical data generating processes, where both the distributions of both the individuals’ covariates and of the outcomes of actions change over time as we change the decision-making policy.
2. Novel Society-aware ML approaches that jointly account and optimize for all stakeholders’ utilities. In the stationary setting, we have relied on practical ML methods for causal inference to propose in-processing approaches for algorithmic decision-making that, by mitigating unfairness in the available data, result in an improvement of all stakeholders’ utilities (WP.2.1). For dynamical settings, we have proposed an optimization problem over and MDP to find a decision-making policy that guarantees convergence to a pre-defined targeted fair state of the system while maximizing the owner’s utility (WP.3).
3. A multi-objective framework for evaluation of ML algorithms (WP.2) that allows comparing, aggregating and, ultimately, trading-off stakeholders’ utilities, even if they are measured in different units or scales. With this framework, we have addressed one of the main challenges of Society-aware ML, namely, enabling an informed discussion among all stakeholders about the performance of algorithms in terms of all stakeholders’ utilities to ultimately agree upon on an algorithm to deploy.
Objective 3: Society-aware application of ML
The third and last objective of the SAML project is to apply the proposed SAML methodologies to find better solutions to the growing ethical concerns resulting from the deployment of ML in society. In this context, we have extensively worked on the fairness of algorithmic decision making as a focus for the first period of the project (WP.5.1). In this context, our main achievements are twofold. First, we have advanced the SOTA on long-term fairness by studying and learning ML algorithms that ensure long term impact of group fairness in dynamically systems. Second, and more importantly, we have proposed a paradigm shift to ensure fairness in non-binary treatment decisions as well as in applications of large language models.
In the following, we list the methodology advancements that we consider breakthroughs in the field as they go significantly beyond the state-of-the-art.
- A flexible and efficient deep generative model for causal inference under hidden confounders, which we refer to as DeCaFlow and have been recently accepted for a spotlight (top3% of the articles) at NeurIPS'25.. We theoretically demonstrate that DeCaFlow accurately estimates all identifiable causal queries (interventional and counterfactual).
- First practical approach for causal inference over time discrete-time stochastic processes (DSPs) from observational data. The proposed framework achieves strong performance in terms of observational forecasting while, for the first time allows practitioners the accurate estimation of the causal effect of interventions on dynamical systems from data. The main results of this line of work have been published at AAAI'24.
- Paradigm shift that incorporates fairness considerations in non-binary treatments conditional on a positive binary decision (e.g. granting a loan). complement existing literature on fair algorithmic decision-making and mark a step toward a more holistic analysis of fairness. Consequently, our work opens avenues for future research where established fairness notions for binary decisions can be adapted to redefine and mitigate (via pre-, in-, and post-processing techniques) treatment (un)fairness. The main scientific results of this line of work have led to several scientific publications, including EWAF'24, AIES'25 and EMNLP'25
- A flexible and efficient deep generative model for causal inference under hidden confounders, which we refer to as DeCaFlow and have been recently accepted for a spotlight (top3% of the articles) at NeurIPS'25.. We theoretically demonstrate that DeCaFlow accurately estimates all identifiable causal queries (interventional and counterfactual).
- First practical approach for causal inference over time discrete-time stochastic processes (DSPs) from observational data. The proposed framework achieves strong performance in terms of observational forecasting while, for the first time allows practitioners the accurate estimation of the causal effect of interventions on dynamical systems from data. The main results of this line of work have been published at AAAI'24.
- Paradigm shift that incorporates fairness considerations in non-binary treatments conditional on a positive binary decision (e.g. granting a loan). complement existing literature on fair algorithmic decision-making and mark a step toward a more holistic analysis of fairness. Consequently, our work opens avenues for future research where established fairness notions for binary decisions can be adapted to redefine and mitigate (via pre-, in-, and post-processing techniques) treatment (un)fairness. The main scientific results of this line of work have led to several scientific publications, including EWAF'24, AIES'25 and EMNLP'25