Sustainability Assessment based on Decision Aiding

Periodic Reporting for period 2 - SAbDA (Sustainability Assessment based on Decision Aiding)

Reporting period: 2020-09-26 to 2021-09-25

• What is the problem/issue being addressed?
To guarantee a smooth transition to a sustainable future, there is an impelling need for providing businesses, policy makers, and the general public with an understanding of the impacts and costs of goods and services. Due to the multitude of impacts and costs, there is a clear need for methods that can convey the overall performance of competing alternatives (e.g. different cars, energy technologies, and also policies). Multiple Criteria Decision Analysis (MCDA) methods are excellent tools that can be used to support these decision-making processes. Many MCDA methods are available, and there was not until now a Decision Support System (DSS) capable of leading a decision analyst in the complex process of selecting the appropriate method(s) for a specific decision-making problem. This was thus the central challenge tackled by this project.

• Why is it important for society?
The development of a DSS to recommend MCDA method(s) is of fundamental importance for a variety of reasons. Firstly, the appropriate method has to be chosen for each decision-making problem to guarantee that the provided decision recommendation is meaningful for the decision makers. Secondly, it is necessary to have a DSS that can help analysts prioritizing efforts for reducing knowledge gaps in the description of the decision-making problems. Thirdly, it is important to have a tool capable of unveiling methodological mistakes in selecting the methods to avoid such wrongdoings in future studies.

• What are the overall objectives?
This project formalizes and contextualizes the current MCDA methods leading to the development of a comprehensive DSS (called the MCDA Methods Selection Software, MCDA-MSS) that selects the most relevant MCDA method(s) for solving decision-making problems. The MCDA-MSS was tested in the areas of Alternatives Assessment (AA) (e.g. materials, products, and technologies assessment) to assess its performance, intelligibility, and updatability.

• What are the conclusions of the action?
This research has confirmed that there is a tendency of mostly focusing on obtaining results from the application of the MCDA methods, rather than on justifying the process followed to select the chosen methods. This has resulted, at least with the literature analysed during this project, in a large share (just under 60%) of misuses of MCDA methods, which unlikely supported good and better decision-making. This finding suggests that decision analysts should allocate more time to learning about the structure of the Decision-Making Problem (DMP) and collect information that can be used to learn the requirements of the decision-makers to select or develop the most relevant MCDA method.

The suboptimal selection of MCDA methods implemented by the analysts so far implies that they have focused their choice of the MCDA methods on a subset of the relevant features to describe the DMP. The MCDA-MSS integrates, in its questions and answers, the 219 features that the authors of the MCDA-MSS (i.e. the MCIF fellow and his team) deem relevant to describe each DMP. Consequently, decision analysts now have software that can support them in their work to select an MCDA method (or a subset) that fits a detailed description of each DMP.

First part
A preliminary version of the decision support framework has been proposed, based on the available approaches that have been advanced to select an MCDA method or to conduct an MCDA process. The framework has been published as a taxonomy in OMEGA journal (https://doi.org/10.1016/j.omega.2020.102261).

Second part
The DSS developed with this action is called the MCDA Methods Selection Software (MCDA-MSS). It is available for free at this link: http://mcdamss.com and it is the most comprehensive software for recommending MCDA methods. It contributes to the meta-decision-making problem caused by the very large number of MCDA methods available nowadays, being the decision of which MCDA method(s) to use for a certain decision-making problem. This software provides systematic guidance for an analyst facing this challenge.

Third part
The MCDA-MSS was tested to assess its applicability and performance based on literature case studies that used MCDA methods for energy systems analysis. This retrospective analysis enabled to “look back” at what has been done and published in the scientific literature, and, at least from the perspective of the developers of the MCDA-MSS, to discuss whether the use of the MCDA methods was relevant or not. This test indicated that close to 60% of the analyzed studies misused MCDA methods. This suggests the need for creating a more streamlined dialogue between those assessing energy systems and those developing MCDA methods for providing decision recommendations.

In addition, given the novelty of the MCDA-MSS, it was tested to understand whether it is comprehensive enough and can be of support to the selection of MCDA methods for people who have to support decision-making. To tackle this research challenge, three virtual workshops were organized in spring 2021, with a total of 20 attendees. Out of a total of 219 features in the MCDA-MSS, those recommended by the attendees were 72. This test thus confirmed the added value of the software, indicating that the decision aiding provided so far from them has been suboptimal. This does not necessarily mean that they have chosen wrong MCDA methods. It rather indicates that the methods that they might have selected were not the best fit for their DMPs.

Contribution to the state of the art

First part
1. Propose a unified and comprehensive high-level representation of the MCDA process characteristics;
2. Show how decision making can be split into manageable and justifiable steps, which can reduce the risk of overwhelming the analyst, as well as the DMs/stakeholders during the MCDA process;

Second part
The DSS includes a vast database of more than 200 MCDA methods. This database has two unique advantages when compared to the ones in the available DSSs that try to accomplish a similar task:
1. It covers a much larger set of MCDA methods;
2. It covers a much broader set of features to describe the MCDA process.

Third part
1. Confirm that the MCDA-MSS can be used as the flagship tool to:
a. Provide a structured and traceable path for the identification of the MCDA methods suitable to each DMP;
b. Perform retrospective analyses to assess the relevant (or not relevant) use of MCDA in previous case studies;
2. Show how the MCDA-MSS avoids the suboptimal selection of the MCDA methods by means of the automatic integration of selection guidelines in its reasoner;
3. Verify that the software can also be used as a tool to train analysts approaching the MCDA domain.

Potential impacts
The overarching framework embedded in the structure of the main outcome of the fellowship (i.e. the MCDA-MSS) has clear potentials for changing the way decision support is performed within organizations of different types. An example is the use of features included in the MCDA-MSS to support the work of a project of the United Nations Environment Programme (UNEP), namely the Global Life Cycle Impact Assessment (GLAM) Method. Some of the MCDA-MSS features have been used to develop a software, called WEighting Methods Selection Software (WEMSS), to select weighting methods for Life Cycle Assessment studies.

Conceptual approach of the research

Working procedure of the MCDA-MSS to support an anlyst

Periodic Reporting for period 2 - SAbDA (Sustainability Assessment based on Decision Aiding)

Share this page

Download