Final Report Summary - SRDTOOLS (Methods and Tools for Evaluating the Contribution of Cohesion Policies to Sustainable Regional Development (SRD))
The SRDTOOLS project focused on the objectives of regional policy and its programme design and implementation; and the scope to improve the evaluation of the sustainability of regional development (SRD). Past experience has shown that regions have tended to give more recognition to the environmental pillar of sustainable development, when developing sustainable development policies and related programmes. Furthermore, previous work undertaken by DG regional policy also indicated that many of the concepts of sustainable development had not been properly understood by the broader population of policy makers and evaluators.
Such concerns led to two main considerations, which the SRDTOOLS project aimed to address:
- the need for better integration of the three pillars of sustainable development;
- the need for more consistency in evaluation methods and for better application of tools and concepts to cohesion policies.
The research was intended to examine and test methods and tools for supporting and informing public policy decision-making when seeking to determine between competing policy choices, in the context of uncertainty and complexity; i.e. where traditional policy evaluation tools (such as CBA) break down. To this end, the SRDTOOLS project was concerned with: defining the policy 'evaluation space' for sustainable regional development (SRD), and the validity, utility and application of evaluation tools designed to support SRD decision-making. The major outcome of the project was the presentation of a 'nested' approach to evaluation, differentiating the policy choices and attendant requirement for normative analysis.
Sustainable development is most conceived in policy and programme design and evaluation as the need for integration of economic, social and environmental interests in order to maximise 'quality of life', without compromising the interest of future generations. These 'three pillars' - economic, social and environmental - are similar to the four capitals; however, the four capitals (4C) model distinguishes in the social pillar between human (individual) capital and social capital, concerned with the institutional arrangements that effect social welfare. The 'added value' of the 4C model can mainly be seen through the scope to formally integrate different spheres or dimensions within a single framework capable of defining non-sustainable development, by formalising the concept of trade-offs and minimum levels of 'capital' within an overall aim of ensuring there is no decline in the level of capital.
The three pillars of sustainable development may be re-interpreted in terms of four capitals:
- Natural - corresponds to the environmental pillar;
- Manufactured - corresponds to part of the economic pillar;
- Social - corresponds to part of the social pillar;
- Human - divides between the economic and social pillar.
The SRDTOOLS project also identified the problems associated with using particular tools and evaluation methods. The evaluation method used depends on several factors: whether the situation is ex-ante / ex-post; degree of agreement over objectives; measurement of impacts; weight given to different stakeholders and their judgments; and the measurements and commensurability of impact measures. In a situation which is predominantly social and with a strong degree of contestation for example, cost-benefit analysis (CBA) would not be suitable, and would more likely exacerbate the problem than provide a solution. In this instance, its use would be limited due to problems in obtaining commensurate data (necessary for CBA) and also because the decision context in this case gives weight to non-economic consideration. Although the CBA approach remains influential and at a project level would be more suitable (greater scope to monetize various impacts), the difficulties encountered with its use in this project suggest that in some cases, it would be better to use CBA together with a complementary evaluation method.
The case studies undertaken in the SRDTOOLS project illustrated that the identification of policy-relevant trade-offs and by implication, the nature of regional development paths, follows logically from the description of regional trends. These trends, measured using indicators on the basis of criteria derived from regional objectives but classified according to the four capitals, enable a structure for regional evaluation. Overall, it was possible to identify trade-offs from consultation with the stakeholders and / or using trend analysis.
Indicators of regional trends were used in the case studies to inform discussion of ideas of 'acceptable' levels of a particular capital or service, and to indicate where regional development was sustainable - i.e. in the 'right' or 'wrong' direction. For example, the Polish case study analysed a situation where the expansion of a local agricultural sector was causing environmental damage through eutrophication. A closer study of critical trends revealed that the situation was not 'win-lose', as initially assumed, but 'lose-lose' - environmental damage was affecting not only natural capital (water pollution), but manufactured capital (decreased attractiveness to tourists) and human and possibly social capital (risk of harmful impact on human health).
The case studies confirmed that the approaches to defining trade-offs worked well in terms of clarifying the key issues, and were especially important where a solid evidence base was needed to confirm the significance of the issue. It must be noted, that although some of the more important trade-offs are obvious, while others remain implicit because of poor cause and effect understanding (particularly in terms of changes in social capital). However, the trade-off analysis appears to be an improvement on 'checklist' appraisal systems that fail to distinguish large from smaller issues. The underlying acceptance of sustainable development as defined by the 4C model is a prerequisite for this. This provides a challenge, as there remains a strong tendency to use the 'three pillars' model despite the lack of any associated, operational definition of sustainability.
The case studies showed that the decision context for key regional decisions is open - with multiple stakeholders and multiple objectives. Multi-governance influences are high, making it often difficult to reach a consensus. To permit a didactic presentation of the process and outcomes of judgments offered by each category of stakeholders, for each of the options or scenarios under evaluation, with reference to a spectrum of governance or quality-performance issues, the KerBabel Deliberation matrix (also referred to as the 'DM', or 'Cube') was developed.
It must be noted that the 'Cube' is intended to support not replace decision-making. It aims to mobilise people's capacity and judgments in support of the decision-making process.
Through the use of the DM, many of the case study partners identified interesting observations - for example, problems with accessing the software; choosing the relevant colour for each ball in the matrix; and most significantly, framing the problem of actions in terms of options, actors and issues intuitively.
One of the DM's strengths, however, was considered to be its 'accessibility' in terms of users being able to describe in relatively simple ways what the impact of an option meant to them, i.e. if the option was likely to strengthen their collective capacity, they would select a green ball; if another criterion was detrimental, they would select a red ball. The DM aims to use forms of thinking that non-technical people would use just as much as specialists. Another strength of the DM is its ability to display which stakeholders are in agreement with each other, and is designed to increase transparency of what is at stake; and facilitates the identification of a rolling agenda for discussion and analysis.
Such concerns led to two main considerations, which the SRDTOOLS project aimed to address:
- the need for better integration of the three pillars of sustainable development;
- the need for more consistency in evaluation methods and for better application of tools and concepts to cohesion policies.
The research was intended to examine and test methods and tools for supporting and informing public policy decision-making when seeking to determine between competing policy choices, in the context of uncertainty and complexity; i.e. where traditional policy evaluation tools (such as CBA) break down. To this end, the SRDTOOLS project was concerned with: defining the policy 'evaluation space' for sustainable regional development (SRD), and the validity, utility and application of evaluation tools designed to support SRD decision-making. The major outcome of the project was the presentation of a 'nested' approach to evaluation, differentiating the policy choices and attendant requirement for normative analysis.
Sustainable development is most conceived in policy and programme design and evaluation as the need for integration of economic, social and environmental interests in order to maximise 'quality of life', without compromising the interest of future generations. These 'three pillars' - economic, social and environmental - are similar to the four capitals; however, the four capitals (4C) model distinguishes in the social pillar between human (individual) capital and social capital, concerned with the institutional arrangements that effect social welfare. The 'added value' of the 4C model can mainly be seen through the scope to formally integrate different spheres or dimensions within a single framework capable of defining non-sustainable development, by formalising the concept of trade-offs and minimum levels of 'capital' within an overall aim of ensuring there is no decline in the level of capital.
The three pillars of sustainable development may be re-interpreted in terms of four capitals:
- Natural - corresponds to the environmental pillar;
- Manufactured - corresponds to part of the economic pillar;
- Social - corresponds to part of the social pillar;
- Human - divides between the economic and social pillar.
The SRDTOOLS project also identified the problems associated with using particular tools and evaluation methods. The evaluation method used depends on several factors: whether the situation is ex-ante / ex-post; degree of agreement over objectives; measurement of impacts; weight given to different stakeholders and their judgments; and the measurements and commensurability of impact measures. In a situation which is predominantly social and with a strong degree of contestation for example, cost-benefit analysis (CBA) would not be suitable, and would more likely exacerbate the problem than provide a solution. In this instance, its use would be limited due to problems in obtaining commensurate data (necessary for CBA) and also because the decision context in this case gives weight to non-economic consideration. Although the CBA approach remains influential and at a project level would be more suitable (greater scope to monetize various impacts), the difficulties encountered with its use in this project suggest that in some cases, it would be better to use CBA together with a complementary evaluation method.
The case studies undertaken in the SRDTOOLS project illustrated that the identification of policy-relevant trade-offs and by implication, the nature of regional development paths, follows logically from the description of regional trends. These trends, measured using indicators on the basis of criteria derived from regional objectives but classified according to the four capitals, enable a structure for regional evaluation. Overall, it was possible to identify trade-offs from consultation with the stakeholders and / or using trend analysis.
Indicators of regional trends were used in the case studies to inform discussion of ideas of 'acceptable' levels of a particular capital or service, and to indicate where regional development was sustainable - i.e. in the 'right' or 'wrong' direction. For example, the Polish case study analysed a situation where the expansion of a local agricultural sector was causing environmental damage through eutrophication. A closer study of critical trends revealed that the situation was not 'win-lose', as initially assumed, but 'lose-lose' - environmental damage was affecting not only natural capital (water pollution), but manufactured capital (decreased attractiveness to tourists) and human and possibly social capital (risk of harmful impact on human health).
The case studies confirmed that the approaches to defining trade-offs worked well in terms of clarifying the key issues, and were especially important where a solid evidence base was needed to confirm the significance of the issue. It must be noted, that although some of the more important trade-offs are obvious, while others remain implicit because of poor cause and effect understanding (particularly in terms of changes in social capital). However, the trade-off analysis appears to be an improvement on 'checklist' appraisal systems that fail to distinguish large from smaller issues. The underlying acceptance of sustainable development as defined by the 4C model is a prerequisite for this. This provides a challenge, as there remains a strong tendency to use the 'three pillars' model despite the lack of any associated, operational definition of sustainability.
The case studies showed that the decision context for key regional decisions is open - with multiple stakeholders and multiple objectives. Multi-governance influences are high, making it often difficult to reach a consensus. To permit a didactic presentation of the process and outcomes of judgments offered by each category of stakeholders, for each of the options or scenarios under evaluation, with reference to a spectrum of governance or quality-performance issues, the KerBabel Deliberation matrix (also referred to as the 'DM', or 'Cube') was developed.
It must be noted that the 'Cube' is intended to support not replace decision-making. It aims to mobilise people's capacity and judgments in support of the decision-making process.
Through the use of the DM, many of the case study partners identified interesting observations - for example, problems with accessing the software; choosing the relevant colour for each ball in the matrix; and most significantly, framing the problem of actions in terms of options, actors and issues intuitively.
One of the DM's strengths, however, was considered to be its 'accessibility' in terms of users being able to describe in relatively simple ways what the impact of an option meant to them, i.e. if the option was likely to strengthen their collective capacity, they would select a green ball; if another criterion was detrimental, they would select a red ball. The DM aims to use forms of thinking that non-technical people would use just as much as specialists. Another strength of the DM is its ability to display which stakeholders are in agreement with each other, and is designed to increase transparency of what is at stake; and facilitates the identification of a rolling agenda for discussion and analysis.