Practical evaluation tools for urban sustainability

Deliverables

This part of the project will develop a set of tools or methodologies than can be applied both to sector specific and holistic projects based on the results and outcomes of WP 2 (Review of existing tools) and WP3 (Selection of case studies. The framework's aim is to integrate across economic, environmental and social factors and at providing knowledge about performance criteria, related to benchmark data. This will aim to fill the gap between theory and practice. Many tools have been identified, and more than 20 frameworks mainly developed with European funds. It has been found through discussions that these are not being used by local authorities or other organisations involved in urban infrastructure. PETUS will therefore help to disseminate the investment in research to European citizens at the local level. By providing methodological tangible help to promote sustainable development will have an economic relevance in itself as sustainable development includes economic concerns. The development of PETUS, via the development of the framework of tools, intends to bring positive issues of sustainable development to the local level, which has been found to be fairly ad hoc to date. These frameworks will provide methodological assistance towards more sustainable urban infrastructures particularly through increasing the understanding of what sustainability incorporates. This step towards sustainability implies better resource management. The economic outcomes will be the most substantial at both local and at regional levels. That should lead to tangible consequences for European citizen. Furthermore, the development of tools should promote use, which means some impacts on financial and human resources management.

This part of the project has resulted in a protocol for testing and validation of evaluation tools. Identification of appropriate case studies within each of the partners countries and across the range of sectors. The case studies used were a mixture of previously used and new case studies to ensure that some recently developed studies were included as well as some more familiar projects. Application of the tools with the local authority staff. Assessment of the success of the tools in evaluating the case studies. Identification and reporting of where limitations arise. Modification of tools/framework developed to ensure compatibility with current working practices and integration into working situations.

Exploitable results

This work package builds on the Review of Existing Evaluation Tools; the Review of Case Studies from a range of European projects from different sectors and scales and at different life cycle stages; and the investigation of Decision Making Procedures, tools and benchmarks used in practice. A framework or decision support system (DSS) containing practical evaluation tools for urban sustainability has been developed to enable a consistent approach to sustainability across a range of urban infrastructure projects including water/sewage, waste, transport, energy, green/blue areas and buildings & land use. The framework consists of two parts (a database and a guidance system), which are intricately linked: The database includes information that can be used to analyse and improve the sustainability of urban infrastructure, whatever the size or type. Data is available for different levels of expertise and therefore provides a facility to enable discussion between decision makers and stakeholders from all different levels to obtain and understand information that is relevant and useful for them. The database contains case study projects from across Europe that illustrated where sustainability has been considered; tools (both theoretical tools and those used in practice) that can be used to guide and analyse consideration of sustainability in a practical way; EU legislation that has to be followed in member countries; and a glossary. The guidance proposes a monitoring process that enables every user to track the inclusion of sustainability within a users own project. The guidance is the interactive part of the DSS that proposes, via a matrix and a checklist, to integrate sustainability concerns in the overall decision-making process of an urban infrastructure project. PETUS has undergone a testing process, which has fine tuned the framework, and it is now a valuable tool available for use by anyone but of particular relevance to those within the urban infrastructure industry.

The PETUS Project Conference was held on 15th & 16th September 2005 in Cardiff, Wales and UK. The conference "Building a sustainable future: Tools and decision making for sustainable urban development" presented research findings and practical experience of incorporating sustainability into urban infrastructure from experts in the energy, waste, water/sewage, transport, green/blue and building and land use sectors together with those cross disciplinary experts, as well as presenting the PETUS Decision Support System website (http://www.petus.eu.com). The Conference Proceedings are a record of the conference publicity, conference material and presentations made at the conference. (Papers on PETUS are being published in the ISBE Journal in 2006 http://www.isbe.demon.co.uk/).

Decision making is a complex task and requires comprehensive approach, from and interaction between, the stakeholder groups involved. Existing decision making structures are, and have always been, questioned whether they need to be changed and adapted to the changing situation at place, including the economy, the environment, new forms of co-operation between stakeholders groups, but also awareness of these terms. PETUS identified that different stakeholders had a variety of requirements for decision making framework for urban infrastructure projects: - Politicians: simple tools and procedures used to give confidence to the people; long term planning component and impact assessment included; ability to view and handle conflict points; support for dialogue and communication; links to relevant stakeholder groups. - Technicians/administration: cases and tools for reference; support for their work; support for communication between different stakeholder groups. - Citizens/general public: find the right person to talk to; communicate from a common starting point; gain expertise and information; know their position and options in the decision making process; have new support for lobbying. PETUS followed a comprehensive or ¿holistic¿ view on the subject of decision making, to help to: (i) achieve interaction between stakeholders and create understanding for each others situations; (ii) to make prepositions for new arrangements in negotiation and communication structures; (iii) to gather information at large prepared for the involved stakeholders. The development of a decision making framework (dmf) designed to support decision makers in their work and to provide the means for such a ¿holistic¿ approach, became a central task. Work focused on the development and collection of sectors for a decision making framework and the design of a supportive tool for practice use. Decision support tools, that include a matrix and checklist, have been developed as a result, from the findings from the case analysis, from workshops and interviews with end users. The work in this work package compliments the work undertaken in the development of the framework for practical evaluation tools for urban sustainability. The different information sources of PETUS build the backbone in a double perspective: (i) case studies and tools serving as an information source and (ii) provide support on how to use and fill in the matrix. Several of the PETUS case studies show that at least parts of their chosen planning and decision making steps comply with standard Strategic Environmental Assessment (SEA) procedural steps. By following a similar procedure to SEA, PETUS provides an introduction to SEA at a basic level. End users utilising PETUS follow a procedure that can be converted into an SEA as the steps involved are similar. PETUS is therefore considered to be supportive to achieve the various steps in the SEA procedure and not to replace it. The final result of this work package has been the production of a (i) matrix; developed to assist the process of increasing sustainability in the decision making process of a project over time. The matrix includes questions relating to the different stages of a project/plan or programmes development and also the different issues that should be considered at the different stages. The matrix is applicable for different stakeholders who are interested or involved in project development. (ii) checklist; set up to provide a quick and easy to use list of questions that can be considered when attempting to incorporate sustainability into urban infrastructure projects. The sections of the checklist, like the matrix, follow a similar format to the Strategic Environmental Assessment (SEA) to prevent duplication of work if an SEA is required for a plan, programme or policy.

The goals for the work package were to: -Explore the consistency of tools used and look for good examples of tools, -Explore how tools have related to decision-making, -Identify knowledge-gaps and barriers for tools amongst decision-makers, -Identify success and practical applicability of tools including transferability, -Identify differences between tools available and tools used in practice. 60 case studies were identified and investigated. The cases include projects (limited in scale and time) and policies (continuous efforts for sustainable development in the whole sector), completed to ongoing, and operating on different scales (from building components to national strategy). PETUS partners to be good examples of tools in use have identified 10 of these case studies. Information was collected from the case study projects through interviews and site visits using a PETUS developed procedure. An overview of the main findings from the case studies (and PETUS meetings and discussions with end users) identified and considered in this work package are: -In most cases, there are multiple tools in use at the same time. -There are a mix of driving forces and motivations for using tools, as well as barriers to prevent a tools implementation. -Tools are adjusted to the specific case (flexibility in relation to context). Tools used internationally have been designed to take local conditions into account. -Decision-makers know little about existing tools and frameworks, but are at the same time seeking suitable tools. This indicates a large potential for developing better information and communication on available tools for end-users. -There are some problems related to defining sustainability in practice, which might confuse communication between initiators and stakeholders. Therefore, practical definitions or understandings on sustainability are needed. -There are many good examples of tools that have provided a more sustainable urban infrastructure. These examples can be used to demonstrate to end-users, and others, how tools can support sustainable infrastructure development in practice. -Implementation of evaluation tools promotes public debate on urban sustainability by raising general awareness. The results from the Review of Existing Evaluation Tools in Practice and Case Studies have fed into the development of the framework of practical evaluation tools for urban sustainability.

This work package sought to test PETUS Decision Support System (DSS) with public and private sector organisations in order to prove its effectiveness. The testing process developed into two main stages: a preliminary stage (development of a testing concept and strategy, before the official start of WP5) and a core testing stage with two sub-stages: a methodological (development of testing criteria, procedures and instruments); and an operational one (provision of structured feedback from end users through interviews and testing case-studies). A pilot testing with a restricted focus on only one sector of urban infrastructure (Green/Blue) was carried out in two Bulgarian towns. The main issues traced were: general public awareness on and local authorities commitment to sustainable development, attitude towards urban public green spaces; evaluation tools currently applied and communication patterns between authorities and citizens; local business involvement into public greenery maintenance. The pilot testing results provided both support for further discussion on PETUS DSS focus and suggestions for the development of the testing process itself. The testing models and instruments were developed in accordance with the initial testing strategy based on the three effectiveness criteria chosen (credibility scientific/technical soundness; saliency ability to address particular practical problems; legitimacy acceptability by end users.). As, being reflexive to end users demands and preferences, PETUS took the form of a web-based interactive DSS, the initially discussed effectiveness testing criteria had to be complemented by a particular focus on its accessibility and usability. Accessibility regarded smooth transformation of information and regarded technical components and content management of the website (to provide information in a clear and simple manner and by an understandable mechanism to navigate within and between pages). Usability meant the degree of practical use and implementation of the tool measured by the satisfaction with the results of the tool implementation. According to the peculiarities of the target groups, identified in the testing concept, three testing models were developed. Testing model 1 (TM1) comprised an individual filling-in of a Respondent Questionnaire (RQ) after a visit to PETUS website or a presentation by PETUS team member. The model was considered appropriate for graduate and PhD students, politicians, local businessmen and NGOs. Testing Model 2 (TM2) supposed face-to-face communication of an interviewer with an individual or a group of respondents. TM2a comprised an individual interview (open but structured discussion after a presentation of PETUS or a personal website visit). It was considered appropriate for contacting academic staff, technical experts and NGOs. An Interviewers Questionnaire (IQ) was to be filled in by the interviewer himself. TM2b was based on a Focus Group Discussion and was to identify the level of readiness for participation in PETUS further development and to provide innovative suggestions about the further development of the DSS content and functions. This model was considered appropriate for academic staff, technical experts and NGO members. Testing model 3 (TM3) was developed for testing the effectiveness of the PETUS DSS with local (technical) experts on real case studies. It was supposed to provide feedback on the PETUS DSS effectiveness through an experimental matrix application to ongoing or recently accomplished urban plans/ programmes. It comprised four steps: - review of the testing case study; - filling in the PETUS matrix; - structured interview with the end users; - analysis of testing results. The testing of PETUS (March November 2005) comprised a number of interviews taken in three periods (March May, June August, September November). TM1and TM2a inquiries were carried out with 84 respondents from all partner countries. Respondents from all the target groups of the project were reached: municipal experts and politicians, academic staff, NGO members. Feedback was provided on the estimation of PETUS usability, effectiveness and accessibility. Guidance needed to facilitate the DSS use was asked about. Answers to open questions provided comments about the contents and structure of PETUS and about technical aspects of the website-based DSS. The PETUS matrix was also tested with end users in six real case studies from on-going urban development practice in 5 European countries - UK, France, Denmark, Austria and Bulgaria. This was the basis for identifying possible improvements, existing implementation barriers and future research challenges.

The PETUS Handbook has been published to provide an overview of the concept of PETUS, including the concept and background to PETUS, in a concise and attractive format. The handbook is composed of: - An introduction to PETUS and the concept of PETUS; - Information and background to the case studies and a brief analysis of the tools contained within PETUS; - Decision making; - EU legislation; - Glossary; - List of PETUS partners and end users. The Handbook also contains Version 1 of PETUS on disc, to allow users to access the database off line.

This work package reviewed the literature for evaluation tools referring to the theoretical background behind the creation of tools to evaluate urban sustainability. The review identified evaluation tools developed from both a theoretical base together with those that are used in practice. A procedure was developed to enable a coherent and consistent review of evaluation tools and benchmark data between partners. The search for tools attempted to identify all types of sustainable development tool. This included those investigating environmental, social and economic factors individually or combined, and also sector specific and holistic tools from a practical and theoretical background. A detailed data extraction sheet was designed to be use as a guide to collect relevant information comprising 24 questions. 126 tools were fully reviewed: 11 generic tools, these are theoretical procedures originally developed to investigate sustainability and have been used to create more practical tools over time; 82 holistic tools, these either investigate the sustainability impacts of buildings (12) or the impact of wider infrastructure developments and planning (70). (This category was changed to Buildings and Land use later in the project); 32 sector specific tools that are used to investigate one sector only and collect information at a more technical level. 9 of these relate to the energy sector, 2 investigate waste, 8 consider the water/sewage sector, 2 look at transport and 11 relate to the green/blue sector. The review of tools has identified that there is a limited set of benchmarks available and those that are available are not often realistically achievable or are difficult to calculate. Benchmarks that do exist vary greatly between sectors under investigation and between countries. The review also identified that in many cases actors are confused and wary of set benchmarks as they are often set with a lack of detailed baseline data to define targets. By using benchmarks set by different organisations or by government, comparisons can be made with other urban infrastructure projects particularly those in the same sector. Benchmarking requires a base knowledge for continual changes, whether positive or negative to be measured. Consideration of the procedure for using the tools and the involvement of public participation has been made. The overall question relating to end users is why the tools available are not being used when significant interest in PETUS has been expressed by many end users and other organisations. End users have agreed that the gap identified between theory and practice exists, and difficulties encountered when trying to assess effectively the sustainability of an infrastructure, to investigate different influences, interactions, and to conclude about their (cumulative) impact. A positive feature identified by many end users is the use of tools to encourage joined up thinking between actors in the urban infrastructure industry and the public. Public participation is seen as a major interest, tools can assist with this by providing technical support to obtain a better argument or to incorporate and assemble an interactive public participation process. The results from the Review of Existing Evaluation Tools have fed into the development of the PETUS framework of practical evaluation tools for urban sustainability.

Two websites have been developed during the PETUS project: The PETUS project website (http://www.petusproject.com/) and the website containing the end results of the PETUS project: The PETUS Decision Support System website (http://www.petus.eu.com/index.html). The PETUS project website is comprised of: - a project summary; - project timetable; - details the ongoing development stages (work packages)and research; - results of the project; - publications; - details of the project team; - end user involvement;

Searching for OpenAIRE data...

Deliverables

Exploitable results

Share this page

Download