Evaluating a Decision Support System
A comprehensive Decision Support System (DSS) was developed during the PETUS project in order to promote the principles of sustainable development in urban development projects. In an attempt to gauge the usefulness, and hence value, of the DSS, the PETUS consortium undertook an extensive evaluation campaign. Feedback from an initial application of the DSS in Bulgaria highlighted the need for a multifaceted approach to the evaluation procedure. Complex yet abstract issues, such as awareness, communication and commitment, have to be assessed for a wide range of stakeholders in the urban environment. The three critical criteria upon which the DSS should be judged: accessibility, effectiveness and usability, were defined within the context of the medium in which the DSS operates - a web-based interactive environment. Exposure to the DSS came in the form of either a presentation by a PETUS partner or a visit to the PETUS website. Four different options were designed in accordance with the needs of the various types of stakeholder, including municipal authorities, politicians, NGOs, academics, technocrats and local businessmen. The four selections, in order of increasing complexity, are: questionnaire, personal interview, discussion group and, finally, application of the PETUS matrix. This carefully thought out approach was executed over a period of six months. Data was collected from respondents from each stakeholder category from eight different Member States spread across Europe. This was supplemented by feedback from six case studies in five of the Member States. The end result is that the PETUS consortium acquired important insight into ways to improve the DSS and to enhance the user experience. The evaluation procedure can also be adapted to other domains.
