Development of sensor-based Citizens' Observatory Community for improving quality of life in cities

Final Report Summary - CITI-SENSE (Development of sensor-based Citizens' Observatory Community for improving quality of life in cities)

Executive Summary:
The CITI-SENSE project developed and tested components of environmental monitoring and information systems based on innovative and novel Earth Observation capabilities. The applications focused on the citizens’ immediate environment regarding urban air quality, environmental quality of urban public spaces and indoor air quality in schools.
The CITI-SENSE approach was to develop “Citizens’ Observatories” (COs), a collaborative concept that focuses on the empowerment of citizens to influence their community policy and decision-making. Several novel technologies, especially a variety of micro sensors, mobile applications and communication solutions, enabled this approach technically.
The concept of CITI-SENSE citizen observatories rested on realizing the chain “sensors-platform-products-users” with the following elements: technologies for distributed monitoring (sensors); information and communication technologies (platform); information products and services (products); and citizen involvement in both monitoring and societal decisions (users). The main novel contributions were:
• Studies of motivations and barriers to citizen involvement with environmental decision making
• Development of tools for citizens monitoring of urban environment
• Deployment of low-cost micro sensor devices and innovative data fusion and scientific analyses
• Combining new sensing technology, ICT Cloud platform with IoT, Big Data and App/Portal services and participatory methods in products and services.
To demonstrate the CO concept and capabilities, we established and for over six months, operated the largest ever sensor network for air quality comprising of 324 units deployed across Europe. We involved nearly 400 volunteers in nine cities to test our personal monitoring devices. We established 24 individual COs (8 COs for outdoor air quality, 12 COs for indoor air quality in schools and 4 COs for personal comfort in public spaces) in the following nine cities across Europe: Barcelona (ES), Belgrade (RS), Edinburgh (UK), Haifa (IL), Ljubljana (SI), Oslo (NO), Ostrava (CZ), Vienna (A) and Vitoria-Gasteiz (ES). We involved volunteers in co-development of sensor devices, visualisation solutions and other tools used in the project. Our air perception app was downloaded and actively used by more than 1200 persons. With the help of the sensor network and additional observation tools, we collected more than 9.4 million observations in the last year of the project.
In any decisions, people need to be in focus. We developed and applied participatory methods for each individual CO. We collected citizens’ feedback on environmental issues through evaluation questionnaires and in-depth focus group discussions and interviews. We have also collected feedback on our tools and services. The results have also helped the tools’ providers to improve their products and we can provide a list of lessons learned to support similar initiatives within Citizens’ Observatories and Citizen Science. The resulting products and services are available through the Citizens’ Observatories Web Portal (http://co.citi-sense.eu).
CITI-SENSE operated within an open e-collaboration framework with projects funded under the same call. Methodologies and standards for data archiving, discovery and access within the GEOSS framework were be coherent with initiatives such as GEO, INSPIRE and GMES. CITI-SENSE also made CO information available through the GEOSS infrastructure.

Project Context and Objectives:
A key challenge for Western society in the 21st Century is to address the deterioration of the quality of the environment and attendant negative consequences on human society. Societal response to this challenge has been led by a large body of legislation aiming to regulate use of the environment. This response has to also take account of the increasing technological nature of Western society. As a result, many measures have been introduced that affect people’s daily lives, not all of them equally accepted in society, partly because the rationale behind the measures is complex (e.g. involving complicated issues regarding environmental health) and difficult to communicate to the citizens.
Moreover, the positive aspects of a healthy environment to human health have been largely overlooked.The focus is shifting from stressing the detrimental (pollution) to stressing the positive (ecosystem services).CITI-SENSE explored ways how to increase the involvement of the public in environmental decisions, both directly and through provision of citizen-collected data.

Increasingly also the concept of citizens’ observatories – of involving citizens as active partners in environmental monitoring decision-making – is seen as central to protecting and enhancing our environment, and thus its ability to provide “ecosystem services”. It states that the environment is in the hands of all citizens, not only in the hands of decision makers remote from the citizen. To truly adapt to environmental changes, we need to harness environmental information in a user-friendly way and empower citizens to actively improve their own environment by using this information. Concepts of participatory governance recognize that it is no longer sufficient to provide lists of environmental facts or reports to inform citizens of environmental changes. Obtaining and using local knowledge helps empower citizens, and gives society an indication of what is needed for citizen participation to be truly sustainable.

To establish citizens’ observatories and to make them useful to society, we need to work with citizens, their groupings and representatives, and with the representatives of the established environmental information collection systems, and identify their issues, interests and needs. We need also to work with the technological community, to identify how to meet these needs.

The main objective of CITI-SENSE was to develop technology enabled “Citizens’ Observatories” to empower citizens and citizen groups:
• to contribute to and participate in environmental governance;
• to support and influence community and policy priorities and the associated decision making; and
• to contribute to European and global monitoring initiatives.
In order to achieve its main objective, the project was aiming for:
• learning from citizens’ experiences and expectations
• raising environmental awareness in the society
• stimulating stakeholder groups’ participation in community and societal decisions and
• providing a transparent link between citizens and the decision-making process.

By Citizens’ Observatories we mean communities of diverse users that will share technological solutions, information products and services, and community participatory governance methods using appropriate communication solutions (e.g. mobile applications, social media), and who will by these activities complement established environmental data and information systems and improve local decision-making about environmental issues.

The concept of CITI-SENSE rested on realizing the chain “sensor-platform-products-users”: technologies for distributed monitoring (sensors); information and communication technologies (platform); information products and services (products); and citizen involvement in both monitoring and societal decisions through participation and empowerment (users).

At the heart of CITI-SENSE are three focused empowerment initiatives (EIs). EI1 - Urban quality has as its basis the multiple existing environmental monitoring systems in place in urbanized areas; these systems do not currently provide citizens with the means to participate in environmental governance, despite the need to improve their living environment and thus their health and well being. The EI Urban quality aims at supplementing these existing systems, developing information products and creating a sensing community of citizens to improve environmental governance. EI2 - Public spaces is based on the assumption that good quality of public spaces in urban areas, such as green areas or public squares, adds significantly to societal comfort and well being and concerns all citizens. Thus, citizens have a legitimate claim to be involved in the development of these spaces, and their associated environmental services. The aim is to involve citizens in the planning of public spaces. EI3- School indoor quality addresses the evidence that the physical and chemical environment at schools is often degraded, thus affecting adversely the well-being, health and learning capabilities of schoolchildren. It aims to provide simple tools to monitor the indoor school environment and based on the results provide information towards improvement.
These EIs will be carried out across culturally, socially and geographically distinct settings, and will provide use cases in a number of cities where the complex links between technological capabilities and their use for societal benefit can be assessed. This will help providing solutions that maximize both technological and societal innovation in the field of environmental governance.

We have initiated so-called Empowerment Initiatives (EIs) to demonstrate this chain in the following nine cities: Barcelona (Spain), Beograd (Serbia), Edinburgh (UK), Haifa (Israel), Ljubljana (Slovenia), Oslo (Norway), Ostrava (Czech Republic), Vienna (Austria), Vitoria-Gasteiz (Spain). Each EI belonged to one of our three topic areas: Urban quality, Public spaces, School indoor quality.

The novel approach of CITI-SENSE was to combine a technical approach with the “people side” in order to empower citizens to engage in environmental decision-making. From the technical side, we used both static and mobile sensors to monitor various environmental components. The data have been paired with innovative data fusion methods to enable an efficient communication and engagement with users and the general public. This was done through a range of products and services that have been developed within CITI-SENSE: the Citizens’ Observatories Web Portal (www.co.citi-sense.eu) a Personal Air Monitoring Toolkit, the CityAir smartphone app, an On-line Air Quality Perception Questionnaire, an Environmental Monitoring Toolkit for Public Places, a Data Visualisation Webpage and a Data Download Webpage.

Project Results:
Introduction
Environment and air in urban public spaces, climate change in cities and indoor environment especially in schools, concern and engage most of us. Often, we may feel that information is not sufficient, and data may vary significantly in quantity, quality and accessibility. Accordingly, the practice of co-decision on urban and environmental matters often falls short on involving those who are directly affected.
CITI-SENSE explores ways how to increase the involvement of the public in environmental decisions, both directly and through provision of citizen-collected data. The main objective is to develop technology enabled “Citizens’ Observatories” to empower citizens and citizens’ groups:
• to contribute to and participate in environmental governance
• to support and influence community and policy priorities and the associated decision making
• to contribute to European and global monitoring initiatives.
The project teams in the nine participating cities (Barcelona, Belgrade, Edinburg, Haifa, Ljubljana, Oslo, Ostrava, Vienna and Vitoria-Gasteiz) explored together with the citizens the information needs and the technological options to be used. As a result, to date the largest urban air quality sensor network was made operational, with 324 air quality sensor units installed in the participating cities. A number of additional sensor devices were also tested and improved.
CITI-SENSE developed and implemented information flows from sensors to users, and designed new methods for visualisation of air quality information using data from several sources at the same time. We developed among other, a method to provide a real-time air quality map that can be done for almost any city where sufficient number of sensor devices are in place.
We engaged a large number of citizens in testing and using our technologies. Through participation of schools, pupils were able to develop their own environmental projects, expanding on our technological solutions in the process.
We aimed to meeting the needs of the end users using a number of approaches. We investigated the knowledge and attitudes to air pollution and environmental quality, and we also collected plentiful feedback from the public regarding their perception of air quality. The CityAir app that allows to share subjective assessment of outdoor air quality, is freely available for Android and iOS. During the project, it was downloaded by more than 1,200 users around the world and used for collecting air quality perceptions of their localities. This app will continue to be available also after the end of the project.
All our results, including information on sensing technologies, communication solutions and computer codes for the communication platforms and for end user products, questionnaires, scientific publications and information materials for the end users are freely available on our web page http://co.citi-sense.eu.
In this report, we will first describe the Citizens’ observatories, or case studies, we performed. Then, we will discuss the citizen engagement and empowerment activities, before we describe briefly the technological basis we have developed. Finally, we describe the portal that provides all information from the project to the public, internal and external evaluations, and lessons learned.

Citizens’ Observatories in Nine Cities
We operated in all 24 individual Citizens’ Observatories across the nine participating cities: eight on outdoor air, one on quality of public places, and then, individual observatories for schools participating in the school environment investigations. We worked with citizens and students to demonstrate that they can play a role in collecting environmental data, and to engage them on local environmental issues. In eight cities, Barcelona, Belgrade, Edinburgh, Haifa, Ljubljana, Oslo, Ostrava and Vienna, we addressed outdoor air quality. In Oslo, Belgrade, Edinburgh, Ljubljana and Haifa we collaborated with schools. And in Vitoria-Gasteiz we examined the personal comfort in public spaces. Over the four years we connected:
• 1,200 CityAir App users with 2,036 reported perceptions
• 334 air sensors units connected in one pan-European network
• 327 volunteers who used our portable air quality sensors
• 1,530 answers to our various questionnaires and evaluation reports
• Students, pupils and individuals connected to 3 universities, 7 secondary schools,17 elementary schools, 54 kindergartens and 9 tenants associations
Throughout these activities, we collected more than 9 million observation during the last year of the project only, most of them in Haifa. Oslo and Ljubljana.
Methodological areas and project phases
The methodologies used in CITI-SENSE span technological development and citizens’ participation. This required a wide collaboration across the following four elements: (i) sensor technologies; (ii) information and communication technologies; (iii) information products and services; and (iv) citizen involvement.
The timeline of the project was divided into four main phases, each including a number of loops allowing us to reflect feedbacks. This led to a cyclical development of the methods and technologies, at each cycle collecting input from the users within the consortium, and the volunteers involved in the project.
• Phase 1: Prototype and pilot phase for selected sensors and locations
• Phase 2: Evaluation of pilot studies
• Phase 3: Full implementation for all sensors and locations
• Phase 4: Finalization and dissemination.

The case studies: CITI-SENSE Empowerment Initiatives
CITI-SENSE has developed “Citizens’ Observatories” (COs) for three types of issues: urban air quality, indoor air quality in schools and quality of outdoor spaces including also noise and thermal comfort. The COs have been implemented in multiple European locations as cases with different users, known colloquially as ‘case studies’ or technically as Empowerment Initiatives (EIs
In creating the COs, the project aimed to: (i) raise environmental (air quality and noise) awareness in citizens, (ii) raise user participation in societal environmental decisions and (iii) provide feedback on the impact that citizens had in decisions. In this manner we addressed issues related to effective participation by citizens in environmental stewardship, based on stakeholder and user involvement in support of both community and policy priorities.

Urban (air) quality
The COs in this Empowerment Initiative focused on a range of services related to air quality, an issue of societal concern: combined environmental exposure and health associated with ambient air quality. The COs naturally require representative citizen participation. In each location, the case studies were designed in collaboration with local citizens’ groups and decision makers. They were based on distributed data collection using innovative static, portable and personal devices (low-cost microsensor packs) that communicate with data repositories through mobile phones or other devices. We conducted the work using participatory methods, data management strategies, and applications that facilitated exploitation of the data and information for policy, and society, as developed in the methodological parts of the project. The work that led to successful full field implementation included the following steps:
• A protocol, compiled of the different pilot study elements. Aimed to give specific operational details for how the work was to be performed in each location, EI and each phase of the EI.
• A detailed description of the set-up and deployment plans (as different phases) of the end-to-end prototype (information chain for the main study), describing aims and objectives of particular pilot case studies, site descriptions, and social and technical methods.
• Evaluation of the pilot phase using key performance indicators. This was used to prioritize the necessary developmental work, and informed the final protocol.
• Detailed main study protocol for the location-based EIs. Upon its delivery, the locations had started to gather environmental sensor-enabled, health-relevant information on outdoor inhabited (urban) spaces. The protocol facilitated the creation of a location-based citizens’ observatory to engage and empower citizens, authorities and policy-makers (as stakeholders).
The protocols gave a brief and general description of the Full Implementation (the main study) process from four aspects: (1) citizens observing their environment; (2) building a community of citizens that are participants in a citizens’ observatory; (3) participatory evaluation of the quality of citizen empowerment in a case study, and; (4) comparative analysis between cities.
The tools we used for public engagement were all associated with atmospheric quality. We engaged subjective reporters (e.g. citizens giving their opinions) as well as objective monitors, measuring air quality levels that were then presented using a novel air quality indicator and online visualisations. The subjective reporters used the Long Air-Quality Perception Questionnaire and CityAir application, which give general and momentary assessments (observational reports) of citizens’ perceptions. The air quality monitors used were the AQMesh pods and the Personal Air Monitoring Toolkit based on the LEO sensor. These tools provided city-wide and personal measurements. The measurement results were expressed as the Air Pollution Indicator (APIN), based on a common air quality index CAQI developed by the CITAIR project (http://www.airqualitynow.eu/about_indices_definition.php) and visualised on the Citizens´ Observatory web portal. The need to develop a specific APIN was motivated by the difference in what measurements are used: while the CAQI is based on compliance monitoring data, the APIN is based on much less accurate and precise low-cost sensor measurements. Due to the differences between reference instruments and the low-cost sensor platforms, the APIN and CAQI are not directly interchangeable. Needless to say, we have investigated the instrument differences thoroughly (as described in the chapter on sensor platforms).
The tools developed, used and evaluated within the environmental quality EI were:
• The CITI-SENSE Citizens’ Observatories web portal (http://co.citi-sense.eu) provides an access point to all the project apps, widgets, web pages and sensor based tools and questionnaires.
• The Personal Air Monitoring Toolkit (Little Environmental Observatory – LEO) allows users to assess air quality in their immediate surroundings. It is based on a sensor device that monitors three gases (nitrogen dioxide, nitrogen monoxide and ozone) and a corresponding mobile application (ExpoApp).
• The CityAir smartphone app allows users to share their perception of air quality, and of the dominant source, anytime, anywhere.
• The Environmental Monitoring Toolkit for Public Places can be used in campaigns to assess thermal comfort, soundscapes and visual qualities of outdoor places such as parks or public areas in need of rehabilitation.
• The On-Line Air Quality Perception Questionnaire was used to assess in-depth perception of air quality.
• The Data Visualisation Web Pages provide an overview of the data collected using all the sensor-based tools.
Evaluation exercises aiming at learning how the use of different tools contributed to the participants’ ability to engage in environmental issues were done using focus groups and interviews in all locations. For example, individual interviews involved the users (hosts) of a LEO, interviewed at the end of their participation period. They were asked about their experience, if the tool helped them to do something differently and if they thought they might use it as part of their daily life. Similar questions were posed in focus groups of multiple stakeholders for other tools.
The case study coordinators (also called Location Officers, LOs) learnt many lessons on using such novel technology with stakeholders at different levels of technological capability and general understanding about air quality. This has enabled them to carry the experiences forward in new initiatives. For example, in Barcelona and in Oslo, a follow-up projects were successfully applied for by the project team in collaboration with CITI-SENSE stakeholders. Another example is in Edinburgh, where a workshop focused on the use of ‘low cost sensor technology to monitor air quality and engage with citizens’ was held. Local stakeholders from research, local authorities, NGOs, sensor manufacturers, etc., shared their opinions on the opportunities and challenges to consider when using such tools to engage with citizens so that efforts in this area can be appropriately moved forward. Yet another example is in Vienna where a university seminar took place over the duration of three months which aimed at making students aware of citizen science, it´s practical applications and the empowering effects these approaches can have if understood and applied correctly.
The project learnt from citizen experience and perception and in some regards enabled citizenship co-participation in community decision making and co-operative planning (in other words, “empowered”). As examples, environmental awareness was raised in young citizens (students) of Barcelona, where mobile sensor technology allowed them to observe the behaviour of important components of their environment such as air quality and noise. In Ljubljana, young citizens (again students) were enabled by online platform technologies to develop their own smartphone application for reporting air quality levels to raise participation by their peers in important decisions regarding their environment Finally, the citizens of Ostrava worked with local Non-Governmental Organisations (NGOs) to address, from another angle, the long-standing issue of air pollution due to industrial emissions to heighten their impact on the political landscape.
The Ostrava case study has posed possibly the most interesting “empowerment” challenge. In this area, air pollution levels are exceptionally high compared to most European cities, and also many locally based studies have shown a relationship between health impairment and air pollution. Thus, providing air quality information is not the main need in this area: people rather ask ‘what can be done to improve the situation?’ It has been the ambition of the project to contribute to such societal dialogue through the involvement of the local community, which also resulted in media attention. Cooperation with the city authorities unveiled a demand for raising children's awareness on air pollution. Subsequently, educational programs about air, air pollution and its prevention have been carried out in kindergartens and primary schools with some success. At the end of all EI activities, a public seminar was organized in Ostrava for all stakeholders presenting the project results and enabling a discussion about air pollution problems in the region and potential solutions. Steps towards empowerment have come through raising awareness not on the level of air quality, but on the methods for monitoring and sharing such information on solutions. In other cases like Vienna, where the authorities had been sceptical about data quality coming from the sensor platforms employed, they still were able to see the benefits from low-cost technology for local air pollution monitoring and sharing information to move towards empowerment.

Indoor air quality in schools: the schools wanted a shift of focus
The initial aim of the schools empowerment initiative was to support citizens’ participation in improving the indoor environment in their schools. The original idea of this case study was to give the schools the possibility to monitor their indoor environment and then identify measures for improving the indoor air quality.
However, through discussions with the schools in the pilot phase it became clear that their main interest was how the project could be used for educational purposes. The school administration and the teachers wanted to give the students, through participating in the project, a better understanding of air pollution, what it is, how it affects their health and what measures can be taken. In addition, they saw the participation in CITI-SENSE as a means to develop the students’ understanding of scientific thinking – asking scientific questions, planning experiments, reading graphs, analysing data and forming conclusions.
In practice, the schools took over and changed the purpose of the case study to a new one, of engaging pupils in monitoring and in learning about the environment and using this as part of their formal schoolwork, within the school curriculum. This change was entirely consistent with the CITI-SENSE aim of supporting empowerment of participants (empowerment: enhancing an individual’s or group’s capacity to make effective choices, effective in the sense of enabling them to transform those choices into desired actions and outcomes) and so was welcomed by the project team. Being a “user-led citizen science project” the main objectives were changed according to the users’ needs.
Five locations have been involved in the school initiatives - Belgrade, Edinburgh, Haifa, Ljubljana and Oslo. A total of 18 schools and more than 1000 students were involved throughout the project period. The main engagement activities included:
▪ Correspondence and meetings with school administration, staff and students
▪ Lectures and information campaigns
▪ Installation of sensor units and organising measurement campaigns
▪ Science projects (primary and high schools)
▪ Training of students, guidance and motivation
▪ Nature Days
▪ Student conferences and competitions (national and international)
▪ Questionnaires/surveys to track students engagement and awareness raising
▪ In-depth interviews with students and teachers
The original plan for the school case studies was dependent on air quality sensor units and ICT support tools to be developed by teams within CITI-SENSE. The risk of a failure of the schools empowerment initiative due to the technical problems and delays was identified quite early in the project. The school partners decided to buy off-the-shelf sensors to measure the most important parameters, such as temperature, relative humidity and CO2. This turned out to be a very good decision and made it possible to go ahead with the planned activities. Many of the off-the shelf sensors on the market are ready to use with minimal set-up effort and a stable/reliable dataflow. The sensors very often measure the parameters that the schools are most interested, that is temperature, relative humidity and CO2. These sensor units are therefore well suited for engagement and teaching purposes in the schools.
The lack of sensor units during the pilot case study had a positive effect as it forced a shift of focus from ”the use of advanced sensor units” to acitivities that led to engagement and empowerment of the students and teachers in environmental issues. This was particularly evident in the Ljubljana case study where several engagment activities have been carried out that did not rely on the use of sensor units at all, such as a Bike and Balloon campaign (D3.4 2016). Non-sensor related engagement activities, such as the Nature Days and the Bike and Balloon campaign in Ljubljana proved to be suitable ways to raise awareness and engagement in air quality issues.
Working together with schools requires close cooperation and transparency since the schools require a relatively high degree of control in defining activities and educational goals. Therefore, it is crucial to be flexible and willing to adjust the scientific work according to the needs and wishes of the schools.
The school case studies demonstrate the interest of both students and teachers in participating in research projects, and that they are able to perform studies of high quality. While working also with younger students, we realised that they provided us with unexpected, but great results. Providing them room for creativity is probably more important for engagement with high school students, but at the same time guiding from teachers and scientists is also required. This applies especially for data management.
Studies like ours, where close interdisciplinary liaison is required (teachers, students, scientists), it is important to recruit experienced and strongly motivated teachers. Engagement of the students is often very much dependent on the teacher’s skills.
Our activities with the schools have demonstrated that students may be valuable collaborators for citizen science or citizens’ observatories projects. They can measure physical parameters, collect observations and perceptions while beeing supported by experienced teachers and scientists. Furthermore, they can provide insight into the priorities and goals of the participants as well as the drivers and barriers for improving school environments. However, there are some challenges that need to be overcome to realize this potential:
• Successful cooperation with schools requires the ability to adapt to curricula and relatively strict schedules. Smaller delays may result in a full year loss.
• Students are motivated when they are free to implement their own ideas and priorities into the projects. This may lead to datasets that are difficult to re-use or compare with others.
• Indoor environments vary widely between rooms and with usage, and meaningful interpretation of measurements and other results often requires that a lot of context be recorded with the measurements. This is not always practicable/feasible. Despite the challenges, there is a tremendous potential of empowering the schools when providing them with tools to conduct research in their premises.

The Public Places EI – Vitoria Gasteiz Citizens’ Observatory
This Empowerment Initiative developed a “Citizens’ Observatory” (CO) to empower citizens to contribute to and participate in environmental governance of public spaces. The purpose was to enable citizens to support and influence priorities and decision making of the local authority regarding the creation, renovation, preservation, and management of public places. The challenge faced in this Empowerment Initiative (EI) was to test in practice in the city of Vitoria-Gasteiz, if a CO can support and enrich citizen empowerment in this field, engaging citizens, city authorities and other stakeholders.
We approached the study in two complementary ways:
1. Build a community of users and followers of the project interested in the outcomes of the project. This was addressed by launching a General Questionnaire for a wider audience with regard to environmental comfort in public spaces. It was a valuable instrument to confirm the influence of the environmental quality and comfort perceived in relation to citizens’ satisfaction and quality of life. Moreover, citizens reclaim more public information about the management of public spaces, comfort and environmental quality. Related to COs, they believed that new technologies, sensors and smartphones can definitively play an important role, particularly in public participation processes and empowerment initiatives
2. Invite volunteers to make observations in diverse urban public spaces, to assess the environmental conditions of those spaces and to enrich the objective data collected with their perceptions. Altogether 55 participants were engaged, recruited from different associations as well as citizens that usually make regular use of the observed areas. They collected up to 139 observations, assessing comfort levels in the four public spaces selected by the municipality. Since the interest of the municipality was to check the contribution of the COs to the management process of public spaces, they selected areas considered for rehabilitation.
Several methodological approaches were employed during the whole process to achieve citizens’ engagement, to support the empowerment process and to evaluate the experience: introductory and feedback workshops, focus group methodology, evaluation questionnaires and in-depth interviews.
The information collected regarding participants’ expectations indicate that they were motivated enough to participate in improving the selected areas. Their participation also enhanced their interest in the scientific experiments and sensors use.
During the data gathering phase in the EI, a toolkit was provided to all the participants. This toolkit, specifically developed for the EI, combines hardware and software tools that measure objective and subjective parameters of the environmental conditions in urban spaces. The user has to follow a protocol that guides them into a mindful observation of the urban places. The data provided by sensors was accurate enough for the project´s aims and goals.
The participants’ feedback was collected in several stages of the project, using different methods. With the aim of providing clear information, a workshop was devoted to the co-design of how to visualize some complex analysis at the webpage results. In that sense, visualization widgets used are one type of outputs from the project.
Collected data was valuable for the decision making process related to the four specific public places observed. It facilitated a diagnosis of each place; the environmental quality assessment (understood as how it is perceived) and collecting ideas or proposal to be taken into account into the urban management strategy.
Moreover, the comparative analysis of the whole set of observations may improve the general understanding of which information on public places is interesting for the participants, and how such areas are perceived. The main conclusions are:
• Stays in the urban public places of the study has positive effects on perceived health;
• There are interdependencies among different aspects of perception: Visual perception at an urban place influences strongly the perception of global comfort;
• The most liked and disliked elements seem common to the places investigated, and the positive effect of natural elements on places’ perception can be quantified.
The Vitoria-Gasteiz experience is seen by the participating public as a successful project. On average, participants assessed the experience as rather positive. They described it as useful and a beneficial, and the use of new technologies was described as an opportunity.
Institutional stakeholders of Vitoria-Gasteiz evaluated CO for Public Places as rather useful although they were slightly judgmental, since they consider that the experience did not create a considerable empowerment among participants. They also recall that the obtained outcome and results were predictable. Nevertheless, they found that the technological product was interesting and potentially attractive for the environmental monitoring, since they recognized that new technologies implementation is a fundamental step. The CITI-SENSE approach was defined as a “great idea”, but still on an experimental trial level, still needing development before it can have a considerable impact among Vitoria-Gasteiz citizens.
Since this EI has been implemented in a single city, we tried to test the experience also in other cities. With that purpose, several interviews were made with diverse stakeholders, and according to them, the Environmental Citizens’ Observatory for Public Places has the potential to be extended, implemented and adapted in other cities. Furthermore, the experience with COs at public spaces was perceived as positive and useful for the environmental improvement. Indeed, they say that municipalities can benefit from citizens observatories into the decision making processes.
Internal evaluation of the initiative was positive. The achievement level measured by the project’s KPIs is Good (86%), being excellent in sensors, platforms and products and medium for the users’ indicators. KPIs’ evaluation helped to discover the product improvement possibilities: i.e. developing the products more friendly for elder participants and improving the degree of accessibility by implementing the apps on their own smartphone.
With regard to the project´s potential impact among the population, according to the social impact indicator made for the project, it can be said that CITI-SENSE has reached 50% of Vitoria Gasteiz citizens engaged in environmental campaigns or actions..
Finally, the practical experience developed in the CITI-SENSE project, that applies Citizens’ Observatories to empower citizens in the management of urban spaces, stems from some general contributions to Citizens’ Observatories development, related for instance to the role of City Authorities, or to the hot topic of Privacy issues related to Citizens’ Observatories. To conclude, we would like to offer some recommendations for municipalities, such as the possibility to undertake a COs experience to empower citizens into the understanding, management and/or the improvement of urban spaces (since COs can complement traditional participation processes). We also recommend being very clear with the type of empowerment framework that is being deployed, in order to manage citizens’ expectations. Another recommendation is to adapt the product development in the project to the peculiarities of each empowerment exercise and to the particularities of places. In that sense, the definition of how products could be adapted to each situation could be a clear opportunity for co-design initiatives.

Citizen participation
The CITI-SENSE communication model was formed as an open dialogue among scientists, technology providers and citizens (including teachers, civil servants) with the aim to learn from each other’s experience and perception and to enable co-operative planning. The cyclical development of the Citizens’ Observatory focused on dialogue between the different actors. The Citizens’ Observatory acts as the ”mediator” or facilitator of these developments and allows evaluation of concepts and continuous improvement. Parallel development of enabiling technologies, scientific methods and user interactions is essential for its success.
In CITI-SENSE, we chose a participation method that went beyond a simple collection of data/information. We wanted to give citizens (stakeholders) a voice, to be able to engage with local decision makers in environmental problem solving in the local society. The CITI-SENSE engagement process consisted of three main parts: preparation, participation and evaluation.
1. Preparatory phase
The preparatory phase refers to the project design. It involves the following aspects:
a. Composition of a balanced project team
b. Context analysis of the issues, location, decisions to be made and clarification of the purpose of the participation process
c. Stakeholder analysis, including stakeholder identification, definition of their character and attitudes and the relationships between different stakeholder groups
d. Engagement plan including expectation management
e. Choice of participation methods on the basis of the stakeholder analysis
f. Planning the process: action and communication plan
g. Recruitment of participants
2. Participatory phase
This phase is the practical part of a project. In this phase, we applied the participatory methods that we developed in the preparation phase. In CITI-SENSE, we used the following participatory methods:
Survey with open questions: Questionnaires can reach a broad audience, obtaining both qualitative and quantitative information.
Interviews: Interviews are very valuable to obtain more detailed information about a certain issue than a questionnaire. This method can also be used in the evaluation phase.
Focus groups: In a focus group, a group of 6-8 people is asked about their perceptions or opinions towards a product, service or concept. This method is valuable for the design of a project or for evaluation purposes.
Perception monitoring: People’s subjective perceptions can also be an interesting method in participative activities. Through questionnaires or mobile applications, participants can report their personal experiences in certain situations, places or about certain topics. Here, the participants can be engaged in the design of the questionnaire, app, web site, etc and also participate through their answers/collected information.
Co-design/ Co-creation: Co-design or co-creation activities are crucial tools not only for participation but also for empowerment activities/projects. Involving the participants in this active way will make them feel more connected to the project, since they have been part of developing it, and the results will be more acceptable to the participants. In addition, you will receive feedback and viewpoints from other perspectives, which can also be very enriching. It is generally recognized that the quality of design increases if the participants' interests are considered in the design process.
3. Evaluation
Evaluation provides feedback necessary to adapt the project methods in all steps of the process, so that the goals can be achieved. The evaluation process needs to be planned and built into the project cycle. An interim evaluation should be carried out after each important step or milestone. In this way, the activities can be monitored, improved and reevaluated. An end-evaluation of the field studies allows to measure the outcomes and impacts. In the CITI-SENSE project, we executed both an internal and external evaluation, involving project members and the participants in each location.

How to work with users: methodological support
In order to prepare the local study coordinators for implementing best practice in engagement and empowerment of a range of stakeholders during the main study, CITI-SENSE had to provide methodological background and practical help. Key were three very interactive and collaborative workshops.
a. The first workshop-meeting was held in late September 2014 in Barcelona. It aimed at developing coherence and skills for communication and empowerment using a stakeholder communication framework, in order to optimise the methodological support to the Location Officers (LOs) in the implementation of the EIs.
b. This was then strengthened and expanded in a larger workshop in Oslo in early December 2014, for all of WP5, 2 and 3. The overall aim of this meeting was to develop a plan for engagement and empowerment in the main study implementation of the EIs.
c. Finally, there were further collaborative meetings during and immediately after the CITI-SENSE full consortium meeting in Barcelona in March 2015.
All these meetings were jointly planned, structured to ensure small team discussions and active participation. They succeeded in establishing collaborative understanding and atmosphere, focused on practical issues that need to be in place for the full implementation.
In addition, a comparative analysis of the experience of the different EIs through the period of the pilot study was carried out, with the aim of identifying cross-cutting issues and integration issues across locations and of drawing lessons from all the pilot studies which will be informative for implementation of engagement and empowerment in the main study. The outcomes of this comparative analysis will also contribute to the final evaluation of engagement and empowerment of the study as a whole.

Working with users: results
The work has led to development of very practical collaboration products for use in the main study fieldwork. These products are focused on crucial issues such as perception monitoring (two methods are now being finalised, an app for ‘just-now’ monitoring of perception, and a questionnaire for assessing longer-term perspectives on air quality); data-related issues have been discussed and draft plans made (for user agreement, data flow including sensitive issues of confidentiality, data visualisation for individuals and for the public – this was led by WP7, assisted by WP1) and more general issues of communication (e.g. on air quality and health) have been considered. The principle of co-design with user involvement has been highlighted and discussed and is being implemented in some locations on particular issues.
The comparative analysis was complicated by the differential quality of information provided; the detailed planning and response forms from D5.2 were been used consistently. The exercise has however helped to highlight the need for consistent reporting, even if this seems a burden to already hard-pressed LOs. In the last months of the project, this work of Task 5.5 (Coordinated analysis across empowerment initiatives) has been further developed and operationalized for D5.4 ‘Methodological study to support the representativeness of citizens' participation and beyond’.

Evaluation of the empowerment potential
CITI-SENSE has been very complex and difficult, but was also a project of great learning. The project had to operate on the interface between expertise on advanced measurement and information technologies on the one hand and expertise about engagement and empowerment of citizens on the other. It had to do this in the context of improving people’s engagement with environmental issues of consequence, and increasing their power to be active participants in improving environment and health. This needed to be applied in the practical context of local case studies called Empowerment Initiatives (EIs). In practice however, most partners involved in the project started from a technology driven background. What made it even more difficult was the fact that the local empowerment initiatives were carried out by teams which consisted principally of physical scientists, exposure assessment experts, experts in measurements, etc. – teams with the expertise necessary for testing and managing the use of static and personal sensors – but CITI-SENSE also asked these EI teams to do the practical work of engagement with citizens to support empowerment.
When it became clear that many of the sensors were not performing as had been hoped for and expected, the approach of the project needed to be and was adapted, but the technological difficulties with the reliability of sensors and the quality of data stayed on the forefront for a long time, together with a preoccupation with resolving those difficulties. Over time, the CITI-SENSE-project developed alternative and/or complementary products that provided options for engagement and empowerment. Nevertheless, this tension, arising because engagement and empowerment were (considered) dependent on the good functioning of the technology, was significantly influencing the project. The technology did not function as had been hoped for and expected, and although big gains were made in its development, it left the engagement and empowerment aspects of CITI-SENSE with three major difficulties which impacted seriously on the engagement and empowerment activities:
1. The work of engagement and empowerment could not go ahead as originally anticipated, even though substantial progress was made on the engagement aspects later on in the project, and the work on empowerment remained limited due to the long delays in getting a citizens’ observatory operational.
2. Those delays related to technology took resources and attention, not only from the teams with responsibility for sensor and other infrastructure development, but also from the Empowerment Initiative teams and from the methodological support team on engagement and empowerment itself.
3. It seriously affected the work on engagement and empowerment, as the methodological development could not always be linked as envisaged with practical applications.
To solve this situation, it was essential to have a project strong on interdisciplinary collaboration and open to learning. In the end, various alternative ideas were worked out into additional CITI-SENSE-products which were used with and by citizens. For example:
• The development of the smart-phone app (the CityAir app) was developed to enable citizens to record their subjective perceptions of air quality at a specific place and moment in time, and to have these results displayed visually as location-specific color-coded results on a map.
• The development of an perception questionnaire to capture the perceptions of air quality that were expected to vary over much longer timescales.
• In parallel, the team of the school empowerment initiatives developed some creative approaches to their work of engagement and empowerment, as the schools shifted their focus of the work to building environmental issues into the curriculum with the help of the project (for instance by working with off-the-shelf sensors).
The Empowerment Initiative team working on maintained public spaces in one city (Vitoria-Gasteiz in the Basque Country of Northern Spain) had from the outset designed their work to include e.g. perception monitoring as well as ‘objective’ measures of soundscape and thermal environment; and so their work was relatively protected from the long-lasting difficulties with sensors that affected other teams, especially on outdoor air.
For many of the local empowerment teams on air quality in the eight participating cities, CITI-SENSE also involved getting to know and understand much better the local policy context in which they were doing their scientific work – the possibilities for change and the barriers to it. This led to some reflection about what might be considered an ideology underlying the use of citizen science and citizens’ observatories which perhaps needs to be made more transparent and examined more critically. The original viewpoint that the Empowerment Initiatives (or the CITI-SENSE-project in general) would influence policy-making or decision-making for the better, mainly by engaging people in taking measurements, has challenges. Although the assumptions that:
• the additional knowledge (gained from sensor measurements that were considered usable and useful) would lead to better decisions, and;
• the additional engagement of citizens, NGOs etc., in gathering and/or understanding these measurements, would lead to better methods of decision making,
...might seem plausible at first sight, they were not at all obvious. Due to the challenging timeline we feel we did not realize the full potential within the project, and we have to be cautious when drawing final conclusions. However, the story of the dimensions of learning – all of which were happening in parallel, but with different intensities at different times – provides important and meaningful way of looking at the project and will be useful for similar initiatives in the future.

User-evaluation of the empowerment potential of the CITI-SENSE-tools
The way in which we define empowerment seems to be very important for the development of the technologies intended for public use. One main concern is that the tools might only enable individuals to better adapt to a fundamentally unfair situation, without addressing the conditions that produce this unfairness in the first place. Within the CITI-SENSE-project, a potential risk of – unconsciously – developing counterproductive tools was found, especially if the empowerment goals would be defined in terms of having the “power-to-choose”. But even with more appropriate definitions of empowerment, challenging situations might occur. Although the tool-developers and scientists might have the best intentions, the tools may become “a Siren’s song seducing us to make poor choices”. This means that – no matter how empowerment is defined exactly – the newly available technologies can always have both empowering and disempowering effects (for instance strongly depending for which purposes the output of the tools would be used). Examples of the possible negative consequences are:
First of all, people might be seduced to make poor choices in terms of real empowerment, even although they might seem to be reasonable choices at first sight.
• The possibility to take an alternative route might give the wrong message. This choice would only re-enforce the status-quo: people who are giving the good example will be punished (by making them walk or cycle longer distances in an attempt to avoid AQ-hotspots).
• When there are no alternative routes available, people might also decide to stay inside. Once again this will rather be a doubtful way of only enabling (vulnerable) people to adapt better to fundamentally unfair situations. And sometimes there will be no other option then leaving the house (e.g. to bring the children to school, to go to work, etc.).
• Deciding to stay indoors might even be more problematic, because air pollution does not stop at people’s doorsteps. Indoor air often seems to be more polluted than outdoor air, this might also be a poor choice. In 2007 the European Scientific Committee on Health and Environmental Risks (SCHER) recommended “that any studies to correlate outdoor air concentration with health effects need to consider the impact of indoor exposure”.
• Similarly, it can also be questioned if the choice to wear a mask when people go outdoors would empower people. Although it can indeed contribute to your health, this poor choice is giving the same wrong message to polluters.
• Moving houses because of bad air quality can also be a poor choice for different reasons. It might for instance re-enforce urban sprawl, and therefore increase the AQ-problems. People who don’t have the financial means to move away, will stay behind in communities that might become even more vulnerable.
• Using these tools to estimate property values was also explicitly mentioned. This means that in a worst case scenario, the tools can re-enforce “gentrified sustainability”, “environmental gentrification” and “smart segregation” (instead of smart growth) due to unrestricted market processes that lead to changing housing prices based on the AQ-data that is made available. Again, for those who are left behind it will probably cause feeling of frustration.
• Although not fully comparable with the Empowerment Initiatives (EIs) about outdoor air quality, similar challenges might occur in the school case studies, for instance when parents would start to select the most appropriate school for their children in relation the AQ-levels that were measured.
Secondly, frustrating situations can occur when there are no real alternatives available for the informed citizens, or if they are not aware of how they can deal with the issue:
• When citizens are made more aware of AQ-problems while they do not have alternatives to solve them, this can often lead to frustrating feelings of “learned helplessness”.
• When citizens are intensively made more aware of AQ-problems without making it equally clear to them how they might be dealt with, this can also lead to frustrating feelings of “learned helplessness”.
Besides the risks of making poor choices, there are also other ethical limitations regarding “individual choice” in terms of a person’s choice to refuse being held responsible for his/her contribution to the problem of air pollution. The very interesting debates on Odum’s/Kahn’s “tyranny of small decisions” and Hardin’s “tragedy of the commons” can be seen as an interesting starting point for this debate. And there might also occur problematic situations when valuable information (e.g. air quality data) would be gathered, but then it is not acted upon:
• There might be situations in which polluters and/or local authorities will try push away their (complicated) tasks and responsibilities to the level of individual citizens who will then have to help themselves (as part of an “empowerment as freedom to choose” approach.
• The inactivity of authorities can force individuals to become activists, because responsibilities for solving AQ-problems are shifted towards them. The potential risks of being threatened by opponents might also increase then, especially when stakes are high.
Other challenges that were mentioned:
• On a more general level, privacy and data security were also addressed as concerns (at least by some of the participants).
• The potential risk of creating “hysteria”, for example when alerts about air quality are send out to individuals and/or schools (especially when risk communication is not been taken into account).
Finally, besides all these challenges mentioned above, also the idea that empowerment would follow rather automatically from the collection of new data about air quality was questioned by some of the participants.
Often, participants were not fully aware of the possible negative side-effects. Therefore, the societal challenges have to be acknowledged by all scientists involved in these kinds of projects, whether these are participatory (citizen science) projects or projects that mainly focus on the development of the technology itself. Also the ethical debates about the tools or the topic of air quality in general should not be avoided. Both positive and potentially negative outcomes should be discussed openly with citizens, right from the start. If not, there is a real danger that these new technologies might become contested.
Answering the question if all these risks for (potentially) negative outcomes should withhold us from making detailed air quality measurements is more difficult. Taking into account the goals for Responsible Research & Innovation, the potential risks should not be underestimated. But these data can also be used very positively to solve (local) air quality problems. Besides that, inactivity would also contribute to the remain of the status-quo (e.g. no empowerment achieved). However, the processes, the activities and the ways in which goals are set should be selected very carefully and the results or final outcomes should be evaluated and communicated appropriately. Tools which aim to support empowerment should be co-developed carefully together with the target-groups (e.g. co-design).

User-evaluation of the overall outcomes of the project
Evaluating outcomes and impact of research in terms of empowerment can be very challenging. Societal impact is much harder to measure than pure scientific impact, societal impact can often take many years to become apparent, and the routes through which research can influence individual behavior or inform social policy are often very diffuse. Within the timeframe of this project, it was not possible to evaluate all kinds of outcomes, especially not the outcomes that might require more time to occur. The user-evaluation is therefore limited to the first (intermediary) outcomes and possible impacts of the project. Although it is important not to overestimate the impact of the CITI-SENSE, it would also be a missed opportunity if the overall impact of the project would be seriously underestimated. Following indications regarding outcomes were found:
• Gain of scientific knowledge: not only scientific knowledge about air pollution, but also other relevant knowledge (for instance about these kind of new technologies in general);
• Learning of new skills (for example regarding the use of these technologies);
• Awareness-raising amongst the wider public (which was also be increased as a result of media attention for the project);
• First (preliminary) indications of behavioral changes, although these were often still rather small practical changes in behaviors (but which might be seen as a first step towards more sustainable behavioral changes in the long run);
• Increased networking (with other interested people involved, with scientists, authorities...) and first indications that there is definitely a potential to start up these kinds of Citizens’ Observatories;
• Increased feelings of having a purpose from now on (e.g. in terms of convincing other people to become more environment-friendly);
• Increased (preliminary) feelings of empowerment, although at the same time other participants also warned about possible negative feelings of disempowerment if other’s would not be interested in their concerns);
• First (preliminary) indications of “giving people voice” (e.g. through the CityAir-app and Long Perception Questionnaire in which participants might share their concerns and ideas);
• Increased communication between different stakeholders (while also authorities mentioned that an increase in communication would be beneficial from their perspective);
• An increase in the personal activities of volunteers (with the aim to influence administrative decisions and processes).
Points of special attention that were raised, are:
• The need to be aware of possible solutions, as being aware of possible solutions is an important step in order to enable people to take more concrete action (which also shows that there is still room for improvement for tools in terms of being more solution-oriented);
• The importance of engaging people in activities involving environmental monitoring and assessments in order to raise awareness;
• The fact that giving voice to citizens in decision-making is definitely not enough (because a lack of interest amongst other citizens and/or authorities or the unwillingness to listen and to actively search for possible solutions might actually also contribute to feelings of frustration and disempowerment).
Not all of the expected outcomes – for instance changes in participants’ behaviors – will necessarily contribute to empowerment. As argued before, some of them might actually be disempowering too, if they mainly enable people to adapt better to fundamentally unfair situations (even if this can contribute in a positive way to the health of the participants for the moment).

Technologies for environmental monitoring and sensor platforms
We tested and improved several sensors and sensor platforms for air quality monitoring, a sensor kit for assessment of public spaces. We and deployed on large scale a subset selected based on their performance and ease of implementation.
In Vitoria-Gasteiz, we used sensing tools developed by Tecnalia that monitor and assess noise and thermal comfort of public spaces. It consists of a smartphone with different applications, an external microphone and a weather sensors unit (Kestrel®).
In the other eight cities, we deployed the Ateknea LEO personal platforms with electrochemical sensors for measuring gases (NO, NO2 and O3), and the Environmental Instruments AQMesh static sensor platform that can measure gases (NO, NO2, NOx, O3, CO, SO2), particles (PM1, PM2.5 PM10), relative humidity, pod temperature, atmospheric pressure and noise. In schools, we deployed the Atmospheric Sensors indoor unit for gases and particulate matter, and the Obeo Radon sensor. In addition, the DNET sensor platform was deployed in Belgrade. In total, we deployed 174 AQMesh platforms, 86 Ateknea platforms, 48 Alphasense/Atmospheric Sensors platforms, 16 Obeo platforms and 10 DNET platforms.
All sensor platforms, also those that were not used in the full field deployment, were repeatedly tested during the project, and improved. A review of each platform design and the tests, performed in collaboration with platform providers, were used as a basis for further improvements of each platform for monitoring air quality. The main findings concern two sensor technologies (metal-oxide and electrochemical sensors); four stationary platforms (AirBase, CVUT, DNET, AQMesh), six pollutants (CO, NO, NO2, O3, PM, TVOC) and three other environmental parameters: ambient temperature (T), relative humidity (RH) and noise; two modes of use of such a technology (stationary wireless distributed sensor network, WDSN, and personal/ mobile sensor nodes). Two portable platforms (Ateknea and JSI) were also investigated.
We evaluated the capability of the installed network to capture spatiotemporal concentration variations in the laboratory, during collocation studies, and during real urban-scale deployment. This was done by independent testing in different geographic locations, for varying time periods and seasons covering varying meteorological and climatological conditions. In general, we found that with the help of careful field calibration the platforms can identify spatiotemporal variability of the pollutant concentrations, both intra-urban (within a city) and inter-urban (across cities). In particular, we show that it is possible to identify intra-urban pollutant “hot-spots”. The similarity among the concentration patterns during weekends across urban areas and the variability during weekdays further support our conclusion that the sensor are sensitive enough to capture air pollution patterns and respond rather instantaneously to their microenvironment changing conditions, although their response may suffer from interference with many other environmental factors rather than reflecting only the nominal pollutant concentrations.
We also found and highlighted numerous limitations of the current sensor (and platform) technology, which require further development before the platforms could find their way to use for applications and by stakeholders that require accurate and precise measurements, globally comparable. The currently most promising sensor technology at present for measuring fine particle number concentrations was not considered in the project as it was not available when the project was proposed and launched. However, for applications that only require information about relative pollutant levels, either in time or space or both, we have documented a detailed account of the capabilities of the sensor technologies, and the maintenance they require. The latter include administrative maintenance (battery replacement, faulty sensor replacement, etc.) as well as repeated calibrations. Thus, apart from the classical laboratory calibration against high-end instrumentation and under fully controlled conditions, which we found to be insufficient for the technology at hand and for the different applications it is required to address, we suggest three different field calibration procedures (while the sensor nodes are collocated and two variations while the nodes are deployed in their required measurement sites – i.e. in-situ calibration approaches).
Typically, the accuracy of WDSN devices is assessed by reporting the mean error (ME) or correlation coefficients with respect to laboratory equipment. However, these criteria do not account for the sensors’ performance during their field deployment and, in fact, may not suit (e.g. be too conservative) many possible applications for this new technology for which accurate measurement are not a must, but for which either precision or even just pollution trends are sufficient. In order to be able to compare the performance of the sensor platforms in the field, we have developed a Sensor Evaluation Toolkit (SET). This work has been done using 25 nodes deployed in eight cities in Europe, as part of WP2 deployment within the CITI-SENSE project. All the nodes were collocated at AQM stations for three months, and their measurements were compared against those acquired by the AQM. While the SET requires a reference device to evaluate the sensor measurements, it does not make an assertion on the nature of this reference equipment. The evaluation involves a comparison of two concentration time-series: one acquired by the sensor and one obtained by the reference device. Without loss of generality, both time-series should be of equal length, i.e. consist of K measurements with the same measurement frequency. The SET consists of two common performance measures, RMSE and correlation, and four new measures: presence (represents the sensor reporting reliability), source-analysis (depicts how accurate a sensor is when it is used for a source identification), match (evaluates the sensor’s accuracy when the measured concentrations are transformed into generalized coarse scales), and Lower Frequencies Energy Content (LFE) (measures the nodes’ ability to capture the temporal variability of the observed pollutant). All the measures are then combined to an Integrated Performance sensor Index (IPI).
We showed that the SET facilitates a comprehensive across-platform analysis of the sensor performance envelope and desired working conditions. Moreover, apart from calibration needs and the benefit of (some) redundancy in WDSN data collection, we also addressed within the project the effect of airflow on the sensor readings, be it the effect of the varying wind field or of the ego-motion of mobile nodes either worn (personal nodes) or mounted on vehicles or carried while cycling (mobile nodes). The impact of sensor motion on its performance was clearly shown, and attributed mainly to the air velocity at the sensor face, since the sensors are inherently designed to measure under diffusive rather than convective regime. This has not been taken into account in nearly all previous studies that used similar nodes for measurement while on motion.
One of the main findings is that field calibration of air sensing devices improves the network performance significantly. The current achievable accuracy and repeatability of the measurements restricts the use of the devices. Especially non-professional users need to be advised about the difference between compliance monitoring and monitoring using currently available sensor platforms. The current state of this technology does not fit the requirements of regulatory agencies, as well as cannot provide reliable (absolute) exposure estimates of urban air pollution, to be used uncritically in epidemiological studies. However, there are numerous possible applications in education and in citizen science, and for raising the public awareness to air pollution as part of a citizen observatory.

Communication solutions
In order to provide a complete end-to-end solution following the concepts used in the Internet of Things domain, we developed data flow tools that allow connecting the proprietary data solutions of the sensor platform providers to a central system with dedicated routines for data processing (including potentially quality control). APIs and other tools were used for user end near-real-time visualisations, and for data downloads, and to provide data to e.g. hackatons or students who developed their own apps.
In any Citizens’ Observatory, interoperability is one of the focuses. It combines interoperability perspectives from INSPIRE information value chain, the European Interoperability Framework (EIF) which includes interoperability for the following aspects: Policy, Legal, Organisational, Semantic and Technical. CO Information Value Chain Interoperability is adapted from the INSPIRE information life cycle value chain.
For a further refinement for technical interoperability the ISO 19119 Service areas provide a good foundation with boundary services with sensors and actuators and user interfaces, processing and data management services as well as communication and workflow services. Finally, non-functional aspects such as interoperability for security and privacy were considered.
Another focus area is data flow, with a particular focus on how to set up a WFS server and the actual data collected through the CITI-SENSE project. The CITI-SENSE architecture supports multiple types of sensors and mobile apps for collecting data, and multiple ways of providing data for further use and processing, with the use of a common data model and WFS storage support for the Citizens’ Observatory data.
The Data Storage component in the SEDS Platform is implemented as a relational SQL database. The SQL query interface will be a common interface at the back end of the majority of the CITI-SENSE services. The SQL interface will not be available to external applications and external applications will communicate with the CITI-SENSE platform via open standard web services. The free and open source database technology PostgreSQL is implemented to realise the SEDS Data Repository. PostgreSQL is cross-platform and supports many different operating systems. Currently PostgreSQL is widely used across the world and underpins some large IT infrastructures.
For storing geographical data, the PostgreSQL database is extended with the PostGIS add-on. PostGIS is a free and open source software application which adds support for geographic objects to the PostgreSQL database. PostGIS follows the Simple Features for SQL specification from the Open Geospatial Consortium. PostgreSQL version 9.4.5 along with PostGIS extension version 2.1.5 was used to store and manage the SEDS backend database.
The solution adopted to create the PostgreSQL/PostGIS Relational Database has been the one provided by the Amazon Web Service. The following image shows how is connected to the different WFS services created for the communication with the actors involved in the CITI-SENSE project:
Data storage is achieved through a WFS API interface. Data from sensors not using the API directly can be stored through a file ingestion process from CSV or JSON files. A set of relevant queries on observations based on type, time and location is supported through a REST interface. The final data sets collected in the CITI-SENSE project are represented through a collection of CSV files, based on the main classes of the data model. These can be used for further analysis directly or through ingestion into a new storage system, which also has been done for the final CITI-SENSE data into a new relational data base system at the end of the project.
The model management interoperability is supported through the use of a common data model in the CITI-SENSE SEDS server. Sensor platforms interact with the WFS server – either through a push (API-based) or a pull (file-based) interface.
The life cycle approach allows to manage and support a number of different resources, including human (citizen), sensor, data and service resources. Each of these resource types will be handled through the following phases Manage resources, Observe, Publish/Discover, Compose/Visualise/Analyse and Act/Notify. The CITI-SENSE platform and architecture has mainly had a focus on the first parts of the information value chain, in particular as much of the collected data has been of more experimental nature.

Tools and services
The project relied heavily on latest technological developments in air sensing technologies (described above). Most sensing platforms were already commercially available at the start of the project, and some solutions were designed specifically for the project. Laboratory testing of the existing platforms has revealed the need for further development and adjustments, and this work continued throughout the first three years of the project, with field and user testing. These necessary developments also required the communication platform and visualisation solutions to be continuously reviewed, and user-tested. Thus, the project timeline was adjusted several times. However, the necessary set of tools and services was ready for field deployment in the last 9 months of the project, and is available from the Citizens’ Observatory Toolbox (COT) at http://co.citi-sense.eu where it can be reused by other cities in the future. A brochure is attached as Annex to this report.
These products were developed, extended or adjusted in this project to fulfil and maintain the overall chain “sensors-platform-products-users” with primary focus on the interface and link between the products and the end users. Some of our products are 3rd party properties and belong to SMEs and partners within this project. These products have been extended or updated based on our internal and external tests, or further developed according to new requirements to align with the aim of this project and to help the case studies to do successful case and field studies. In some of these cases the development code is not open for the public. This can be complete code of smartphone applications like the EXPOAPP, CIVIC-FLOW, or parts of algorithms and procedures developed. But we also have open source codes and products:
• Data visualization web page and widgets code
o Widgets code: http://citisense.u-hopper.com/
o Web page: http://srv.dunavnet.eu/new/citisense/OutdoorDataPortal/
• SENSE-IT-NOW – Cross platform smartphone application for environmental monitoring toolkit for public places
o Code: https://git.nilu.no/citi-sense/sense-it-now
• CityAir Cross platform smartphone application for collecting citizens air quality perceptions
o Code: https://git.nilu.no/citi-sense/cityair
Widgets are reusable user interface components that can be reused in future Citizens’ Observatories. The CITI-SENSE widgets are based on HTML5 to enhance the support for portability across platforms. The CITI-SENSE project has provided widgets for a number of areas including maps with sensor locations and index values, real-time and historical sensor values, physical activity level maps and graphs, thermal & acoustic measurement graphs and widgets for questionnaires.
Surveys provide a good approach for interaction with citizens. CivicFlow is a service that can support surveys both from a smart phone and a web portal. The widget framework for questionnaires by U-Hopper is the basis for interaction with the survey services. We developed three surveys to map knowledge and get feedback from users, on outdoor air quality perception, on indoor air quality in schools and on environmental quality in public spaces, for distribution as web or smartphone applications.
Mobile Apps are important for the support citizen's participation. CITI-SENSE has aimed at creating mobile apps that can run on multiple smart phone platforms through the use of frameworks like PhoneGap/Cordova and widgets supporting portability through HTML5. The SENSE-IT-NOW and CityAir App are presented as examples of the approach for the development of Mobile App apps. The CityAir app is provided from a user point of view as apps available through the Google Play store and through the Apple iPhone app store. From a developer's point of view the code is available as open source through a GitHub repository.
We developed tools for users to browse and download data from the CITI-SENSE Citizens’ Observatories in schools and in cities.
Each Citizens’ Observatory used a local web portal as a main information hub in local language. These portals can be accessed from the central web portal of the project, http://co.citi-sense.eu.
Of particular interest may be the novel mapping of air quality that requires a “base map” and sensor output.

Citizen Observatory Toolbox
The Citizens Observatory Toolbox (COT) that collects the technical elements developed in the project is now freely accessible from the portal http://co.citi-sense.eu. It has in particular a developer perspective supporting organisations that want to create a new Citizens’ Observatory. It includes software that can be used by organisations that want to create new business and services in the area of Citizens’ Observatories and Citizen Science.
The main elements of the toolbox are the following seven parts:
1) A method for Citizens’ Observatory Interoperability – a methodology approach for considering aspects of interoperability when creating a new Citizens’ Observatory.
2) Citizens’ Observatory Data Management Services - WFS Model and database setup (for a new Citizens’ Observatory) including specifications for data creation and access (WFS-T) for new sensors/apps, CSV file ingestion for third party sensors and queries and performance aspects from widgets and portals. Further, there has been a focus on support for the FAIR Data management principles of data being Findable, Accessible, Interoperable and Reusable – supported by a final CSV archive/query structure and GEOSS .
3) Sensor connections (ingestion principles) with both push and pull interfaces for the data server.
4) A survey tool for interactive questionnaires are being supported through the CivicFlow tool.
5) Mobile Apps - With a platform independent approach that is exemplified through the CityAir app example and architecture – serving also as a basis for the creation of new apps.
6) Web Portals - including a map query and CSV download support – and general guidelines for the creation of a Citizens’ Observatory Web portal
7) Widget Library containing reusable HTML5 user interface widgets - for both web portals and apps.
Further, there has been a focus on the final offerings of results from CITI-SENSE through GEOSS and a focus on the synergies between the running (pioneering) Citizens’ Observatory projects CITI-SENSE, CobWeb and WeSenseIt. There have also been interactions with the "New Citizens’ Observatory projects” SCENT, LandSense, Ground Truth 2.0 and GROW with respect to exchanging experience and technology with the new projects. CITI-SENSE has taken a responsibility in organising events and workshops for such final external activities presenting the results from the project for the ECSA workshop in Berlin in May 2016 and the GEO project workshop in Berlin in June 2016, and through a 3-day event at INSPIRE'2016 in Barcelona with a 2-day Hackathon workshop and 1 day set of sessions for both the existing "Pioneering Citizens’ Observatory projects" and the "New Citizens’ Observatory projects" and also a set of VGI projects with SDI4Apps, FOODIE and OpenTransportNet. In the Hackathon workshop there was a focus on sharing of data through common data model mappings and data set access. A successful "hack" was done, demonstrating how the collected CSV datasets in CITI-SENSE could be transformed into a new database and being used for exploratory visualisation showing the observations on a map with associated measurement data, with compatible mappings to the SensLog model of the SDI4Apps project. Resources such as the web portals from the CITI-SENSE project have been registered in GEOSS, but due to GEOSS requiring "production data", the "test data" from the CITI-SENSE project has not been registered as online data through the GEOSS registry. The CITI-SENSE data sets are now provided as a long term archive through a set of CSV data files on a FTP server hosted by NILU.

Citizens’ Observatory Portal design and implementation
The CITI-SENSE Citizens’ Observatories Web Portal (COWP, http://co.citi-sense.eu) is the main user related product. It is designed to provide access to all CITI-SENSE user and scientific products. As such, it reflects the overall structure, aims and objectives of the CITI-SENSE project and of Citizens’ Observatories. It is a contribution to empowering citizens to participate in environmental monitoring, enabling them to observe and understand environmentally related problems, as well as report and comment on events or issues of importance. In particular, it provides access to Products and Services created in the project.
The portal has several key sections, including ‘The Citizens’ Observatory Toolbox (COT)’ and ‘Use Examples’. Both these sections contain information and links on the project outcomes as well as current implementations of the various applications and services developed during the lifetime of the project.
The key principle to the CITI-SENSE COWP is that it provides a ‘one stop’ access to products and tools of the project, making finding relevant information or services simple/easy for the end users and other stakeholders. The COWP will remain active and available to all even after the end of the CITI-SENSE project. The consortium is exploring the potential ways to keep the COWP sustainable.

Internal and external evaluations of tools and services
Internal and user evaluations formed the basis for improvements both, of the project processes and of the tools and methods that we were developing.
Comprehensive internal evaluations were done as the last part of the pilot studies. A set of key performance indicators was developed (as described e.g. in Annex to Deliverable D2.4) and applied to all case studies. The results were used to prioritize the work prior to the main field study, the so-called “full deployment”.
At some point it became obvious that the tools and methods that CITI-SENSE was presenting to users also would benefit from user evaluation. From the KPIs assessment results, we can conclude that developers gave a relatively higher score for the methodologies they developed than the project internal methodologies users did. This may indicate that from the methodology developers’ point of view, the methodologies they developed achieved good or quite effective success toward their planned objective. However, from the users’ perspective, for several of the methodologies, there was lack of enough user involvement in the methodologies development process. From this perspective, several of the methodologies attained a fair or relatively low level of the project’s defined objectives.
To average both the developers and users evaluations, we can conclude that:
Sensors and Sensor Platforms: LEO, Obeo radon sensor and Atmospheric sensor package have fair or relatively low achievement of the project objective defined, AQMesh has reached moderate or quite effective success, Environmental Monitoring Toolkit in Public Spaces reached good or effective success, SEDS platform has reached moderate or quite effective success;
Surveys: both CityAir App and the Online Air Quality Perception Questionnaire have achieved good or quite effective success;
Web portals and Maps: the Data visualization Web Portal for Public Spaces Empowerment Initiatives achieved very good or very effective success, the Data Visualization Web Portal for Outdoor Air Quality and the Data Download Web Page have moderate or quite effective success, the Citizens’ Observatories Web Portal has reached moderate or quite effective success, and the Data Fusion Maps reached good or effective success.
Among the users who have evaluated one or several of the CITI-SENSE methodologies: (i) there are more male than female; (ii) more people are in the middle age (25-54) than in the younger age (18-24) and older age (+65); (iii) most are highly educated either with Ph.D or master’s degree, less are relatively lower educated either with bachelor or secondary school degree; (iv) more people are from Oslo than other cities in CITI-SENSE; (v) most work and/or live in the city, less are students and/or visit the city occasionally in which they have evaluated the CITI-SENSE methodologies; (vi) most are interested in or very interested in air quality related issues in general, and less are somehow interested in air quality issues, and no one is not at all interested in air quality issues.
From the “usability” evaluation results related to the users’ feedback about the user friendliness of the methodologies, more participants (above 50% of the total participants) indicated that they:
• would not use the LEO frequently, can use it without the support from a technical person, did not need to learn a lot of things before they could get going with it, have learned something useful by using it, may not recommend it to their friends and/or family, did see the great potential to use the it in the future, and have not used it many times;
• thought the CityAir App is quite simple and easy to use, they can use it without the support form a technical person, thought the various function in it were well integrated and there was not so much inconsistency in it, thought it is easy to learn how to use the it, felt very confident using it, did not need to learn a lot of things before they could get going with it, did see the potentials to use it in the future;
• would not use the Online Air Quality Perception Questionnaire frequently; they thought it is quite simple and easy to use, they can use it without the support from a technical person, thought the various function in it were well integrated and were not too much inconsistency in it, thought it is easy to learn how to use it, felt very confident using it, did not need to learn a lot of things before they could get going with it, were satisfied with it as a whole, and have not used it many times;
• would like to use the Environmental Monitoring Toolkit in Public Spaces frequently, thought it is quite simple and easy to use, can use it without the support from a technical person, thought the various function in it were well integrated and were not too much inconsistency in it, thought it is not easy to learn how to use it, felt very confident using it, have learned something useful by using it, did see the potentials to use it in the future, were satisfied with it as a whole, have not used it many times;
• thought the Data Visualization Web Portal for Outdoor Air Quality is quite simple, can use it without the support from a technical person, thought the various function in it were well integrated and were not too much inconsistency in it, and did not need to learn a lot of things before they could get going with it;
• can use the Citizens’ Observatories Web Portal without the support from a technical person, thought there were good consistency in it, have learned something useful by using it, did see the potential to use it in the future, and have not used it so often.

Reaching audiences
From the start of the project, we planned significant efforts to reach internal audiences (our volunteers and stakeholders) as well as external interested parties.
The teams in each city have engaged the public, and potential participants, in a number of different ways; through help of partner organisations such as NGOs, by media, through participation in seminars, workshops and conferences, and by social media. Where necessary, materials were prepared in national languages. These efforts were coordinated within the EIs and supported by the dissemination and coordination teams.
The final dissemination of the project was done in a “distributed” way instead through one dedicated event. This has given partners the opportunity to address varying professional and lay audiences, and also allowed to communicate in local languages when the target group required that.

Lessons learned
A project can always be seen as a learning process, and it is important that the lessons derived are not lost. Many of these are not new or ground-breaking but prove to be essential for success of any project that aims to provide value to its stakeholders, such as the public.
The first key message is “to match expectations”. A successful project needs to understand well what actors both within and outside the project (citizens, local authorities or other interested parties, but also the project partners) expect from the project, and what motivates their involvement. A continuous dialogue on how these expectations match what the project will deliver, and what are the technology options, is necessary.
An important aspect in projects involving citizens and aiming to serve them is to engage with people from the very beginning. Maximizing the use of feedback from real-world users is a main component that defines success. Knowing the users’ interests, preferences and abilities will lead to solutions fit for purpose and should always be valued in the design and implementation phases. The leading question that has to be answered in the citizens participation process is "What's in it for me?", what the project will give back and why one should be interested to join. This requires understanding of the audience, who is it for and who will respond to it. After the initial engagement, on-going motivation is crucial for the success of the project and its sustainability.
The experience with technology did not always bring the anticipated outcomes. The lack of maturity of low cost sensors, the user unfriendliness, electronics competence was critical for the project development and time consuming to make things work. Another big part of the technology was the data quality that can seriously undermine the whole project as people required firmer information to "be empowered" and act. The simplicity in tools, sensors and other means of interaction proved to be a key element to facilitate and engage people to return in those tools. On the other hand, the project itself moved the sensor technology forward and updated existing tools. And when things did not go as planned, creativity and teamwork led to desired results by using other means for engagement, data collection and other methods/tools.
Even though the project was comprised by a big number of partners, the teamwork and collaboration functioned well. The interdisciplinary team worked in most cases in favour of the project objectives and goals; however, there is available room for improvements in transparency, communication, information sharing, and time efficiency.
Overall, people within and outside of the consortium acquired a better understanding on environmental monitoring and air quality issues in cities. More technical oriented people recognised the value of the social science and the efforts to truly engage people. Cooperation between scientists with different background was at some points time-consuming and a bit frustrating, but more importantly very rewarding for individuals and the project as whole.
Recommendations for similar projects in the future
For (research) organizations who are planning to start a similar project in the (nearby) future, the following recommendations can be made (based on our experiences and on the evaluation of the project and the tools by participants involved):
1. Put the principles of “co-design” and “co-development” at the core of these projects.
2. Make sure right from the start that all consortium-partners are aware of the best-practices that need to be shared in order to maximize success.
3. Set up your project in such a way that it can become a real “learning organization”.
4. Make sure that every project-partner takes care of “expectation management”.
5. Take into account possible barriers for implementation which can reduce the outcomes and final impact of the project.
6. Make sure that the EU-goals for “Responsible Research & Innovation” (RRI) are taken seriously by all project-partners involved.
7. Double-check also during the research process if all project goals are still really shared.
8. Double-check regularly if the consortium is really balanced enough to deliver on all the necessary aspects of this research (including the social aspects).
Particular recommendations for the EU are:
1. Put the principles of “co-design” and “co-development” at the core of these projects.
2. Make sure that every project-proposal takes care of “expectation management”.

Potential Impact:
Potential impact, main dissemination activities and exploitation of results (10 p)
Expected impacts listed in the work program
The outcomes of CITI-SENSE addressed the call requirement for “new and innovative environmental monitoring and information capabilities for effective participation by citizens in environmental governance.”
Expected impacts listed in the work program and the corresponding results from CITI-SENSE project:
EU FP7 work programme CITI-SENSE
Boost European competitiveness Promotion of novel applications and tools (tools and services developed by CITI-SENSE) for improved use of environmental resources. In particular, we have:
- improved sensor platforms for monitoring environmental quality from: Alphasense (Atmospheric Sensors), Ateknea, Geotech (and third party EI), Dunavnet, JSI, Obeo, Tecnalia, QU
- improved information and communication services provided by U-Hopper, Snowflake, Sensing&Control, Dunavnet
- Increased knowledge of performance of low-cost sensor platforms for air monitoring and contributed to development of quality assurance and control systems, especially relevant to partners NILU, Technion, Tecnalia and UCAM
- Provided new best practices for public involvement relevant to environmental decision making, by improving methodological basis and improving services rendered by SMEs GAC, IBATUZ, IOM, NAAF and PVDH
Stimulate participation of SMEs 15 SMEs participated actively in CITI-SENSE.
AirBase, Alphasense, Ateknea, Dunavnet, EI, Geotech, Obeo could demonstrate their sensing technologies and were able to develop them further to obtain improved versions of the ones tested in the project.
GAC, IBATUZ, IOM, NAAF and PVDH participated in implementation of the case studies and in development of methods for participant involvement, stakeholder communication and citizen empowerment.
Dunavnet, Sensing & Control, Snowflake and U-hopper have improved their ICT services.
Provide support for evidence-based decision making CITI-SENSE has contributed by technological improvements for low-cost sensing of the environment, and by demonstrating the concept of Citizens’ Observatories. We developed a number of tools and services to complement the institutionalized environmental data and information provisioning, giving thus citizens possibilities to generate and contribute to the monitoring systems with information of relevance to them personally.
Strengthen the GEOSS initiative CITI-SENSE partners were active in standardization activities (OCG), and participated in a number of activities such as Implementation Pilots and GEO and GEOSS/INSPIRE meetings and working groups. Many of these activities were done in collaboration with the other citizen observatories funded from the same call; this critical mass undoubtedly contributed to strengthening the relevant activities.
CITI-SENSE provided also datasets to GEOSS; close cooperation with GEOSS was mainly realized through attendance of several workshops and demonstrating CITI-SENSE tools and services in workshops/hackathons.

Expected impacts specific to the project
The CITI-SENSE project aimed to empower citizens in the EU and partner countries in two ways:
(i) by giving them unprecedented effective participation tools in the EU’s environmental governance
(ii) by providing them with unprecedented quantitative environmental information at citizen level, including quantitative estimates of the errors in this information.
To achieve these outcomes, CITI-SENSE realized the chain “sensors-platform-products-users”, linking information producers and information users. This had two components, technological and social.
Technological component was realized by developing and deploying:
• new and innovative microsensor technologies to measure environmental status
• innovative network / platform capability to distribute and visualize environmental information
• innovative mobile phone and web technology to realize effective, flexible, transparent and user-friendly two-way transfer of information between information providers and citizens
• state-of-the-art data assimilation and data fusion techniques to add value to the environmental information provided.

Activities that were carried out to achieve expected impacts and their realization in CITI-SENSE
Proposed activities Their realization
Empowering citizens and citizen’s associations with a “citizens’ observatory” Establishment of 24 COs in Europe; provision of tools and services for observing their immediate environment; wide public and school engagement; engagement with decision-makers
Developing innovative tools based on microsensor technology to process environmental information from static and mobile platforms CITI-SENSE has improved a number of sensing platforms based on low-cost micro-sensor technologies, for air quality, noise, thermal comfort and radon concentrations. In addition, CITI-SENSE has implemented at that point, the largest urban sensing network for air quality consisting of 334 sensor devices distributed over eight European cities; data from this network were visualized in near-real time using innovative visualization tools. Uncertainties of the measurements were addressed in the visualization as well as in the tools, improving understanding of monitoring technologies and their outcomes.
Assessing the quality of data gathered by personal instrumentation, addressing needs for calibration, and sensor replacement and modification Dedicated tasks were carried out regarding quality assurance and control. Instrumentation (including personal monitors) was tested in the laboratory and in the field, and the results were made available to the owners of the platforms (as a service to improve their performance), reported in relevant forums (such as similar projects, EU COST action EuNetAir, and in a relevant CEN standardization working group). Results of field comparisons between the low-cost technologies and standardized reference instrumentation were used for calibration of the low-cost devices, and selected results are submitted for publication. A method was developed and published (“Sensor Evaluation Toolbox, SET).
Developing a platform to provide a flexible, transparent, effective and user-friendly information and service chain between providers and end users of environmental information A data model and data flow was realized, enabling harvesting of heterogeneous data, and enabling access to the information using several methods, including APIs. All methods are freely available from the Citizens’ Observatories Toolbox, http://co.citi-sense.eu. The data were also successfully provided to third parties, e.g. hackathons and students who developed their own mobile apps based on data generated from CITI-SENSE.
Providing decision makers with models that facilitate connections between environmental governance, global policy objectives and citizens’ needs In designing and implementing the individual Citizens’ Observatories, the team has successfully liaised with local authorities in the nine areas; they became part of the team and in this way, actively interacted with citizens and citizens’ organizations. For the most, the Citizens’ Observatories model was seen as positive.

Scientific impacts
Air pollution is defined by the atmospheric composition of gases and particulates near the Earth’s surface. It is modified by meteorological conditions and modifies the outdoor and indoor human environment. It impacts human society, as high concentrations of pollutants can have adverse effects on human health, and prevention of such impacts is the motivation for e.g. European legislation. The ability to monitor air pollution, and the factors that modify it and/or are modified by it, is thus crucial for society. Currently, long-established air pollution monitoring systems are based on networks of ground-based stations (e.g. as part of the European EMEP database). The capabilities of satellites to monitor air pollution and the potential for combining ground-based and satellite data are being explored. For noise, models are successfully used to establish exposure levels. Radon is monitored e.g. in buildings in areas suspected of or with known high natural background levels.
As a result of the technologies used, available budgets and support systems, there are data gaps at the spatio-temporal scales associated with an individual citizen. These data gaps have led to critical barriers to successful development of key research issues in air pollution studies. These include, among others :
1. Human exposure. Currently most epidemiological studies base their results on averaged air pollution data, and estimate the human exposure on home postcode, despite the fact that many people spend large parts of the day at other locations or moving through the urban environment;
2. Model validation. Conventional data sources on air pollution (ground-based networks, satellites) do not provide sufficient detail of resolution to enable current models for traffic, vehicle emissions and pollution dispersion to be validated at the spatio-temporal scales reflecting citizen activity, e.g. at street level;
3. Urban pollution management. Lack of observational information at spatio-temporal scales needed to reflect citizen activity and validate models representing air pollution at these scales has implications for the design of strategies to manage urban pollution. Strategy elements which would benefit from this information include implementation of actions to reduce pollutant concentrations in critical areas, and development of techniques to perform real-time monitoring at street level, with potential extension to short-term forecasting to be done using air pollution (and other) models initialized by data provided by CITI-SENSE.
Integration of activities in the chain “sensors-platform-products-users” using the concept of the citizens’ observatories demonstrated the ability to provide key environmental information using new micro-sensor based technologies made into products that can be used by the members of the public. In order for the public to fully draw benefits of such devices, a number of practical issues have to be solved (e.g. easiness of operation of the device, easy access to own measurements, reliable communication of the device with repositories and visualisation tools, access to troubleshooting and maintenance). CITI-SENSE has successfully solved most of the practical issues. By operating a network for a period of several months (and in some localities, over a year), we have
- provided an operational benchmark for future providers
- demonstrated capabilities of a wireless sensor network for air quality
- demonstrated how the data from the sensor platforms can be used, taking into account their properties such as uncertainty, precision and accuracy
- developed scientific methods to support assessment of the sensor platform performance
- developed modelling tools that take advantage of the sensor outputs
- developed novel visualisations that provide the end user with comprehensive information at a glance.
In addition to the technological perspective, Citizen’s Observatories are based on involving the public. In CITI-SENSE, we have developed scientific methods that enabled the technological teams in the locations to engage with citizens and stakeholders in an efficient way, and we have also been able to evaluate these activities. As the ultimate aim of CITI-SENSE was empowerment, we have also carried out an analysis of empowerment potential of the technological tools and services. This body of evidence is now freely accessible to the public in our scientific outputs including publicly open deliverables.
The CITI-SENSE project helped to improve our knowledge of how air pollution affects and is perceived, the individual citizen and, by extension, society. Application of the concept of Citizen’s Observatories will enhance the scientific and technological opportunities to understand air pollution and develop the capability to monitor it.
Commercial impacts
CITI-SENSE focuses on removal of technology and knowledge barriers for the acquisition and exploitation of citizen sourced environmental data. This is particularly important for Small & Medium-size Enterprises (SMEs) where data acquisition cost and service provision cost are often a hindrance in the commercial exploitation of environmental data. Microsensor technology linked to sophisticated networks will increasingly be used to provide two-way environmental information between data providers and the individual citizen, and contribute to empowering the citizen. Participation of ten technological SMEs in CITI-SENSE opened up opportunities to develop and exploit the next generation of microsensor technology, building on the experience and knowledge gained through the CITI-SENSE empowerment initiatives. In addition to microsensor technology, CITI-SENSE developed a methodology and a platform of reusable services from which SMEs and commercial organisations can innovate and commercially exploit citizen empowered environmental data.
Importantly, the open access to a number of our results and findings through the CITI-SENSE Citizen’s Observatory Toolbox platform creates a ‘level playing field’ in which SMEs do not have to invest in enterprise infrastructure, thus removing ‘go to market’ barriers and enabling SMEs to focus on end user innovation and value add application development.
The outcomes of CITI-SENSE provided an opportunity for the participating technological SMEs to further develop their products and to learn how to link to an open infrastructure providing environmental information to the individual citizen. Beyond CITI-SENSE, SMEs will be able to use the experience and knowledge gained to stay at the forefront of world microsensor technology, application development and innovative service delivery, with benefit to EU business and employment opportunities.
The SMEs that have their main business area oriented to provide social sciences related services have also benefited from CITI-SENSE; they have gained further methodological improvements for engagement of stakeholders and towards empowerment of citizens, and will thus be better able to compete in the future.
Policy impacts.
Under European air pollution (or, alternatively, air quality, AQ) legislation, most local authorities make use of air pollution monitoring stations to provide daily public air pollution information via the Internet. The legislation assumes comparability of measurements across Europe, and for that reason, requires standardized methods and comprehensive quality control and quality assurance systems. There are disparities and missing areas of data coverage across Europe especially in areas that are not required to monitor, sometimes in industrial hot spots. In other cases, the information is difficult to access and understand by the citizen. As a result, citizens often do not have a clear understanding of how and where air pollution is measured in their locality, or where they can find this information, and although information is provided, the citizen is not always able to make best use of it.
Information on key factors such as pollution concentration or individual human exposure has traditionally not been available or only been available at high levels of spatio-temporal aggregation, which average out local variations. Furthermore, permanent monitoring stations (e.g. from ground-based networks) are located to measure ambient background concentrations or at “hot-spot” locations that do not represent individual exposure to air pollutants. As a result, little or no data are available at the level of individual exposure to pollutants. This affects regulatory obligations, especially in terms of demonstrating compliance with existing and future directives and/or legislation.
Currently, environmental information for end users (e.g. citizens) is generally provided on the “push service” principle, where an end user subscribes to various information channels. Whenever new content is available on one of these channels, the provider pushes that information out to the end user. This method does not work to the benefit of the citizen, as it does not allow the end user (citizen) to interact with the provider and, thus, does not empower the citizen to influence decisions made by policy makers regarding his/her environment.
CITI-SENSE has developed and evaluated some of the mechanisms needed to implement the two-way interaction between policy makers and citizens that is fundamental to empower the citizen and allow him/her to influence environmental governance.
Urban impacts
Fifty percent of the world population lives in a city and between 2010 and 2050 the urban population will almost double, and reach over 70% of all population. Cities occupy 2% of the world’s geography but account for 75% of the world’s greenhouse gas emissions. Associated with this situation, in 2014 there were 1.2 billion cars on the road, a ratio of 1 car for every 6 people. To address these issues, the concept of the smart city, which builds on a ubiquitous high-speed internet infrastructure, is rapidly becoming mainstream within the EU.
CITI-SENSE contributes towards the infrastructure needed to design smart cities. The practical experiences with the development and operation of the sensor network and user-end products will further contribute to better design elements of the smart cities infrastructure. We have also designed and evaluated tools and services that can directly lead to improvement of urban environment where the citizen is empowered and benefits from timely, tailor-made environmental information, and influences the development of the urban landscape.
Societal impacts.
The proliferation of low-cost sensor platforms is central to the concept of the citizens’ observatories. It has a number of highly desirable features as it offers citizens the following: an active experience; an instantaneous link with his/her immediate environment; an instantaneous link with the immediate environment of other citizens; active participation in the chain “sensors-platform-products-users”; a mechanism to inform policy makers of their own immediate environment; and a mechanism to inform policy makers of his/her needs. As a result, the mobile sensor promotes curiosity, reflection, learning, and awareness of a user’s immediate environment and can influence their actions in response to changes in this environment. Furthermore, by providing environmental information to the citizen, the mobile sensor helps improve understanding of the impact of the environment on the citizen; helps improve the “social conscience” of the citizen, including the understanding of the relationship between the environment, the citizen and society; and helps provide the citizen with an increased feeling of participation in society and the decision-making process.
The concept of Citizen’s Observatories, that combines the technological and the social and societal aspects in one system, has been shown to have the potential to become a powerful social innovation. Our case studies/empowerment initiatives have significantly increased understanding the motivations of the participants, their knowledge basis, expectations and requirements, and have provided tools that can be used by anyone for the purpose of understanding their own local communities and how they can become an actor in environmental decision making. CITI-SENSE thus provided an opportunity to gain insight into how to empower the citizen, and by extension society, to participate fully in the decision making process concerning environmental governance.
Standards impacts.
The CITI-SENSE consortium has in practice cooperated with relevant European or other standardization bodies in order to guarantee that the project builds upon existing standards and contributes to emerging standards and specifications. This was done in two areas, the ICT and the sensor technologies for environmental monitoring.
CITI-SENSE has developed modular, extensible solutions, building on top of standards whenever possible. Where existing standards proved to be insufficient, CITI-SENSE contributed to their extension. The consortium members contributed to different task groups for establishing standards, working groups, and special interest groups. The goal was to bring the technological and organizational solutions developed and successfully applied in CITI-SENSE towards standardization. In this way, the results and experiences gained from the RTD efforts provided input to these standardization efforts.
The following groups were relevant:
• ISO (International Organization for Standardization): (1) TC211 (Geographic Information/Geomatics - http://www.isotc211.org ) : ISO 19154 Geographic information - Ubiquitous public access - Reference model (KICT is the current project leader); and ISO 19119 Geographic information - Services (SINTEF is the current project leader);
• OGC (Open Geospatial Consortium - http://www.opengeospatial.org) : OGC GeoSPARQL (A Geographic Query Language for RDF Data); OGC CityGML (An open information model and XML-based encoding for the representation, storage, and exchange of virtual 3D city models); OGC indoorML (A space-event model and XML-based encoding for indoor navigation that simultaneously addresses routing planning, multiple localization methods, navigation contexts, and different locomotion types); OGC Topic 12 (The OpenGIS Service Architecture OGC Abstract Specification)
• CEN/TC 287 (http://www.centc287.org/ ) – currently in the process of revising TR15449 Geographic information — Standards, specifications, technical reports and guidelines, required to implement Spatial Data Infrastructures
• BuildingSMART International (http://www.buildingsmart.com/) : Building SMART Korea, and cooperation with TC211 and TC59/SC13 for developing new standard project about BIM-GIS
• W3C (World Wide Web Consortium - http://www.w3.org/ ) : SSN (result of the W3C Semantic Sensor Network Incubator Group, which may reopen soon as a working group)
• CEN/TC 264/WG 42 - a protocol of evaluation of sensor is being developed.
CITI-SENSE provided an opportunity to contribute to standardization activities, with a particular focus on standardization of geospatial data and services.
Need for a European approach.
CITI-SENSE addresses the empowerment of the citizen to participate in environmental governance. It has a European dimension: the environment respects no borders; environmental conditions and experiences vary across Europe; European environment policy is done at the EC level.
The geographic coverage of CITI-SENSE, from North to South and East to West, with member states with different length of membership in the EU, candidate countries and associated states, provided a variety of social and geographical settings for the case studies. This has allowed the team to better understand both, the limits of the technologies (tested in varying environments), differences in institutional settings, and the differences in attitudes, approaches, knowledge and interest of citizens of different countries. These experiences were valuable for developing the Citizen’s Observatories concepts and underlying methods.
CITI-SENSE provides the framework for the European approach needed to address empowerment of the citizen to participate in environmental governance.
Motivation for EU – South Korea collaboration.
With the inclusion of KICT and SALTLUX in the consortium, CITI-SENSE realized the intentions in the Agreement on Scientific and Technological Cooperation between the European Community and the Government of the Republic of Korea . Ubiquitous City (U-City) is a Korean brand of the smart city and KICT and SALTLUX are important technology contributors to this research area in Korea. Over the last two decades the Republic of Korea has continuously developed strategies for knowledge-based and sustainable urban development by incorporating ICT and involving citizens. According to U-City law established in 2008, the U-city concept aims on creating a built environment “where any citizen can get any services anywhere and anytime through any ICT devices”. Through KICT and SALTLUX integration in the project, CITI-SENSE, and implicitly the European community, had access to the latest developments in Korea in this research area, and has also contributed to development of air quality related services and thus proliferation of European know-how to South Korea.
External factors affecting impacts.
A number of external factors affect achievement of impacts, and did occur also in CITI-SENSE:
(i) Participation of the public. Difficulty in convincing citizens (or citizen groups) to participate in the concept of the Citizen’s Observatories;
(ii) Difficulty in convincing policy makers at various levels (European, national, local) of the merits of the concept of the Citizen’s Observatories;
(iii) Difficulties in addressing issues of privacy;
(iv) Shortcomings in measurement capability of microsensors; and
(v) Shortcomings in the robustness and characteristics of the chain “sensors-platform-products-users”.
CITI-SENSE addressed all these factors during the cyclical course of the project. Internal evaluations and increased level of communication in the consortium were helpful in finding solutions, but some issues, such as performance of the sensor platforms, remained throughout the project.

It is perhaps natural that novel technologies are accompanied by expectations of their performance that are difficult to meet, and the consortium took a number of corrective measures (improvement of the platforms, re-orientation to other “of the shelf” solutions, re-orientation towards non-technological instruments). Increased understanding of determinants and barriers to public participation and to the participation of policy makers was met by dedicated campaigns and dissemination activities. Internal and external evaluations and openness of the capabilities and shortcomings of our solutions has also made it possible to successfully complete the project.
Exploitation
By help of a business plan (D9.15) we have identified 17 knowledge items that were considered “exploitable”, i.e. presenting a clear and significant potential for exploitation (both in terms of commercial activities as well as further research ones), leading to the following set of exploitable knowledge items (EKIs). Details are provided in D9.15 and in part B of this report.
• Platform for civic engagement through multichannel surveys – CIVICFLOW
• IoT platform for remote sensing – EkoBus
• Mobile phone app for the collection and visualization of environmental monitoring data
• Web portals for Citizen’s Observatories
• Methodology and toolkit for citizen engagement and communication with stakeholders
• Protocol for validation of sensor data
• Extension of a home automation platform, integration with environmental sensors
• Personal Air Monitoring Toolkit
• Spatial and Environmental Data Services (SEDS) Platform
• Methodology and algorithms for calibration and operation of sensors
• An ontology for Citizen’s Observatories
• Program for applying the indicator of thermal comfort
• A method for computing indicators of acoustic comfort
• A questionnaire on the safety of public places
• Facility for validation of air quality sensors in the field
• Visualization widgets for Citizen’s Observatories
• Radon sensors for remote monitoring.

List of Websites:
www.citi-sense.eu - official project web site
www.co.citi-sense.eu - web portal with tools, services, documents and other relevant information about Citziens' Observatories in CITI-SENSE and for those that want to start a CO. Main contact: senior scientist Hai-Ying Liu from NILU (hyl@nilu.no)

Final Report Summary - CITI-SENSE (Development of sensor-based Citizens' Observatory Community for improving quality of life in cities)

Share this page

Download