Variety, Veracity, VaLue: Handling the Multiplicity of Urban Sensors

Project Information

VaVeL

Grant agreement ID: 688380

DOI

10.3030/688380

Project closed

EC signature date 4 November 2015

Start date 1 December 2015

End date 30 November 2018

Funded under

• INDUSTRIAL LEADERSHIP - Leadership in enabling and industrial technologies - Information and Communication Technologies (ICT)

Total cost

€ 3 999 668,75

EU contribution

€ 3 999 668,75

Coordinated by

ETHNIKO KAI KAPODISTRIAKO PANEPISTIMIO ATHINON
Greece

Periodic Reporting for period 2 - VaVeL (Variety, Veracity, VaLue: Handling the Multiplicity of Urban Sensors)

Reporting period: 2017-06-01 to 2018-11-30

Summary of the context and overall objectives of the project

Technologies that discover knowledge from urban data, such as new techniques for the detection of disastrous events, tracking health issues, monitoring crucial environmental factors, or improving energy efficiency, will have impact in a lot of aspects of the citizens everyday life. The toughest obstacles in using urban data are that such data are heterogeneous, noisy, and unlabeled. In addition, urban data can include massive and high-speed data streams (for example video feeds). Our goal is to provide the tools that will enable the exploitation of big urban data in novel applications that focus on citizen needs and improve the quality of urban life.

The main objectives of VaVeL were the following:
- Objective 1 (Functionality): Address the most critical issues of current (big) data management and stream frameworks regarding coping with emerging urban sensor data.
- Objective 2 (Accessibility): Make European urban data more accessible and easier to utilize by providing a set of black box reusable components that provide access and analyze urban data.
- Objective 3 (Value): Enhance European industries involved in Big Data management and analytics by addressing critical issues and disseminate and exploit knowledge emerged from the project.

The project delivered one final prototype to each one of the cities that participated in the project as an end user. The VaVeL system, installed at the premises of Dublin City and Warsaw City is a fully operational smart city monitoring system focusing on dealing with emergencies, improving public transport as well as providing better services to the citizens. The two cities have evaluated the system and provided feedback for improvement. Both of the cities agreed on the value of VaVeL for managing and getting value out of their data. The prototypes will continue to run after the completion of the project while the two cities have expressed strong interest in continuing research and in developing and deploying product solutions in collaboration with VaVeL partners.

The knowledge and expertise obtained from the project is directly applicable not only to other cities with similar infrastructure, but also to additional applications like environmental modeling, event management, city security, and urban planning. Moreover, the technologies have been demonstrated to scale and allow monitoring situations in a country-wide scale.

Work performed from the beginning of the project to the end of the period covered by the report and main results achieved so far

- Research Advancements: The consortium delivered many state-of-the-art techniques, reflected in the 93 publications (42 in the secnd period), the invited talks, and the 6 awards.
- Data Availability and Quality: The consortium focused on that and managed to overcome most of the technical and legal obstacles to make easily accessible multiple sources: traffic volume information from sensors, BUS GPS real time locations, Twitter messages, Tram location information, Crowdsourcing data, weather and pollution data, and CCTV data. The availability of video data was a significant achievement since there were multiple legal and technical difficulties that had to be overcome.
- Exploitation: A detailed plan has been established, including the development of the exploitation committee and an extensive market research.
- Dissemination: With its website, flyer, social media activity, and participation in multiple events and fora, including organizing the MUD workshop at SIGKDD 2018, the VaVeL team had an substantial outreach outcome.
- Development of Realistic Applications. The consortium has made significant progress working in close collaboration with the use partners: Dublin City and the City of Warsaw. Together with the research and industry partners, they developed a detailed set of requirements, resolved data issues, designed the architecture, and built two working prototypes. The two use cases we have developed are complementary and serve as the vehicle to first understand the needs of users and state-holders and subsequently evaluate the proposed solutions in a realistic environment.
- Collaboration: Collaboration among partners were strong during the first period of the project.

Progress beyond the state of the art and expected potential impact (including the socio-economic impact and the wider societal implications of the project so far)

The consortium advanced the state-of-the art in distributed computer systems, data mining and machine learning, and Smart City applications with novel techniques that: i) Enable communication and computation efficient monitoring of distributed streams; ii) automatically balance the distributed stream processing system’s load; iii) determine whether and how compression should be applied and minimize the overhead in a Distributed Stream Processing Stream; iv) balance the trade-off between makespan and budget in a MapReduce center and provide cost efficient scheduling of MapReduce workloads across multiple clusters; v) enable reusable elements in situation of multi-level information availability; vi) build and update digital maps from GPS trajectories and identify frequently traveled sub-trajectories; vii) preserve user location and data privacy; viii) provide traffic modeling, route planning, delay prediction, and pollution estimation in urban areas; ix) estimate traffic density and identify emergency vehicles in real-time using video data; x) understand the reachability provided by public transportation in cities; xi) offer context aware services that inform citizens.

Impact.
a. Business and Innovation. The evaluation of pilots validated our approach and demonstrated the demand for our techniques. Through the publication of new open data sources, and through the use of hackathons we promoted the use of open urban data for new applications, thus fostering Smart City Entrepreneurship.
b. Research and Academia. We are confident that the work currently developed by the VaVeL team represents the leading edge on what can be achieved by leveraging both access to real time data and analysis of historical data on traffic related applications. VaVeL partners recognized early on that the next step in advancing the state of the art is to effectively combine both historical and real-time data to provide more accurate longer term analytics to the user. Throughout the duration of the project new technology was created for the industry partners and several students have being trained through their involvement in the research challenges of the project.
c. Social and Environmental Impact. The focus of the work has been to develop techniques that benefit the cities and citizens. Novel analysis techniques that can enhance the situation awareness of the authorities; Context-aware services that can aid in the improvement of the citizens’ daily life in the urban environment; Methods that can predict and estimate pollution. The ultimate result of our efforts is to provide novel tools and techniques for city understanding and eventually improve the city environment, and the quality of life of the citizens.
In addition the novel distributed and real-time data analysis techniques that the consortium introduced are general and powerful enough to enable the development of powerful systems for a wide area of applications, from monitoring and predicting pollution and environmental impact in a country-wide scale to effectively handling massive streaming data sources (such as multiple video streams).
d. Impact to Europe. The technologies developed in VaVeL have been designed to be applicable in many situations and cities with minimum adaptation costs, thus facilitating their deployment in other municipalities and contexts.

Picture of the Warsaw City trip planner application.

Picture of the prototype for Dublin City running.

Vavel dissemination results.

The two cities involved in the use cases.