Skip to main content

Virtual Centre of Excellence for Ethically-guided and Privacy-respecting Video Analytics

Final Report Summary - VIDEOSENSE (Virtual Centre of Excellence for Ethically-guided and Privacy-respecting Video Analytics)

Executive Summary:
Summary description of project context and objectives

VideoSense integrated leading European research groups to create a long-term open integration of critical mass in the twin areas of Ethically-Guided, and, Privacy Preserving Video-Analytics where the advent of new data intelligence technologies against the background of dynamic societal and citizens‘ goals, norms, expectations, safety and security needs have all contributed to a complex interplay of influences which deserve in-depth study and solution seeking in order for the European society, citizen and industry to strike an optimal balance in resolution of the various challenges in this arena.
The VideoSense Virtual Centre of Excellence has played a significant role in bringing together a critical mass of leading experts and resources that have fostered major advances in the domain of ethically- aware data and video-analytics with a synergic and integrated approach. VideoSense efforts will fill capability gaps and provide clear added-value to security needs both from the technical perspective as well as from the ethical and regulatory one
VideoSense Project Actual Achieved Results

Respect for civil liberties and privacy were the guiding principles as reflected in the actual delivered results including the VideoSense Demonstrator as showcased at the IEEE DSP, Jakarta 16th September 2014. VideoSense results have filled socio-ethical and technical gaps in video-analytics and provided clear added-value to privacy by design for security surveillance through establishing:

• VideoSense Centre of Excellence (VCE, see; for integration of existing researchers to support socially responsible privacy preserving analytics solutions for security needs.
• Standard framework for Ethical Compliance Audit Management bas and associated Training and Certification as a service and a VCE revenue stream
• Framework to support various standardisation stakeholders e.g. JPEG and MPEG communities
• Organised over 20 research exchanges at senior research leader, post-doc and internships levels with significant participation of female researcher at all levels
• Summer and Winter Schools training for socio-ethical privacy preserving system development and Dual-Use/Malevolent-Use risk analysis
• A number of Privacy Filtering Grand Challenges and related databases under the MediaEval 2012-2015
• Joint Activities of Research as “mini projects” leading to a significant number of Video-Analytics and Evaluation innovations as reported in project deliverables

Project Context and Objectives:
Overall VideoSense VCE through delivering its role as a community builder-leader in methodologically-guided socio-ethical privacy-preserving video-analytics has widened and deepened the European and international integration of research in Privacy Preserving Video and Audio Analytics thus setting stronger foundations that have endured beyond the lifecycle of the VideoSense project funding. Significant VideoSense outreach efforts disseminated information about the opportunities available for joint research activities appropriately supported through exchange visits; in particular, for researchers with excellent potential for synergic work from throughout Europe and internationally.

Accordingly VideoSense VCE organized over 20 such exchanges at senior research leader, post-doc and internships levels with significant participation of female research leaders and research students and held 3 summer schools and one winter school as well as a number of training sessions for both internal and external trainees on the socio-ethical privacy preserving system development and operational deployment of surveillance systems including further training sessions on Dual-Use/Malevolent-Use risk analysis; the proceedings have included both demonstration session as well as a young researcher’s Poster session.

Furthermore, the VideoSense Grand Challenge was established as a series of Privacy Filtering solution innovation challenges calling for context-aware and ethically reflective privacy-preserving design by the competition entrants. This series of Grand Challenge tasks initiated and managed by the VideoSense Consortium initiated and developed a framework for the conduct of such open innovation approaches to privacy-preserving solutions and their open innovation benchmarking (

For the VideoSense Grand Challenges, 5 privacy- filtering research benchmarking datasets were produced (including ground-based indoors-outdoors surveillance video datasets and drone- enabled aerial imagery car park surveillance dataset. These have been annotated and relevant legal documentation, ground truth, questionnaires etc. have all been developed for authorised access by the participants in the VideoSense Privacy Filtering Grand Challenges.

Joint activities of research were successfully conducted as “mini projects” responding to gaps in the state-of-the-art as identified by the academics and industrial Partners as well as the wider Privacy-Preserving Video-Analytics community. These mini projects researched, developed and tested the targeted sub-systems that were integrated as part of the VideoSense VCE Demonstrator as exhibited at the IEEE DSP, Jakarta 16th September 2014; plus results as joint publications focusing on Camera Calibration, Context-aware Privacy Protection, Performance Evaluation, User-Driven, Ethically-Guided CCTV Data Capture Planning and Privacy Protection Analytics, Privacy-Preserving Mini-drones operation and annotation and crowd-sourced based as well as automatic performance evaluation of privacy filters.

Project Results:
2) Technological Innovation Results

Regarding the technical developments, in part resulting from the mini projects, various algorithms have been developed and integrated, including:

• Person appearance modelling
• Various Multimedia Privacy Filters
• Motion extraction
• Action recognition
• Cultural Event Recognition
• Event Detection and Retrieval
• Object recognition and classification based on behaviour patterns
• Behavioural fuzzy classification
• RTSP- and XML server and client modules
• ONVIF serialization and de-serialization classes
• ONVIF web services for video-analytics and receiver devices
• New and/or extended evaluation methodologies for evaluation of privacy filters
• Fully Integrated Socio-ethically reflective Privacy by Design Practice Framework
• Social Privacy Impact Assessment
• Guidelines for fully Integrated Socio-ethically reflective Privacy by Design Practice for mini-drones
• Context-aware Personally-Identifying-Information-specific Privacy Filtering (Face, Skin, Gender)
• Rule-based Continuous dynamic context-responsive Privacy Filtering
• Wider Scale Empirical Studies on various aspects of privacy protection practice and impacts
• Drone Surveillance Privacy Filtering
• Objective, subjective, and crowdsourcing methodologies for privacy filtering benchmarking
• Completion 3 Privacy Task Grand Challenges within MediaEval 2012, 2013, and 2014 Benchmarking Initiative
• Establishing the MediaEval 2015 Drone Protect Challenge 2015
• Various standardisation contributions JPEG, MPEG, ISO 22311
• Contributions to a number of high impact (IEEE, CPDP) workshops and conferences
• Significant outreach to other networks and a large number of Publications
• EIP Commitment to Privacy-Preserving systems design (Smart Cities)
• Privacy and ethical compliance maturity model framework
• User-driven improved performance in terms of accuracy / complexity for current Video-Analytics
• Ethical- and privacy-related assessment of many analytics modules based on potential use cases together with ethical experts
• Identification of potential solutions for ethics- / privacy-related issues (different levels)
• Development of a set of visual privacy filters and evaluation of them on new datasets
• Proposed 5 business models
• Produced 5 video datasets for privacy evaluation
• Proposed a general scalable architecture for Socio-ethically Privacy Preserving Video-Analytics
• Contributed to Grand Challenges by anther Networks such as ChaLearn
• Enhanced website information access and Social Networks Activity

Potential Impact:
3) Example of Dissemination Results

The Consortium Partners have organised many events to support the project mission and its specific Privacy by Design research and dissemination objectives; including holding workshops on this theme, delivering invited keynotes etc.; all in high impact events with multiplier benefits in terms of awareness raising, dissemination and promoting the sharing of insights and research collaboration; these have included:

The special session focused on related Privacy Filtering Technology Evaluation and Standards within the 2nd International Workshop on Standards and Technologies in Multimedia Archives and Records (STAR) (18th - 19th January 2013, EPFL Lausanne, Switzerland.

The Special Session on Socio-ethical and Privacy Preserving Video-Analytics at the IEEE International Workshop on Multimedia Signal Processing (MMSP2012), 17-19th September 2012.

Preparatory work for follow-on contributions to the JPEG Privacy Filtering interest group in presenting the results of the VideoSense VCE joint research activities in this area in a next meeting in London (8-9 July 2013); a further contribution included the description of 10 Use-Cases as input to the co-located JPEG and MPEG standardisation conference in Strasburg October 2014 and further contributions towards a framework for privacy by design compliance assurance through the British Standards Institute.

Established a VideoSense VCE Privacy by Co-Design Cluster, for Methodologically-guided Socio-ethical Privacy By Co-Design in Video-Analytics for Security; with a first workshop with presentations from 14 related projects (22nd March 2013).

Co-organised the Privacy By Design Workshop Wireless World Research Forum (WWRF) Annual Conference 22nd – 25th October 2013, Vancouver

Provided input from the VideoSense Consortium to the concertation process for the Digital Agenda for Europe, specifically Governance , Security and Privacy; occasioned by the invited tutorial talk delivered by Professor Atta Badii at the Internet of Things Workshop, IERC Delft 7-8 February 2013; as a follow-on to the invited talk given at Ispra earlier (December 2012); elaborating on the UI-REF-based socio-ethical methodology framework for Privacy-by-CoDesign, participative engagement for requirements and evaluation prioritisation and socio-ethical impact assessment in Security Surveillance and IoT Solutions.

For contributions to the ICT Conference Digital Agenda for Europe 2103 (Vilnius on 6-8 November 2013), the Consortium proposed a VideoSense VCE Networking Session dedicated to Methodologically-guided Socio-ethical Privacy By Co-Design in Video-Analytics for Security.

JPEG Privacy standardisation sessions within meetings in London 2013, Geneva 2013
MPEG meeting in San Jose 2014, MPEG meeting in Valencia 2014 , IoT Annual Conference, Privacy and Security Applications Group, 17th – 20th June 2013, Helsinki, DSP Conference, Special Privacy by Design Workshop organised by the VideoSense Consortium, 30th June 2013 to 3rd July 2013,MediaEval 2013 Workshops, Cyber Physical Systems -Uplifting European’s the Future, 29th – 30th October 2013, Brussels

Second VideoSense ICDP 2013 Workshop, 17th December 2013, Kingston University, London
Computers, Privacy and Data Protection (CPDP2013) 23-25 January 2013 in Brussels. VideoSense planned forthcoming workshops, VideoSense Demonstration planned for MMSP2014, Jakarta, September 16, CPDP 2014 Workshop on Privacy by Design, Brussels October 2014 17.

The Consortium has disseminated the VideoSense activities and results to the research communities via other relevant QMUL projects such as Advise, 3DLife, MISSA, Cubrik and other initiatives such as DSP, ICIP, WIAMIS, MMSP and ECCV conferences.

4) Established High potential pan-European synergetic clustering and linkups; for example:

• The European Security-Privacy By Co-Design Specialist Interest Group (1st Workshop: 22-03-2103)
• The European-funded R&D Research Projects to which Privacy is highly relevant e.g. PARIS, ADVISE.
• The European COST Action Projects specifically focused on the socio-ethical aspects of Privacy.
• The European Gaming Technology Network GaLA with interest in exploiting persuasive gaming to support the UI-REF-based methodologically guided mass-participatory engagement for eliciting the prioritised socio-ethical requirements and evaluation criteria of end-users re privacy protection.
• The European Observatory for Crowds-Sourcing funded under the FP7 ICT Programme Collective-Awareness Platforms with a mission to support e-Democracy, Participative Engagement about issues of Governance, Security and Privacy.
• The European Institute of Innovation Technology (EIT)
• ICT Knowledge and Innovation Communities (KICs) Laboratories addressing the areas of SmartCities (Internet of Things), Privacy and Governance.
• The Ambient Assisted Living (AAL) Forum (
• The Project: A Sharing Approach to Promoting Science (
• The JPEG and MPEG Communities Privacy Standardisation Workshop and Interest Group
• The Internet of Things Applications Cluster
• The European Network of Regions

The Consortium have held a series of Summer Schools led by QNUL and on topics related to Ethically-guided and Privacy-respecting Video Analytics. The first VideoSense Summer school took place on 21-22 May in Dublin. The Second Summer School was organised on 30 June, 2013 in DSP2013 premises, Santorini, Greece. The Third Summer School took place n Sophia Antipolis, France on 16-17 April. Finally, the fourth VIdeoSense Winter School was organised in cooperation with ADVISE project as part of the liaison workshop on Social Acceptability of Smart Surveillance held on 24-26 November 2014 in Pont Saint Martin, Italy. All four Summer Schools represented important events for dissemination of the VideoSense project, aiming at bringing together young researchers working in the fields of:
• Pattern recognition, computer vision, and machine learning
• Motion detection and object tracking
• Event, situation, behaviour, threat modelling/simulation/recognition
• Biometrics, face detection and recognition
• Security and safety
• Privacy: legal, ethical, social, and economic issues
• Multimodal surveillance
• 3D vision
• Multi-camera calibration
• Video/audio-based human behaviour analysis
• Emotion-based content classification and organisation
• 2D/3D feature extraction
• 3D data processing and visualisation
• Multimedia content analysis and understanding
The Summer School has further offered opportunities for participants to take part in organised ethical training challenges and to further facilitate communication, the poster session has been organised where all attendees could present their work and research interests
The Consortium have held a series of MediaEval Grand Challenges on Video Privacy led by UoR. Four consecutive Visual Privacy challenges have been organised within the MediaEval benchmarking initiative and taken place during five MediaEval workshops in Pisa, Barcelona and Wurzen. The fourth series of the VideoSense MediaEval Grand Challenge was titled “DroneProtect: Mini-drone Video Privacy Task”.

List of Websites:
Professor Atta Badii: