Ubiquitous perception – by sensing of objects, subjects and gestures – is a pivotal opportunity for future technology: it enables personalized services such as smart living, automated logistics or interaction through free-
space gestures. However, it also challenges ethical and moral boundaries and threatens privacy. Particularly, the charter of fundamental rights of the EU (CFR-EU) highlights that balanced and sustainable development requires the protection of fundamental rights (art. 8: everyone should have the right to protect personal data).
HOLDEN proposes a radically new approach to perception by concisely analysing ethical constraints and privacy risks while re-thinking Radio Frequency (RF)-based sensing. The technology has a transformative positive effect on our economy and society by implementing article 8 of the CFR-EU through establishing necessary conditions for privacy and ethically compliant perception and derives adaptive sensing tools able to respect these constraints.
Implementation of article 8 into technology will enforce personal data protection as a fundamental right (GDPR). HOLDEN constitutes the first concentrated effort to explore social aspects of RF-sensing to guide technological
advance towards compliance of ethical and privacy constraints (WP2): we develop a model of appropriation, to obtain better understanding of public acceptance of novel sensing-based technology by stimulating the public
awareness, considering social interactions, privacy threats and ethical implications. From these findings we derive privacy and ethically compliant concepts for RF-based perception. In a radical paradigm shift from a technology-centric perspective to a privacy-centric perspective, we develop a distributed multi-antenna system for simultaneous multi-target recognition, which is private by design. HOLDEN achieves this goal along three high-risk, complementary, and privacy-centric paths:
Path 1: Continuous-space measurement points (WP3): Ethically compliant radio-based 3D vision by holographic image processing of RF wave-fronts.
Path 2: Discrete-space measurement points (WP4): Ethically compliant advanced 3D beamforming for bodypart-scale recognition and tracking through dense massively connected antenna arrays.
Path 3: Signal processing and learning (WP5): Ethically compliant high-dimensional tensor processing for the distinction of complex activities and motion from massive-dimensional RF data.
Resulting breakthrough approaches and algorithms will be compared against application-level benchmarks (WP6), including cybersecurity aspects and attacks on the system, via scenarios spanning privacy and ethical issues in the fields of logistics (INDUSTRIAL:LOW PRIVACY & ETHICAL CONCERNS), free-space user interfaces (PUBLIC:MEDIUM P&E) and smart living (PRIVATE:HIGH P&E).
Traditional RF sensing ignores human factors. Radical re-thinking of the technologies’ foundations and new paradigms are required for scalable, socially compliant perception. Through our interdisciplinary integrated
approach, human factors are identified and considered to develop a socially compliant recognition system.
HOLDEN exceeds any existing technological paradigms. No concentrated effort has been taken so far to comprehensively address ethics and privacy in RF sensing & perception. In HOLDEN (see Figures in sec. 1.2 and text-box below), ethical and privacy compliant perception is derived through targeted design constraints on all layers of the RF-sensing process (transformative positive effect):
Physical layer (static): Continuous-space measurements achieve privacy and accuracy adaptable 3D images from electromagnetic scattering (RF-holography)
Physical layer (dynamic): HOLDEN captures ethically filtered multi- target simultaneous gesture & motion in dynamic regimes from discrete-space measurements by developing smart antennas, beamforming/steering for RF perception
Signal processing and learning: We re-think signal processing and learning for ethical RF-perception to master massive RF data beyond 5G (b5G) from a conglomeration of distributed multi-antenna devices.
HOLDEN develops approaches to reach privacy through (meta-)materials in buildings (WP3), specific waveforms (WP4), as well as by pseudo-random patterns (frequency, phase or power) (WP5).
Perception through HOLDEN is disruptive: sensing under occlusion, operation in the dark, millimetre-scale precision, paired with ethical and privacy compliance. HOLDEN becomes possible now through advances in
wireless communication (b5G), where millimetre-scale (sub-THz) wavelength and Gigahertz-scale spectral width are being deployed in consumer devices - tools that had been restricted to military radar.
Overall and specific objective of HOLDEN:
Our overall objective is ethically compliant RF perception from multi-antenna devices for ubiquitous perception and multi-target recognition and with adaptable accuracy, implementing privacy by design.
