Periodic Reporting for period 1 - HOLDEN (Ethical Design of Holography with Dense wireless Networks)
Berichtszeitraum: 2023-06-01 bis 2024-05-31
Ubiquitous perception – by sensing of objects, subjects and gestures – is a pivotal opportunity for future technology: it enables personalized services such as smart living, automated logistics or interaction through free-
space gestures. However, it also challenges ethical and moral boundaries and threatens privacy. Particularly, the charter of fundamental rights of the EU (CFR-EU) highlights that balanced and sustainable development requires the protection of fundamental rights (art. 8: everyone should have the right to protect personal data).
HOLDEN proposes a radically new approach to perception by concisely analysing ethical constraints and privacy risks while re-thinking Radio Frequency (RF)-based sensing. The technology has a transformative positive effect on our economy and society by implementing article 8 of the CFR-EU through establishing necessary conditions for privacy and ethically compliant perception and derives adaptive sensing tools able to respect these constraints.
Implementation of article 8 into technology will enforce personal data protection as a fundamental right (GDPR). HOLDEN constitutes the first concentrated effort to explore social aspects of RF-sensing to guide technological
advance towards compliance of ethical and privacy constraints (WP2): we develop a model of appropriation, to obtain better understanding of public acceptance of novel sensing-based technology by stimulating the public
awareness, considering social interactions, privacy threats and ethical implications. From these findings we derive privacy and ethically compliant concepts for RF-based perception. In a radical paradigm shift from a technology-centric perspective to a privacy-centric perspective, we develop a distributed multi-antenna system for simultaneous multi-target recognition, which is private by design. HOLDEN achieves this goal along three high-risk, complementary, and privacy-centric paths:
Path 1: Continuous-space measurement points (WP3): Ethically compliant radio-based 3D vision by holographic image processing of RF wave-fronts.
Path 2: Discrete-space measurement points (WP4): Ethically compliant advanced 3D beamforming for bodypart-scale recognition and tracking through dense massively connected antenna arrays.
Path 3: Signal processing and learning (WP5): Ethically compliant high-dimensional tensor processing for the distinction of complex activities and motion from massive-dimensional RF data.
Resulting breakthrough approaches and algorithms will be compared against application-level benchmarks (WP6), including cybersecurity aspects and attacks on the system, via scenarios spanning privacy and ethical issues in the fields of logistics (INDUSTRIAL:LOW PRIVACY & ETHICAL CONCERNS), free-space user interfaces (PUBLIC:MEDIUM P&E) and smart living (PRIVATE:HIGH P&E).
Traditional RF sensing ignores human factors. Radical re-thinking of the technologies’ foundations and new paradigms are required for scalable, socially compliant perception. Through our interdisciplinary integrated
approach, human factors are identified and considered to develop a socially compliant recognition system.
HOLDEN exceeds any existing technological paradigms. No concentrated effort has been taken so far to comprehensively address ethics and privacy in RF sensing & perception. In HOLDEN (see Figures in sec. 1.2 and text-box below), ethical and privacy compliant perception is derived through targeted design constraints on all layers of the RF-sensing process (transformative positive effect):
Physical layer (static): Continuous-space measurements achieve privacy and accuracy adaptable 3D images from electromagnetic scattering (RF-holography)
Physical layer (dynamic): HOLDEN captures ethically filtered multi- target simultaneous gesture & motion in dynamic regimes from discrete-space measurements by developing smart antennas, beamforming/steering for RF perception
Signal processing and learning: We re-think signal processing and learning for ethical RF-perception to master massive RF data beyond 5G (b5G) from a conglomeration of distributed multi-antenna devices.
HOLDEN develops approaches to reach privacy through (meta-)materials in buildings (WP3), specific waveforms (WP4), as well as by pseudo-random patterns (frequency, phase or power) (WP5).
Perception through HOLDEN is disruptive: sensing under occlusion, operation in the dark, millimetre-scale precision, paired with ethical and privacy compliance. HOLDEN becomes possible now through advances in
wireless communication (b5G), where millimetre-scale (sub-THz) wavelength and Gigahertz-scale spectral width are being deployed in consumer devices - tools that had been restricted to military radar.
Overall and specific objective of HOLDEN:
Our overall objective is ethically compliant RF perception from multi-antenna devices for ubiquitous perception and multi-target recognition and with adaptable accuracy, implementing privacy by design.
space gestures. However, it also challenges ethical and moral boundaries and threatens privacy. Particularly, the charter of fundamental rights of the EU (CFR-EU) highlights that balanced and sustainable development requires the protection of fundamental rights (art. 8: everyone should have the right to protect personal data).
HOLDEN proposes a radically new approach to perception by concisely analysing ethical constraints and privacy risks while re-thinking Radio Frequency (RF)-based sensing. The technology has a transformative positive effect on our economy and society by implementing article 8 of the CFR-EU through establishing necessary conditions for privacy and ethically compliant perception and derives adaptive sensing tools able to respect these constraints.
Implementation of article 8 into technology will enforce personal data protection as a fundamental right (GDPR). HOLDEN constitutes the first concentrated effort to explore social aspects of RF-sensing to guide technological
advance towards compliance of ethical and privacy constraints (WP2): we develop a model of appropriation, to obtain better understanding of public acceptance of novel sensing-based technology by stimulating the public
awareness, considering social interactions, privacy threats and ethical implications. From these findings we derive privacy and ethically compliant concepts for RF-based perception. In a radical paradigm shift from a technology-centric perspective to a privacy-centric perspective, we develop a distributed multi-antenna system for simultaneous multi-target recognition, which is private by design. HOLDEN achieves this goal along three high-risk, complementary, and privacy-centric paths:
Path 1: Continuous-space measurement points (WP3): Ethically compliant radio-based 3D vision by holographic image processing of RF wave-fronts.
Path 2: Discrete-space measurement points (WP4): Ethically compliant advanced 3D beamforming for bodypart-scale recognition and tracking through dense massively connected antenna arrays.
Path 3: Signal processing and learning (WP5): Ethically compliant high-dimensional tensor processing for the distinction of complex activities and motion from massive-dimensional RF data.
Resulting breakthrough approaches and algorithms will be compared against application-level benchmarks (WP6), including cybersecurity aspects and attacks on the system, via scenarios spanning privacy and ethical issues in the fields of logistics (INDUSTRIAL:LOW PRIVACY & ETHICAL CONCERNS), free-space user interfaces (PUBLIC:MEDIUM P&E) and smart living (PRIVATE:HIGH P&E).
Traditional RF sensing ignores human factors. Radical re-thinking of the technologies’ foundations and new paradigms are required for scalable, socially compliant perception. Through our interdisciplinary integrated
approach, human factors are identified and considered to develop a socially compliant recognition system.
HOLDEN exceeds any existing technological paradigms. No concentrated effort has been taken so far to comprehensively address ethics and privacy in RF sensing & perception. In HOLDEN (see Figures in sec. 1.2 and text-box below), ethical and privacy compliant perception is derived through targeted design constraints on all layers of the RF-sensing process (transformative positive effect):
Physical layer (static): Continuous-space measurements achieve privacy and accuracy adaptable 3D images from electromagnetic scattering (RF-holography)
Physical layer (dynamic): HOLDEN captures ethically filtered multi- target simultaneous gesture & motion in dynamic regimes from discrete-space measurements by developing smart antennas, beamforming/steering for RF perception
Signal processing and learning: We re-think signal processing and learning for ethical RF-perception to master massive RF data beyond 5G (b5G) from a conglomeration of distributed multi-antenna devices.
HOLDEN develops approaches to reach privacy through (meta-)materials in buildings (WP3), specific waveforms (WP4), as well as by pseudo-random patterns (frequency, phase or power) (WP5).
Perception through HOLDEN is disruptive: sensing under occlusion, operation in the dark, millimetre-scale precision, paired with ethical and privacy compliance. HOLDEN becomes possible now through advances in
wireless communication (b5G), where millimetre-scale (sub-THz) wavelength and Gigahertz-scale spectral width are being deployed in consumer devices - tools that had been restricted to military radar.
Overall and specific objective of HOLDEN:
Our overall objective is ethically compliant RF perception from multi-antenna devices for ubiquitous perception and multi-target recognition and with adaptable accuracy, implementing privacy by design.
The technical workpackages 2, 3, and 4,have started as planned.
In WP2 (Ethics and privacy), Social implications and preconditions for responsible Design (D2.1) as well as Social implications and preconditions for Technological Design (D2.2) have been derived.
Particularly, the guidance ethics approach has been applied at a workshop conducted with project partners and external stakeholders. Ethical concerns have been formulated and discussed among project partners.
Particularly, the risk of ubiquitous application of the technology, fostered by an ubiquitous availability of the communication infrastructure has been identified. A potential of the technology to alter the human condition has been identified.
Some risk mitigation strategies have been identified.
In WP3, a simulation model of a typical indoor scenario (D3.1) has been provided. Particularly, It was demonstrated that various ways of simulating typical indoor propagation scenarios are available, ranging from full-wave simulations to ray tracing approaches. Despite the suitability of alternative methods, our recommended ray-PO hybrid method exhibits a good balance between accuracy and efficiency, as demonstrated by various simulation results. Although our simulations primarily focus on the frequency domain delivering synthetic data as basis for holographic imaging, obtaining the time behaviour is also achievable through Fourier transformation, providing a time delay spread function for dynamic channel analysis.
WP4 did not have their firs deliverable yet.
In WP2 (Ethics and privacy), Social implications and preconditions for responsible Design (D2.1) as well as Social implications and preconditions for Technological Design (D2.2) have been derived.
Particularly, the guidance ethics approach has been applied at a workshop conducted with project partners and external stakeholders. Ethical concerns have been formulated and discussed among project partners.
Particularly, the risk of ubiquitous application of the technology, fostered by an ubiquitous availability of the communication infrastructure has been identified. A potential of the technology to alter the human condition has been identified.
Some risk mitigation strategies have been identified.
In WP3, a simulation model of a typical indoor scenario (D3.1) has been provided. Particularly, It was demonstrated that various ways of simulating typical indoor propagation scenarios are available, ranging from full-wave simulations to ray tracing approaches. Despite the suitability of alternative methods, our recommended ray-PO hybrid method exhibits a good balance between accuracy and efficiency, as demonstrated by various simulation results. Although our simulations primarily focus on the frequency domain delivering synthetic data as basis for holographic imaging, obtaining the time behaviour is also achievable through Fourier transformation, providing a time delay spread function for dynamic channel analysis.
WP4 did not have their firs deliverable yet.
The social implications and preconditions for technological and responsible design advance the state of the art in the domain to foster a better understanding of ethical implications of radio sensing.
Likewise, the derived ray-PO hybrid method for full-wave simulations in ray tracing approaches advances the state-of-the-art.
Likewise, the derived ray-PO hybrid method for full-wave simulations in ray tracing approaches advances the state-of-the-art.