Periodic Reporting for period 3 - SILENSE ((Ultra)Sound Interfaces and Low Energy iNtegrated SEnsors) Reporting period: 2019-05-01 to 2020-04-30 Summary of the context and overall objectives of the project SILENSE researched acoustic technologies and developed concepts to activate & control devices by gesture, data communication, and indoor positioning. These concepts can be used in many domains: wearables, automotive & smart home application.The technologies SILENSE developed, take the way we naturally communicate one step further: from communication between people to communication between humans and objects. The application of the developed technologies in numerous areas will bring lots of benefits to the general public, ranging from enhanced user experience to improved health & safety, e.g.- Intuitive user interface in mobile and wearable devices. Audio sensing allows the activation and & control of mobile devices without touching them.- Improved hygiene by touchless control, which can lead to improved hygiene conditions in buildings, hospitals and home.- Enhanced safety by touchless control: sound/voice activation/control of systems in the car & control of machinery in industrial applications.- Enhanced security by gestural authentication increasing the diversity, hence, the robustness of authentication scenarios.- Enhanced quality of life for disabled persons or elderly unable to move and the ones suffering from hearing/talking impairment could benefit from sign language interpretation.SILENSE developed smart acoustic technology blocks on different levels – hardware, software and system – to achieve these many applications.- SILENSE results contribute to lower the cost & energy consumption & improve the performance of micro-acoustic transducers.- SILENSE developed package and assembly technology. More specifically heterogeneously and monolithically integrated arrays of micro-acoustic transducers with their supporting electronics. The project also provides dedicated low-power IC design.- SILENSE developed smart algorithms for acoustic data communication and sensing.- SILENSE research results include a combination of voice/speech, digital sound modulation and gesture control by means of the same transducer(s). Work performed from the beginning of the project to the end of the period covered by the report and main results achieved so far The SILENSE partners researched the application of ultrasound and ultrasound components as key driver for future gesture recognition, under-water data transmission, localization in buildings & the automotive sector, implementable in all types of smart devices such as smart wearables, incl. underwater wearables. Ultrasound transducer technologies were investigated, suitable for various applications incl.micromachined capacitive & piezoelectric single as well as array transducers. Packaging & integration of these transducer elements for the demonstrator applications was a vital part of the SILENSE research. Another research area was electronics incl. ultrasound & audio signal processing, connected with low-level ultrasound algorithms and high-level software development. For gesture recognition machine learning schemes were applied with a lot of effort on training of the algorithms. Results of the technical work are included in the use-case demonstrators, e.g. ultrasound for device pairing & indoor navigation, gesture recognition to activate & control devices in smart homes/medical care environment & cars, haptic touch feedback on smart phones & mobile devices, voice activation & speech recognition. Other use cases incl. underwater communication & navigation for swimmers & divers have been shown. In this COVID19 situation, the exploitation areas of ultrasound touchless control devices can provide advantages to minimize screen contacts in daily life, work environment and care centers.SILENSE disseminated project results in 7 scientific journals and presented them at 22 international conferences with publication at proceedings targeting >1.000 scientists & reached > 40 citations in scientific articles so far. Dissemination in industrial sector contained presentation at trade fairs & industry oriented events with >1.500 people reached. General public audience were addressed at 13 events by project consortium partners and during other events. SILENSE website reached >10.000 unique visitors. To create links to other EU projects, extend the project impact in other sectors & identify further applications and development demand of ultrasound technology, project team attended 10 networking events with active presentation of the project. Finally consortium touched standardization field attending relevant standardization meetings (e.g. MPEG-V), where project scope & targets were presented and discussions for future proposals were made. Progress beyond the state of the art and expected potential impact (including the socio-economic impact and the wider societal implications of the project so far) The novel hardware & software components developed further advances the world of Smart and Connected Things. They enable secure & healthy Human-Machine Interfaces (HMIs) by improved communication not only between humans, but also between humans and all kinds of smart devices, by means of ultrasound gesture recognition, voice recognition and speech control and/or haptic feedback. Regarding transducer technologies a big step beyond SToA was made with micromachined transducers that become available especially for (ultrasound-) loudspeakers. SILENSE low power, high bandwidth transducer devices were demonstrated, enabling applications envisioned by the SILENSE use cases. Another SILENSE realization in the transducer area is the demonstration of large arrays (up to several cm², with up to 1000 elements) on glass or flexible substrates, suitable for a broad range of applications (fine gesture recognition, haptic feedback, medical imaging,...). Concurrently novel electronics interfaces that can integrate multiple operations (i.e. sound and ultrasound) have been developed and verified for both ultrasound emission (smart amplifiers) and detection activities (microphones with ultrasound functionalities). For heterogeneous systems built from discrete components (transducers and corresponding ICs), a Roll-to-Roll manufacturing process on flexible substrates was developed, allowing for a variety of applications. In order to avoid substrates after all and to achieve a high degree of integration and thus miniaturization, a modified FOWLP approach was developed, which allows for integration of even the most delicate cMUTs with fragile membranes.SILENSE improved state of the art for acoustic localisation and communication methods, as illustrated in several demonstrators for underwater communication. In terms of action and gesture recognition, SILENSE advanced significantly the knowledge related to usage of deep learning models, ranging from simple and fast to recognize actions to very complex gestures. The potential of the developed technologies has been demonstrated in a wide range of applications, such as wearables (including underwater wearables), smart homes and automotive. The SILENSE consortium developed 9 different demonstrators for these markets, which were validated in relevant scenarios reaching TRLs 5-6.These novel applications will bring many benefits to the general public, ranging from enhanced user experience to improved health and safety. Enhanced safety by touchless control is a highly relevant aspect as is security by gestural authentication as it increases robustness of authentication scenarios. In the present COVID-19 situation, the use of ultrasound gesture control can be applied in touchless control of devices both in everyday life, work as well as health care centres, to help in containing the spread of the virus. SILENSE research and results allow enhancing the quality of life for all people. presentatie-sheet.jpg
