Periodic Reporting for period 3 - ECOMODE (Event-Driven Compressive Vision for Multimodal Interaction with Mobile Devices)
Reporting period: 2017-07-01 to 2018-12-31
The results of ECOMODE have been extensively disseminated through publications, and participation to international conferences. A full exploitation plan was included in D8.3 Exploitation Plan and Updates.
-DVS cameras are becoming commercially attractive. Besides Chronocam (now called Prophesee), there are a few other companies commercializing variants of DVS cameras: Inivation (Zurich), Celex Ltd. (Singapore/China), and even Samsung has announced developments for DVS cameras. In ECOMODE, for the first time, this type of sensor has been used on a mobile platform running Android for Gesture recognition for vision impaired people. We expect that this could be one of many application domains of this new type of camera sensors. Having available such low-cost systems for vision impaired people has a high potential for societal impact. At the more technological level, the specific developments within WP2 open new ways for other application domains (autonomous cars, robots, surveillance, smart vision-triggered always-on IoT devices, etc).
The developments within WP2 have advanced with respect to the state-of-the-art in the following aspects:
-A compact USB plug-in DVS camera module is available for edge devices (tablets or mobile phones),
-A USB interface for Android systems is available,
-A DVS-MIPI interface module that can be synthesized for ASIC or FPGA is available
We have also increased knowledge of the field of event based computation in different aspects. We improved the machine learning algorithm based on time surfaces (HOTS). We added a suppression of redundant and overlapping event by centering events on main activity events. We also explored the space of parameters to define an optimal architecture that allows for a robust recognition of the database collected We introduced a background removal while the phone is in motion indoors and outdoors. This method is genuinely new as it does not necessitate the computation of optical flow, or the inertia measurement unit of the phone. We showed for the first time that the relative activity of events of a background and the foreground (where gesture happen) can be a reliable measure to separate a gesture from a dynamic background. We implemented and validated the approach and most of all managed to integrate of these into the mobile phone.