So far, even four years later after the project started, there is no new technique that allows for human-machine interfaces indoors and outdoors as efficiently as the one developed within the ECOMODE project. We have shown that a cloud-deprived set of algorithms operating only on the power budget of a mobile phone is feasible. The interface truly introduces several features that go beyond any other existing platform. We were able to develop a new EDC sensor, connect it to a mobile platform and develop a whole new range of visual processing that runs on that limited power budget. The interface provides reliable results and more importantly it introduced new features for both vision and speech. So far no one has shown that a camera can help improve speech recognition, nor an algorithm that can remove background and at the same recognize gestures on the power budget of a mobile phone. When the project started, EDC technology was available at the laboratory level. Some labs had custom USB interfaces to Windows or Linux/MAC computers. There was no possibility to interface to Android based systems. Therefore, exploitation of such technology on portable edge Android-based devices was not feasible. Now, ECOMODE has made available a highly compact and integrated plug-in module that interfaces directly through the USB connector of an Android device. This allowed the full development of Work Packages 3 to 8, resulting in useful handheld devices tested on the target users, and with commercial possibilities.During the final year, we also intensively worked on integrating all components of the WP into the mobile phone with our partner Experis. We consolidated the final databases from recordings made by Streetlab and FBK with real end users. We also improved the recognition module by fine tuning the parameters and by filtering camera noise more efficiently. An integrated system that brings together most of the works carried out during the project life within the different technical workpackages has been produced. The facilitator consists of 6 modules that cope with the main functionalities required by the end users.All these modules coexist inside the facilitator and remain isolated from each other thanks to the modularized architecture implemented within the Android application. This facilitates the maintenance, implementation of improvements and branch independent testing in order to ensure the expected behaviour and speed up bug resolutions. A mobile application that provides a set of native applications, also named as applets, that provide a redesigned visual interface offering the same functionalities in a more intuitive way for the end users has been developped. Thus, the multimodality of the ECOMODE project enables to move through these applications by combining gestures and speech commands, selected after analysing several aspects such as cognitive demand, which aims to improve the communication between mobile devices and people that is not familiarized with emerging technologies. These applets have been considered in common with the end users’ representatives.
The results of ECOMODE have been extensively disseminated through publications, and participation to international conferences. A full exploitation plan was included in D8.3 Exploitation Plan and Updates.
