The FANCI near-market Innovation Action, funded under the EC H2020 Grant Agreement No. 645267, has researched, innovated, architected, designed, developed and validated innovative multimodal face and body analytics and natural computer interfaces (NCI), developed a reference platform fitting a variety of applications, and provided validation through user-experience evaluation and benchmarking in the rapidly expanding automotive field.
The project focused on intelligent user sensing methods, including face analysis, eye tracking, emotion, intention & authentication; hand gesture recognition; voice tone analysis; and head gesture & pose analysis. It complemented those with smart user intention or distraction estimation and system reaction methods, supplying visual (HUD, etc.), voice and haptic feedback as well as autonomous system actions.
FANCI delivered these capabilities in a market ready reference architecture, demonstrated by building a creative Automotive application-suite demonstrator. An Application Programming Interface for software application developers of pervasive natural computer interaction devices, was supplied. The platform empowers embedded products that capture and act upon user intentions, and imaginative utility applications fusing real and virtual content.
The project’s overall objectives were:
- Designing a multi-modal, modular, low power, real-time, “always-on’, user sensing speech and vision system, cutting power consumption of existing systems by factor of 10-50x
- Developing the FANCI API suitable for application developers who are non-domain experts
- Providing a real-time Cloud-based API
- Building a reference platform for ultra low-cost devices applicable to a range of mass market applications and products
- Validating FANCI capabilities in automotive infotainment user experiences
To achieve these goals FANCI combined multi-modal sensor data in a hardware platform to enable robust operation in natural environments, sensing explicit, implicit, and emotional actions of the user in a simple, yet powerful extensible software architecture; provided a low-cost, low-power optimised reference design for embedded products; and validated the approach by building an automotive application demonstrator. The project’s developed methods were integrated into the FANCI creative automotive applications on the partners’ universal reference platform, and validated for its user experience value as well as developer experience value.
The FANCI partners include an effective mix of innovative pioneering SMEs, industry leaders and academic researchers, guided by a large industrial Automotive systems market leader. The project, partially funded by the European Commission (EC), responded to the EC’s Horizon-2020 ICT-22-2014 call’s C focus action. It had a 28-month duration, and had a total budget of over 3.5 M€, of which the EC contributed a total funding of 2.5 M€, though in fact consumed 4.6 M€ with the difference covered by the project partners.
The consortium led various dissemination and communication activities, spearheaded by the FANCI public website, and aided by the professional participation of its industrial partners’ in-house Marketing Communications specialists, leveraging social media (Twitter, LinkedIn, other) interaction and local TV and press exposure. The FANCI partners have also taken active leadership and participation roles in dissemination events, automotive and high-standard conferences and tradeshows such as CES-2017, a concertation event (ICT-2015), industry exhibitions, webinars and seminars.
The consortium’s publication activity yielded results in par with its Integration Action near-market orientation, with one Paper nominated as Best Industrial Paper at IEEE Sensors 2016. The standardisation and certification strategies activities related to formally establish acceptance and compliance of FANCI.
