Community Research and Development Information Service - CORDIS


FANCI Report Summary

Project ID: 645267
Funded under: H2020-EU.

Periodic Reporting for period 1 - FANCI (Face and body Analysis Natural Computer Interaction)

Reporting period: 2015-01-01 to 2015-12-31

Summary of the context and overall objectives of the project

The FANCI near-market Innovation Action, funded under the EC H2020 Grant Agreement No. 645267, researches, innovates, architects, designs, develops and validates innovative multimodal face and body analytics and natural computer interfaces (NCI), develops a reference platform fitting a variety of applications, and provides validation through user-experience evaluation and benchmarking in the rapidly expanding Automotive field.
The project focuses on intelligent user sensing methods, including face analysis, eye tracking, emotion, intention & authentication; hand gesture recognition; voice tone analysis; and head gesture & pose analysis. It complements those with smart user intention or distraction estimation and system reaction methods, supplying visual (HUD, etc.), voice and haptic feedback as well as autonomous system actions.
The project’s overall objectives are:
- Designing a multi-modal, modular, low power, real-time, “always-on’, user sensing speech and vision system, cutting power consumption of existing systems by factor of 10-50x
- Developing the FANCI API suitable for application developers who are non-domain experts
- Providing a real-time Cloud-based API
- Building a reference platform for ultra low-cost devices applicable to a range of mass market applications and products
- Validating FANCI capabilities in automotive infotainment user experiences

Work performed from the beginning of the project to the end of the period covered by the report and main results achieved so far

The project began its 24-month work in January 2015, and – at the end of its first year of operation – is nicely progressing according to the plan.
The project’s early prototype and automotive application demonstration will soon be presented in the January 2016 CES-2016, the world largest consumer electronics trade show, at Harman’s booth. The partners’ building blocks forming this prototype were presented at the FANCI booth in the October ICT-2015 conference in Lisbon. The project’s developer reference platform has been developed, and work in underway to develop FANCI’s user sensing methods, user-state estimation methods and system-reaction methods.
In the coming months, those developed methods will be integrated into the FANCI creative automotive applications on the partners’ universal reference platform, in order to then be analysed and validated for its user experience value.
10 deliverables were submitted as planned: FANCI Conceptual Software and Hardware Requirements Definition; HMI Reference Platform Architecture; Automotive Demonstrator Architecture; Developer reference platform; Automotive application demonstration running on prototype platform; Project Public Website; FANCI Knowledge Portfolio; Dissemination, Standardisation & Certification Plan; First Project Periodic Report; Ethics Report.
During this period, the project partners held face-to-face consortium meetings and technical integration workshops, as well as periodic conference calls, to ensure optimal synergy and collaboration. An online collaboration framework and document repository was set-up to further this purpose.
The consortium led various communication activities, spearheaded by the FANCI public website, and aided by the professional participation of its industrial partners’ in-house Marketing Communications specialists, leveraging social media (Twitter, LinkedIn, other) interaction and local-TV and press exposure. The FANCI partners have also taken active leadership and participation roles in the two major dissemination events held by the project in 2015 and early-2016: ICT-2015 and CES-2016. The partners have also contributed to the project’s first technical paper, Innovative System of Interaction between the Driver and Vehicle, and to the project’s standardisation and automotive certification efforts. The FANCI consortium has also begun early exploitation planning deliberations.

Progress beyond the state of the art and expected potential impact (including the socio-economic impact and the wider societal implications of the project so far)

Progress beyond the state of the art:
a) Multi-modal, real-time, low-cost, interoperable, low power NCI platform
b) Driver predictive, multimodal and natural interaction assistive system
c) Face and body Human Machine Interfaces (HMI)
d) Emotion and intention-triggered interaction (understanding human behaviour)
e) Creative applied augmented reality

Expected potential impact:
The multi-modality capabilities implemented at the core of the FANCI project will increased robustness and accuracy of biometrics and user state estimation. Combined with an always-on, low cost, and real-time platform, the platform will enable creation of new use cases not possible before. This will further strengthen the European business of the projects partners who each have either an OEM or licensing business model, supplying technology and services to other European companies. FANCI will have impact in several important domains, some of which are directly related to impacts expected by the call, including:

● Expected Impact A: Enable better uses of ICT technologies within the creative industries by providing directly usable solutions addressing their specific needs
One of the largest employers inside the creative industries sector of the EU is the software consultancy industry employing over 1.6 Million people. This group is dominated by SMEs, with micro-SMEs and free-lancers representing 85% of all actors. The FANCI concepts and high-level API will address needs of ICT and SME developers to enable new types of interface devices and new services based on the user sensing data.
The improved robustness and accuracy of biometrics and user emotion estimation together with the always on platform can be used to create new forms of user generated digital content. This is particularly interesting for the broadcasting and publishing industries. While the system runs in the background, identification of the person or the emotional engagement of the person can be used to select and propose dynamic content which will be more interesting to the user. This will bring interactivity to what was before non-interactive media.
Such a system is also very appropriate for the advertising industry. Currently advertisement becomes more and more specific as soon as the person is somehow identified. This usually requires external digital profiles and can be a problem for privacy. By analysing the emotion or body language of people, some new generic profiles will be made possible which can then be instantly detected without affecting privacy issues. Content and experience can be made interactive based on real-time emotional feedback from users.
User generated content is also very useful for computer games, in order to adapt the storyline or the difficulty of the game to the player's emotional and behavioural state. The improved effectiveness of the multi-modal interfaces will make them deployable in real environments. In the computer games industry, these interfaces will be made available to explore new ways of interaction with digital worlds. In the digital films and video industries, non-invasive interfaces will be available to create new digital content. This could be something similar to the currently heavily used motion and performance capture interfaces.
FANCI’s modular API, reference platform and automotive infotainment demonstrator will bring directly usable solutions to the creative industries. The demonstrator will be especially of interest to the automotive infotainment industries. It will show how they can improve the user experience and comfort in the car while simultaneously improving security and safety.
By abstracting the complicated technology required by user sensing methods, will enable non experts to use advanced human machine interfaces. It will enable the integration of these interfaces in many existing systems. For example, if gestures have already been used to play virtual instruments in a virtual room, it will be easily extended with face analysis and eyes tracking to control more parameters of the virtual instruments.
The impact of the FANCI will also be linked to saving lives through safety measures:
- Reduce distraction through improved and simplified user experience
- Increase car sharing usage via personalisation
- Technology available to everyone through low-cost platform
We expect FANCI to improve the life of millions of drivers around the world. FANCI Security and personalisation will promote car sharing for better environmental practises. We won't need a personal car since every car can be adapted to a specific person. FANCI Safety and comfort will prevent many accidents from happening. FANCI Advanced interfacing and FANCI Creative applied augmented reality will make interaction with car systems more natural. Finally, FANCI technology will be available to almost everyone.

● Expected Impact B: Provide a large spill over of the knowledge acquired to a maximum of European industries
FANCI will not only benefit the creative industries. A lot of knowledge will be acquired during the project which will be easily shared. First, benchmarks will be created to evaluate the combined performance of multi-modal user sensing methods. These will be a base to be able to make a fair comparison of different systems. In our case, the focus is the combination of different natural computing interfaces. The FANCI API will help many industries to experiment with these multi-modal interfaces and also introduce new interfaces. Finally, the automotive application demonstrator will be an easy tool to show the performance achieved on multi-modal user sensing, but also to show how to achieve it. Thus the results of the project, but also the possibility to run fair experiments in an easy way, will help spill over the knowledge acquired during the project.
The expected improvement in biometrics will be helpful for the security industries, investment and banking industries to deliver secured and personalised services. Already the demand for touch-less biometrics is on the rise owing to the accuracy on the part of the system and FANCI knowledge around multi-modal capabilities will further accelerate this trend. The touch-less sensing market is expected to reach $11.0 billion by the end of 2020 at a CAGR of 24.03%.
Another area with large impact is the gesture recognition market which is expected to reach $11.03 billion in 2020 from $322.30 million in 2012 at a CAGR of 42.89% from 2013 to 2020. Gesture recognition is an emerging technology that has the potential to revolutionise the way humans interact with machines; not only in gaming and automotive applications but also in day-to-day activities on the home front. FANCI knowledge and capabilities will help spread the awareness of this new revolutionary human machine interaction technology. In this application, consumer electronics will gain popularity first, and then slowly automotive, automation, healthcare and others will embrace gesture recognition.
In the automotive industries, the identification of drivers will promote the use of car sharing: it will be possible to retrieve some personal settings in different cars, plus it will be possible to have the car settings adapt to many people. The demonstrator is also an obvious way to benefit the automotive industries.
In recent years, children are opting for more sophisticated video games and electronic toys. Toy manufacturers are obliged to introduce new products regularly and focus on innovation. The worldwide toy market is expected to represent $100 billion by 2015. The prospects are so good that large ICT companies such as Microsoft and Intel are adding their offerings to the market. Smart and power-efficient vision is expected to be a key capacity that will bring intelligence and human-like behaviour to toys. And the expected improvements in the multi-modal interfaces of FANCI will also benefit this creative industry.
Most of the service-oriented industries will be interested in the knowledge acquired about user sensing methods, since tuning the services to the users are at the core of their business.

● Expected Impact C: Improve the competitive position of the European industries through the provision of cost effective, innovative and high-value products and services
FANCI focuses on providing a modular, low power, low cost, reference developer platform with advanced HMI functionally. This will positively impact the competitive positioning of the embedded ICT sector within Europe touching multiple European industries such as consumer electronics, automotive, robotics, and healthcare.
The value of the embedded ICT market is estimated at more than 850 billion Euros and currently Europe has a 25%-30% share of the world market. However, the global ICT sector in Europe has been experiencing declining figures in the last years. Less than 10 percent of the global ICT sales of the top-100 ICT companies are generated by European companies and only 15 of the top-100 ICT companies are headquartered in Europe.
Europe must take steps to be competitive and counter low development costs, high innovation and shorter time-to-market. A study by Accenture assessed the required skills for companies integrating embedded software into their products. The top skill needed was “Speeding up time-to-market”. A key focus of FANCI. By providing an optimised developer platform providing both hardware and associated middleware required to develop and deploy applications, FANCI will contribute to speed up development of differentiated embedded ICT products across Europe.
Highlighted are three examples where FANCI will improve competitive positioning:
- Automotive. There is an enormous innovation potential for advanced HMI based functionalities in the car: detection of drowsy or distracted drivers, driver recognition, gesture recognition, and personalization of in-car experiences. Apart from directly improving the end consumer experience, understanding user behaviour through sensing interactions including emotional states of the driver opens up new analytics opportunities in providing manufacturers better insights on how their customers are engaging and using features of the car. The total available market for embedded vision in automotive applications has a potential for installation in nearly 100 million vehicles by 2016.
- Advanced Robotics: One of the most disruptive technologies that will transform life and business with an estimated economic impact by 2025 of $1.7 trillion to $4.5 trillion per year. Of particular importance will be autonomous vehicles (terrestrial, aerial and submarine) and personal and home robots. These vehicles and robots rely on vision for navigation as well as for human-robot interaction. FANCI capabilities will be a fundamental for human-robot interactions.
- Wearables: The global wearable technology market is expected to reach $5.8 billion in 2018, at an impressive CAGR of 40.8 percent from 2012 to 2018. Gesture-enabled consumer-electronics devices are expected to reach penetration levels of 35% by 2018. Wearable and augmented reality systems are intended to provide intelligent functions which require perception of people and the world. Low power capabilities of FANCI will be a core element of such devices.

Knowledge and technologies accumulated and developed during this project will be used after its completion in commercial, private and public sectors. The results are expected to meet a lot of attention in the near future. The commercialisation and further production of multisensory platforms can be predicted. Cooperation within the created collaborative research and development net is expected to further increase the income over the coming years. For example, experience acquired in this project will be used in other projects linked to NCI, ADAS and other technologies. Possibilities of joint venturing and joint activities among some of the partners will further increase following the success of the project, to result in a better outcome and higher-quality results to be achieved vs. working separately. The effect of this organised, close, intense collaboration will also provide valuable advances to each of the partners and will create an extensive R&D foundation, which will be of further use in further endeavours.

Related information

Record Number: 186668 / Last updated on: 2016-07-14