FAST track 2 FACE REcognition Camera dominance

Project Information

FASTFACEREC

Grant agreement ID: 820615

DOI

10.3030/820615

Project closed

EC signature date 20 September 2018

Start date 15 September 2018

End date 14 March 2021

Funded under

• Horizon 2020 Framework Programme

Total cost

€ 4 190 771,25

EU contribution

€ 2 913 624,50

Coordinated by

PHOTONIC SENSORS AND ALGORITHMS SL
Spain

Periodic Reporting for period 2 - FASTFACEREC (FAST track 2 FACE REcognition Camera dominance)

Reporting period: 2019-09-15 to 2021-03-14

Summary of the context and overall objectives of the project

This project developed the most competitive cameras for Face-Recognition, Mixed Reality and Augmented Reality for mobile phones, tablets and laptop computers: specifically an embedded camera module, a highly competitive device aiming towards disruptive marketshare and global leaderhip.
The announcement by Apple of the IPhone X (September 12th, 2017) ignited the need in all mobile phone manufacturers to incorporate 3D cameras for "face recognition." These cameras not only have application in pone unblocking, cybersecurity and digital trust but also in augmented reality and biometric analysis, among others.
Our device reduces by more than 50% the costs of the BOM (Bill of Material) for the camera, increases drastically the resolution of the depth map (from 40 Kpixels to 1,45 Mpixels), provides real-time-video and consumes much lower power than TOF (Time of Flight) and SL (Structured Light) technologies.
Our most disruptive competitive advantage is based on our algorithms (with only 1% of the computing power required by the competition): we create high-quality depth-maps, 3D-content and all-in-focus images, we can process 60 fps videos under ANDROID processors while competitors need seconds to minutes to process a single frame with powerful GPUs.
The disruptive new ideas described herein are the result of a synergetic interdisciplinary value-chain bringing together highly skilled SMEs none of whom would be able to achieve the breakthroughs alone: a manufacturer of world-class micro-optics; a cutting-edge supplier of highly automated micro-assembly and testing solutions for the photonics industry; and a disruptive SME specialized in 3D-image acquisition and algorithms to compute depthmaps and display 3D-images.
Our specific objectives aim to disrupt into an established value-chain (mobile telephony) accelerating the development of ideas into business-driven new products (a face recognition camera) targetting to bring turnovers of € 1.09 billion by 2025 to the partners of this consortium. Synergies with existing products guarantee a swift exploitation of results starting before the end of the project. To achieve that goal photonicSENS will quickly scaleup production upon successful achievement of the other partners objectives: NILT aims to quickly scale-up production of nanometer precision lenses using wafer optics manufacturing, ficonTEC aims to produce the world´s quickest alignment and assembly machine with 100 nm precision to scale-up production throughput at low costs.

Work performed from the beginning of the project to the end of the period covered by the report and main results achieved so far

The main objective of FastFaceRec is to develop the most competitive cameras for Face-Recognition, Mixed Reality and Augmented Reality for mobile phones, tablets and laptop computers. Our specific objectives aim to disrupt into an established value-chain (mobile telephony) accelerating the development of ideas into business-driven new products (a face recognition camera) targeting to bring turnovers of € 1.09 billion by 2025 to the partners of this consortium.

The specific objectives for the project which are relevant for the current report – and the necessary work (tasks, deliverables and milestones) – to conduct in order to achieve them completely during the 30 months of the project, and were focused mainly in obtaining the first set of prototypes and optimizing the manufacturing process.

In order to reach those goals, Photonicsens, defined the product required specifications. Once defined, the Optical design of the different components was performed and followed by the assembly of the different elements that conform to the camera. The camera has been tested in order to ensure the good performance of the set of manufactured products, obtaining the expected results.

In parallel, a new concept of a high throughput alignment and the mounting machine has been implemented within FiconTEC and PhotonicSens.

At the end of the project, with the project successfully finished, the consortium has reached functional prototypes of a newly designed camera and a concept for mass production.

Progress beyond the state of the art and expected potential impact (including the socio-economic impact and the wider societal implications of the project so far)

The members of this consortium aim to reshape the market of “3D imaging and sensing” cameras for mobile telephony, a new market started in 2016 and according to Wooside Capital partners is expected to reach volumes of 712 million units and revenues of $ 6000 million by 2022 (Smartphone Depth Sensing report). Our face recognition depth camera constitutes a major scientific breakthrough in smartphone cameras that together with the wafer level manufacturing techniques used for the ad-hoc lens and the unique highly automated alignment and mounting techniques developed in this project improve the “innovation capacity strengthening the competitiveness and scale-up of the industrial partners of the consortium”.

PhotonicSENS has access and fully exploits the scientific and technological capacity from NILT (advanced wafer optics) and ficonTEC (world-class manufacturing processes and machines). NILT and ficonTEC enjoy the product innovation capacity from photonicSENS, likely to become one of the best customers with the results from this project. These synergies “improve the innovation capacity” of the 3 partners in a more effective way than working in isolation, very effectively “strengthening the competitiveness and scale-up of the industrial partners in the consortium”.

The main needs identified from our direct users (manufacturers of mobile phones) are to provide a reliable Depth map with more pixels than competition at lower costs and with low requirements in terms of computing power (as well as specific image formats, image quality, physical buses, standards and regulatory issues. Our solution provides 1.45 Megapixels depth maps while other suppliers of the mobile pone industry can only offer 40 Kpixels (our depth maps are 36 times better), and we do it at 50% of the cost of their Bill of Materials. We achieve this by using a single CMOS image sensor with ad-hoc optics and algorithms while our competitors need at least two components: InfraRed camera(s) and InfraRed Transmitter(s). This will allow our customers to offer more reliable face recognition, which is becoming an important biometric identification feature to secure online payments as a second identity check in addition to fingerprint sensing, and much more secure (especially if we increase the number of megapixels in the depth map). We will also enable our customers to outcompete other solutions by offering better applications in mixed reality (merging graphics into real world images), facial morphing, gesture recognition and avatars that react in real-time face movements, besides additional functions like all-in-focus images and advanced Bokeh effects (images unfocused at the background, at the foreground or at any depth range decided by the user).

Camera Module