Periodic Reporting for period 1 - AWARE (Advanced Face Recognition and CroWd Behavior Analysis for Next GeneRation VidEo Surveillance)
Reporting period: 2019-09-01 to 2020-02-29
With terrorism and COVID-19 growing threat, governments and law enforcement bodies worldwide are looking for next generation face recognition (FR) systems able to address increasing security concerns in crowded places. There is a need to move FR systems to large-scale and more unconstrained environments (e.g. large indoor and outdoor places with changing lighting conditions, views, occlusions…). It is also of the utmost importance to introduce crowd behavior analytics capacities and detect abnormal events, such as mass panic, stampedes or bottlenecks. Particularly, the automatic monitoring of crowd density, and thus of the distance between individuals, becomes vitally important to control the spread of COVID-19.
Current solutions are no longer sufficient to cover these needs; important customer pain points arise:
1. Computational requirements exponentially increase because of the vast amount of video streams and faces to concurrently analyze in real-time.
2. Lack of scalability. When it comes to covering geographically extended scenarios, current FR systems have to be replicated (including databases of enrolled persons). The installation process becomes much harder, prone to errors, vulnerable and time- and cost-consuming.
3. Current FR systems do not provide crowd behavior analytics.
4. There are increasing concerns about privacy, which demand a trade-off between security and privacy, and ensuring compliance with the EU General Data Protection Regulation (GDPR) and other local regulations.
The AWARE project aims at bringing to the market a ground-breaking product that will change the way Face Recognition (FR) systems are deployed. The proposed FR system will be able to cover geographically extended, crowded and unconstrained scenarios, and to massively process faces with high performance; altogether with the detection abnormal crowd behaviour such as mass panic, stampedes or bottlenecks.
HERTA will solve state of the art limitations in the deployment of FR and crowd analysis algorithms with a hybrid edge and distributed computing pipeline, keeping secured databases in the back end. Part of the algorithmic pipeline is executed on-premise, in the front end, on low-power embedded Deep Neural Network (DNN) accelerators. This allows saving costs and increasing the system scalability, efficiency, ease-of-installation and performance. While face detection and crowd analysis are carried out on the edge, identity matching takes place in the back end, so that the database of subjects and their personal data are stored in a secure location and remain protected according to the GDPR.
Current solutions are no longer sufficient to cover these needs; important customer pain points arise:
1. Computational requirements exponentially increase because of the vast amount of video streams and faces to concurrently analyze in real-time.
2. Lack of scalability. When it comes to covering geographically extended scenarios, current FR systems have to be replicated (including databases of enrolled persons). The installation process becomes much harder, prone to errors, vulnerable and time- and cost-consuming.
3. Current FR systems do not provide crowd behavior analytics.
4. There are increasing concerns about privacy, which demand a trade-off between security and privacy, and ensuring compliance with the EU General Data Protection Regulation (GDPR) and other local regulations.
The AWARE project aims at bringing to the market a ground-breaking product that will change the way Face Recognition (FR) systems are deployed. The proposed FR system will be able to cover geographically extended, crowded and unconstrained scenarios, and to massively process faces with high performance; altogether with the detection abnormal crowd behaviour such as mass panic, stampedes or bottlenecks.
HERTA will solve state of the art limitations in the deployment of FR and crowd analysis algorithms with a hybrid edge and distributed computing pipeline, keeping secured databases in the back end. Part of the algorithmic pipeline is executed on-premise, in the front end, on low-power embedded Deep Neural Network (DNN) accelerators. This allows saving costs and increasing the system scalability, efficiency, ease-of-installation and performance. While face detection and crowd analysis are carried out on the edge, identity matching takes place in the back end, so that the database of subjects and their personal data are stored in a secure location and remain protected according to the GDPR.
At the beginning of phase 1, AWARE was at a Technology Readiness Level 6 (TRL6). At the same time, HERTA is well-positioned worldwide as one of the few face recognition providers commercializing real-time facial identification in crowded and unconstrained environments over multiple IP cameras.
During phase 1, we have carried out a feasibility study where we have assessed AWARE’s potential to reach a TRL9 and successful commercialization. It has been conducted with an objective, rational and unbiased approach to provide credible information, and has covered four frames of analysis:
- Market feasibility: We have identified, analysed and quantified market opportunities and economic benefits for the company that would result from AWARE. Next steps for commercialization will require funding, that will be hopefully acquired by applying to an Enhanced EIC Pilot grant (formerly SME Instrument phase 2) in the form of blended finance.
- Technical feasibility: We have determined at which extent HERTA has the technical expertise and resources to yield the completion of the project.
- Legal feasibility: We have thoroughly studied whether the proposed platform may arise in conflicts with international legal requirements, especially regarding data protection regulations.
- Operational feasibility: We have estimated the extent in which the proposed platform takes advantage of the observed market opportunities and established operational requirements.
According to the previous points, we have concluded that the AWARE project has a clear potential to successfully reach a TRL9 and a smooth commercialization. AWARE feasibility has therefore been confirmed.
During phase 1, we have carried out a feasibility study where we have assessed AWARE’s potential to reach a TRL9 and successful commercialization. It has been conducted with an objective, rational and unbiased approach to provide credible information, and has covered four frames of analysis:
- Market feasibility: We have identified, analysed and quantified market opportunities and economic benefits for the company that would result from AWARE. Next steps for commercialization will require funding, that will be hopefully acquired by applying to an Enhanced EIC Pilot grant (formerly SME Instrument phase 2) in the form of blended finance.
- Technical feasibility: We have determined at which extent HERTA has the technical expertise and resources to yield the completion of the project.
- Legal feasibility: We have thoroughly studied whether the proposed platform may arise in conflicts with international legal requirements, especially regarding data protection regulations.
- Operational feasibility: We have estimated the extent in which the proposed platform takes advantage of the observed market opportunities and established operational requirements.
According to the previous points, we have concluded that the AWARE project has a clear potential to successfully reach a TRL9 and a smooth commercialization. AWARE feasibility has therefore been confirmed.
Herta’s face recognition and crowd behavior analysis algorithms are based on Deep Learning. In the last five years, multi-core CPUs and GPUs have been the dominant hardware platforms for Deep Learning inference. Starting from 2018, the need for more efficient hardware acceleration of Deep Learning algorithms has been widely recognized both in academia and industry. Nowadays, a significant number of world-leading information technology firms have started to work on DNN accelerators, i.e. custom dedicated hardware highly optimized for computing with DNNs. As opposed to general-purpose CPU and GPU architectures, DNN accelerators improve the overall power efficiency at least by a factor of 10X, while dramatically reducing costs. The major players in the semiconductor industry (Intel, NVIDIA and ARM) and startup companies are now in a race to develop DNN accelerators. Some -but very few- DNN accelerators are already available in the market, but most promising hardware is announced for the end of 2020 and the beginning of 2021.
AWARE will impact the market with a ground-breaking product that will change the way face recognition systems are deployed at a large scale. It will contribute to boost European security industry, by providing new tools to improve citizens’ global and daily-life safety and health. It addresses four important priorities on the European Commission (EC) agenda:
- AI-driven technology. AWARE’s innovation potential is driven by Artificial Intelligence (AI), which has been established by the EC as one of the most strategic technologies of the 21st century, considered essential to preserve the EU technological leadership and the current and future sustainable economic growth. They deem AI as an indispensable tool to provide law enforcement bodies with suitable equipment protecting the privacy, security and dignity of citizens, to fight against crime.
- Energy sustainability. The use of low-power edge devices instead of traditional high-consuming workstations is in-line with EC energy sustainability directives. For example, a DNN accelerator consumes about 10W, while a workstation has a typical consumption of 400W.
- Ethics, fairness and trustworthiness. Contrary to other FR solutions, algorithms by HERTA are universal and unbiased in terms of gender, age and ethnicity; according to AI ethic guidelines from the EC.
- Coronavirus response. The EC is currently coordinating a common European response to the outbreak of COVID-19. They are taking resolute actions to reinforce public health and mitigate the impact of the disease in the European Union. AWARE would be a valuable tool to help control COVID-19 spread thanks to its crowd analysis capabilities.
AWARE will impact the market with a ground-breaking product that will change the way face recognition systems are deployed at a large scale. It will contribute to boost European security industry, by providing new tools to improve citizens’ global and daily-life safety and health. It addresses four important priorities on the European Commission (EC) agenda:
- AI-driven technology. AWARE’s innovation potential is driven by Artificial Intelligence (AI), which has been established by the EC as one of the most strategic technologies of the 21st century, considered essential to preserve the EU technological leadership and the current and future sustainable economic growth. They deem AI as an indispensable tool to provide law enforcement bodies with suitable equipment protecting the privacy, security and dignity of citizens, to fight against crime.
- Energy sustainability. The use of low-power edge devices instead of traditional high-consuming workstations is in-line with EC energy sustainability directives. For example, a DNN accelerator consumes about 10W, while a workstation has a typical consumption of 400W.
- Ethics, fairness and trustworthiness. Contrary to other FR solutions, algorithms by HERTA are universal and unbiased in terms of gender, age and ethnicity; according to AI ethic guidelines from the EC.
- Coronavirus response. The EC is currently coordinating a common European response to the outbreak of COVID-19. They are taking resolute actions to reinforce public health and mitigate the impact of the disease in the European Union. AWARE would be a valuable tool to help control COVID-19 spread thanks to its crowd analysis capabilities.