Skip to main content
Go to the home page of the European Commission (opens in new window)
English English
CORDIS - EU research results
CORDIS

Understanding Videos Automatically with the SensifAI Deep Learning Technology

Project description

Automated tagging of audio and visual content paves the way for pioneering advances

Search engines have ushered in a new era of knowledge with their ability to find information on any topic within fractions of a second. Text-based searching is relatively straightforward. Enabling searches for video content is much harder. The Belgian SME Sensifai has developed pioneering audiovisual deep-learning technology that recognises audio and video content and tags videos accurately. The EU-funded SensifAI project is helping the team prepare for commercialisation. The product will revolutionise the way we search for video content and open the door to applications from automated screening of inappropriate or dangerous content to real-time descriptions of captured video that support those with vision or hearing impairment.

Objective

Google has a value of $1 trillion because it has managed to make texts searchable. However, 80% of internet traffic is videos, audios and images and they are not searchable. Making videos searchable is extremely challenging. This is why most of the video tagging is done manually and the results in automated video recognition are still limited. Mobile video recognition is also starting to emerge. SensifAI has developed a cutting-edge audio-visual deep-learning technology trained on millions of videos to recognize audio and video content and to tag them accurately. SensifAI automatically tags videos, images and audio, which makes them searchable and can be customized for a range of use cases. We believe our approach to contextual video analysis is unique and on the leading edge as it recognizes, scenes, actions, celebrities, landmarks, logos, music genre, moods and emotion and speech. SensifAI delivers the video recognition technology on the cloud on the Amazon Web Services Marketplace and can be embedded on devices such as smartphones (by OEM’s). Our software just became available on the Amazon Web Services Marketplace where we follow a unit-based pricing model ranging from €0.01/minute for recognizing landmark images/objects/celebrities/unsafe contents to €0.05/minute for general tagging and action/sport recognition. SensifAI bvba was founded by three alumni and scientists from MIT, ETH Zurich and KU Leuven, who acquired an accumulative experience in audio-visual data processing through involvement in many international projects. Imagine a day when the 30 million visually impaired Europeans use a wearable camera equipped with a software describing them the surrounding environment automatically by recognizing the semantic concept of the captured video. includes the description of the scene, objects, and activities. Similarly, imaging a technology when the 119 million aurally impaired people use a wearable microphone equipped with a software helping them.

Fields of science (EuroSciVoc)

CORDIS classifies projects with EuroSciVoc, a multilingual taxonomy of fields of science, through a semi-automatic process based on NLP techniques. See: The European Science Vocabulary.

You need to log in or register to use this function

Programme(s)

Multi-annual funding programmes that define the EU’s priorities for research and innovation.

Topic(s)

Calls for proposals are divided into topics. A topic defines a specific subject or area for which applicants can submit proposals. The description of a topic comprises its specific scope and the expected impact of the funded project.

Funding Scheme

Funding scheme (or “Type of Action”) inside a programme with common features. It specifies: the scope of what is funded; the reimbursement rate; specific evaluation criteria to qualify for funding; and the use of simplified forms of costs like lump sums.

SME-1 - SME instrument phase 1

See all projects funded under this funding scheme

Call for proposal

Procedure for inviting applicants to submit project proposals, with the aim of receiving EU funding.

(opens in new window) H2020-EIC-SMEInst-2018-2020

See all projects funded under this call

Coordinator

SENSIFAI
Net EU contribution

Net EU financial contribution. The sum of money that the participant receives, deducted by the EU contribution to its linked third party. It considers the distribution of the EU financial contribution between direct beneficiaries of the project and other types of participants, like third-party participants.

€ 50 000,00
Address
DREVE DE NIVELLES 182/9
1160 BRUSSELS
Belgium

See on map

SME

The organization defined itself as SME (small and medium-sized enterprise) at the time the Grant Agreement was signed.

Yes
Region
Région de Bruxelles-Capitale/Brussels Hoofdstedelijk Gewest Région de Bruxelles-Capitale/ Brussels Hoofdstedelijk Gewest Arr. de Bruxelles-Capitale/Arr. Brussel-Hoofdstad
Activity type
Private for-profit entities (excluding Higher or Secondary Education Establishments)
Links
Total cost

The total costs incurred by this organisation to participate in the project, including direct and indirect costs. This amount is a subset of the overall project budget.

€ 71 429,00
My booklet 0 0