Skip to main content

Methods for Managing Audiovisual Data: Combining Automatic Efficiency with Human Accuracy

Objective

Audiovisual media content created and used in films and videos is key for people to communicate and entertain. It has also become an essential resource of modern history, since a large portion of memories and records of the 20th and 21st centuries are audiovisual. To fully benefit from this asset, fast and effective methods are needed to cope with the rapidly growing audiovisual big data that are collected in digital repositories and used internationally. MeMAD will provide novel methods for an efficient re-use and re-purpose of multilingual audiovisual content which revolutionize video management and digital storytelling in broadcasting and media production. We go far beyond the state-of-the-art automatic video description methods by making the machine learn from the human. The resulting description is thus not only a time-aligned semantic extraction of objects but makes use of the audio and recognizes action sequences. While current methods work mainly for English, MeMAD will handle multilingual source material and produce multilingual descriptions and thus enhance the user experience. Our method interactively integrates the latest research achievements in deep neural network techniques in computer vision with knowledge bases, human and machine translation in a continuously improving machine learning framework. This results in detailed, rich descriptions of the moving images, speech, and audio, which enable people working in the Creative Industries to access and use audiovisual information in more effective ways. Moreover,the intermodal translation from images and sounds into words will attract millions of new users to audiovisual media, including the visually and hearing impaired. Anyone using audiovisual content will also benefit from these verbalisations as they are non-invasive surrogates for visual and auditory information, which can be processed without the need of actually watching or listening, matching the new usage of video consumption on mobile devices.

Call for proposal

H2020-ICT-2016-2017

See other projects for this call

Sub call

H2020-ICT-2017-1

Coordinator

AALTO KORKEAKOULUSAATIO SR
Net EU contribution
€ 752 207,50
Address
Otakaari 1
02150 Espoo
Finland

See on map

Region
Manner-Suomi Helsinki-Uusimaa Helsinki-Uusimaa
Activity type
Higher or Secondary Education Establishments
Other funding
€ 0,00

Participants (8)

HELSINGIN YLIOPISTO
Finland
Net EU contribution
€ 397 398,75
Address
Yliopistonkatu 3
00014 Helsingin Yliopisto

See on map

Region
Manner-Suomi Helsinki-Uusimaa Helsinki-Uusimaa
Activity type
Higher or Secondary Education Establishments
Other funding
€ 0,00
EURECOM GIE
France
Net EU contribution
€ 401 598,75
Address
Route Des Chappes 450 Campus Sophia Antipolis
06410 Biot

See on map

Region
Provence-Alpes-Côte d’Azur Provence-Alpes-Côte d’Azur Alpes-Maritimes
Activity type
Higher or Secondary Education Establishments
Other funding
€ 0,00
UNIVERSITY OF SURREY
United Kingdom
Net EU contribution
€ 370 680,00
Address
Stag Hill
GU2 7XH Guildford

See on map

Region
South East (England) Surrey, East and West Sussex West Surrey
Activity type
Higher or Secondary Education Establishments
Other funding
€ 0,00
YLEISRADIO OY
Finland
Net EU contribution
€ 354 850,00
Address
Uutiskatu 5
00240 Helsinki

See on map

Region
Manner-Suomi Helsinki-Uusimaa Helsinki-Uusimaa
Activity type
Private for-profit entities (excluding Higher or Secondary Education Establishments)
Other funding
€ 0,00
LIMECRAFT NV
Belgium
Net EU contribution
€ 633 080,00
Address
Sint Salvatorstraat 18b/301
9000 Gent

See on map

SME

The organization defined itself as SME (small and medium-sized enterprise) at the time the Grant Agreement was signed.

Yes
Region
Vlaams Gewest Prov. Oost-Vlaanderen Arr. Gent
Activity type
Private for-profit entities (excluding Higher or Secondary Education Establishments)
Other funding
€ 0,00
LINGSOFT OY
Finland
Net EU contribution
€ 157 717,50
Address
Etelaranta 10
00130 Helsinki

See on map

SME

The organization defined itself as SME (small and medium-sized enterprise) at the time the Grant Agreement was signed.

Yes
Region
Manner-Suomi Helsinki-Uusimaa Helsinki-Uusimaa
Activity type
Private for-profit entities (excluding Higher or Secondary Education Establishments)
Other funding
€ 0,00
INSTITUT NATIONAL DE L'AUDIOVISUEL
France
Net EU contribution
€ 212 528,75
Address
Avenue De L'europe 4
94366 Bry-sur-marne

See on map

Region
Ile-de-France Ile-de-France Val-de-Marne
Activity type
Higher or Secondary Education Establishments
Other funding
€ 0,00
LINGSOFT LANGUAGE SERVICES OY
Finland
Net EU contribution
€ 151 532,50
Address
Kauppiaskatu 5 A
20100 Turku

See on map

SME

The organization defined itself as SME (small and medium-sized enterprise) at the time the Grant Agreement was signed.

Yes
Region
Manner-Suomi Etelä-Suomi Varsinais-Suomi
Activity type
Private for-profit entities (excluding Higher or Secondary Education Establishments)
Other funding
€ 0,00