Final Report Summary - AVID-MODE (Efficient Location-Aware Audio-Visual Delivery of High-Quality Content to Mobile Devices)
The central objective of the AVID-MODE project (http://www.avid-mode.eu) is to enhance the delivery of multimedia content to mobile devices. Rich, immersive and portable multimedia is a much-needed feature that both industry and academia are pursuing. In order to achieve this objective, a number of problems need to be studied, understood and solved. First of all, a key aspect of portable devices is that they are lightweight and they have - compared to a desktop PC - limited computational power, memory and bandwidth. Moreover, their graphics/display and audio rendering capabilities are also - compared to home video-audio equipment - limited. Finally, but equally importantly, these devices cannot be too expensive. These constraints limit the range of current multimedia technologies that can be applied to the mobile world. An important aim of this project is to design, test and integrate novel, computationally efficient techniques that will present high-quality audio and video in portable devices, in a way to maximise the quality perceived by the user. At the same time, mobile devices can offer a feature that home-theatre environments cannot: they combine some kind of location-aware capability that, together with the use of a return-channel, allows the content to be customised and to be presented in an interactive manner.
The AVID-MODE project aims to:
1. Maximise audio & video quality for mobile devices, in the presence of errors, with computational constraints.
2. Combine GPS databases with DTV data services to allow customisation of such services.
3. Offer a sense of immersion in the multimedia content presentation, taking advantage of location-awareness.
The AVID-MODE project is oriented towards an area with great potential regarding research as well as commercial interest, which remains open for novel research, i.e. multimedia on mobile devices. Our aspirations are to derive immersive media capabilities on mobile devices, and to new integrated applications for such devices. The project aims to address issues related to many aspects of everyday activities, such as improved collaborative environments and entertainment. Therefore, our final and most ambitious objective in the AVID-MODE consortium is to derive technologies that will eventually improve the quality of everyday life.
In the four years of the project duration we performed research in a variety of fields, in audio and video signal processing, hardware and middleware issues for mobile devices, achieving the following results:
* Improvements to audio rendering of mobile devices by: (i) proposing improvements to parametric audio coding methods, through better parameter selection; (ii) implementing interactive 3D audio presentation capabilities on such devices; (iii) proposing a novel method for shifting the parametric audio codec complexity from the encoder to the decoder by using the emerging theory of Compressed Sensing.
* Design of a novel directional audio coding scheme enabling immersive communication and robust sound capturing using mobile devices.
* Interactive multimedia content presentation using mobile devices based on the GPS location and compass information.
* Optimization of the H.264 baseline decoder for portable devices; implementation of a comprehensive comparative study of major video codecs, including AVS - the Chinese video compression standard, revealing some of the latter’s key features.
* Testing of the new video coding standard, H.265 on mobile devices.
* Design of a novel hand-gesture recognition method, showing improved performance on limited-complexity devices.
* Improvements in the interference mitigation among the various modules of mobile devices (GPS, GSM, DVB-T/H receiver, WiFi, etc.).
* Integration of sensors with mobile devices which are normally available on boats such as speedometers, wind, temperature and humidity gauges
* Integration of all control and display capabilities of modern boats on portable, tablet-like devices.
* Design of a unified platform for digital TV and multimedia playback suitable for a variety of media platforms including mobile devices
* Development of novel methods for remote control communication of mobile devices.
* Analysis of acoustic echo cancellation and noise reduction for audio capture in noisy environments with a remote listener.
* Room and car equalization improvements for mobile audio playback.
Given the inter-sectorial nature of the AVID-MODE project, implemented via staff exchanges between academic and industrial partners, several of the developed technologies are close to commercial exploitation. Therefore, some of the abovementioned key results have hastened the possibility of today’s mobile device users of benefiting from a rich, immersive media experience.
For further information:
Prof. Athanasios Mouchtaris
Foundation for Research and Technology-Hellas (FORTH)
Institute of Computer Science (ICS)
N. Plastira 100
Heraklion, Crete, Greece, 70013
Tel.: +30-2810-391753
FAX: +30-2810-391601
email: mouchtar@ics.forth.gr
The AVID-MODE project aims to:
1. Maximise audio & video quality for mobile devices, in the presence of errors, with computational constraints.
2. Combine GPS databases with DTV data services to allow customisation of such services.
3. Offer a sense of immersion in the multimedia content presentation, taking advantage of location-awareness.
The AVID-MODE project is oriented towards an area with great potential regarding research as well as commercial interest, which remains open for novel research, i.e. multimedia on mobile devices. Our aspirations are to derive immersive media capabilities on mobile devices, and to new integrated applications for such devices. The project aims to address issues related to many aspects of everyday activities, such as improved collaborative environments and entertainment. Therefore, our final and most ambitious objective in the AVID-MODE consortium is to derive technologies that will eventually improve the quality of everyday life.
In the four years of the project duration we performed research in a variety of fields, in audio and video signal processing, hardware and middleware issues for mobile devices, achieving the following results:
* Improvements to audio rendering of mobile devices by: (i) proposing improvements to parametric audio coding methods, through better parameter selection; (ii) implementing interactive 3D audio presentation capabilities on such devices; (iii) proposing a novel method for shifting the parametric audio codec complexity from the encoder to the decoder by using the emerging theory of Compressed Sensing.
* Design of a novel directional audio coding scheme enabling immersive communication and robust sound capturing using mobile devices.
* Interactive multimedia content presentation using mobile devices based on the GPS location and compass information.
* Optimization of the H.264 baseline decoder for portable devices; implementation of a comprehensive comparative study of major video codecs, including AVS - the Chinese video compression standard, revealing some of the latter’s key features.
* Testing of the new video coding standard, H.265 on mobile devices.
* Design of a novel hand-gesture recognition method, showing improved performance on limited-complexity devices.
* Improvements in the interference mitigation among the various modules of mobile devices (GPS, GSM, DVB-T/H receiver, WiFi, etc.).
* Integration of sensors with mobile devices which are normally available on boats such as speedometers, wind, temperature and humidity gauges
* Integration of all control and display capabilities of modern boats on portable, tablet-like devices.
* Design of a unified platform for digital TV and multimedia playback suitable for a variety of media platforms including mobile devices
* Development of novel methods for remote control communication of mobile devices.
* Analysis of acoustic echo cancellation and noise reduction for audio capture in noisy environments with a remote listener.
* Room and car equalization improvements for mobile audio playback.
Given the inter-sectorial nature of the AVID-MODE project, implemented via staff exchanges between academic and industrial partners, several of the developed technologies are close to commercial exploitation. Therefore, some of the abovementioned key results have hastened the possibility of today’s mobile device users of benefiting from a rich, immersive media experience.
For further information:
Prof. Athanasios Mouchtaris
Foundation for Research and Technology-Hellas (FORTH)
Institute of Computer Science (ICS)
N. Plastira 100
Heraklion, Crete, Greece, 70013
Tel.: +30-2810-391753
FAX: +30-2810-391601
email: mouchtar@ics.forth.gr