Project description
New tools based on telecommunications and computer vision
Telecommunications and computer vision belong to distinct scientific fields. This will change with the creation of a new joint research field resulting from the advent of wireless communications based on millimetre-wave radios with line-of-sight operating ranges. This has opened the possibility of leveraging on computer vision data to predict wireless channel dynamics and help build high-definition 3D maps for localisation and sensing, while on the other hand, radio-based data/imaging can help computer vision applications become more resilient against occlusion and poor luminosity. The EU-funded CONVERGE project will develop an innovative toolset consisting of vision-aided antennas and base stations, a vision-radio simulator and 3D environment modeller, as well as machine learning algorithms for multimodal data, including communications signals and video streams.
Objective
Telecommunications and computer vision have evolved as separate scientific areas. This is envisioned to change with the advent of wireless communications with radios characterised by line-of-sight ranges which could benefit from visual data to predict the wireless channel dynamics. Computer vision applications will also become more robust if helped by radio-based imaging. This new joint research field relies on wireless communications, computer vision, sensing and machine learning, and it has a high innovation potential because of the large domain of innovative applications it enables and the relevant know-how available in Europe. However, the full potential of this new area can only be evaluated if adequate Research Infrastructures (RI) and tools are available.
The main objective of the CONVERGE project is the development of an innovative toolset aligned with the motto “view-to-communicate and communicate-to-view”. This toolset is a world-first and consists of vision-aided large intelligent surfaces, vision-aided fixed and mobile base stations, a vision-radio simulator and 3D environment modeler, and machine learning algorithms for multimodal data including radio signals, video streams, RF sensing, and traffic traces. This toolset will be deployed into 7 RIs mostly aligned with the ESFRI SLICES-RI and improve their competitiveness.
CONVERGE will also provide the scientific community with open datasets of experimental and simulated data obtained with the toolset in the RIs, meet scientific and industrial requirements by addressing relevant 6G verticals, enhance the competitiveness of the involved companies, extend the European influence to world-wide recognised RIs, enable the creation of new RIs, contribute to the development of new environment-friendly tools, and help European Union to address its societal challenges.
Fields of science
- natural sciencescomputer and information sciencesartificial intelligencecomputer vision
- engineering and technologyelectrical engineering, electronic engineering, information engineeringinformation engineeringtelecommunicationsradio technology
- natural sciencescomputer and information sciencesartificial intelligencemachine learning
Keywords
Programme(s)
Funding Scheme
HORIZON-RIA - HORIZON Research and Innovation ActionsCoordinator
4200 465 Porto
Portugal