Broadcasters (DW, RAI) cooperated with the technical partners towards finalizing the user requirements for sports shooting. Three different shooting scenarios have been detailed. The user requirements were translated into system specifications in terms of drone hardware, ground station and software functionalities. Furthermore, the general architecture of cooperative mission planning and mission execution has been designed.
A specific novel language has been designed to allow the Director describe shooting missions. A centralized algorithm for planning the mission has been designed and on-drone software modules have been developed for mission execution. The tasks assigned to each drone are executed, when the associated event has been detected. In emergency situations, drones can compute a safe path to the closest landing spot. Drone cinematography has been modeled in terms of more than 20 shooting modes and various shot (framing) types. Furthermore, novel algorithms for formation control, autonomous trajectory tracking and multi-drone collision avoidance have been proposed. The semantic 3D map analysis and enrichment is combined with human crowd detection algorithms to provide semantic information (landing sites, crowd gathering regions etc). Research effort on human-centered visual information analysis focused on deriving novel lightweight deep learning architectures for cyclist detection, football player detection, boat detection, human crowd detection etc. Finally, novel research work has also been performed on visual quality assessment in various sports environments by employing realistic simulated videos and subjective testing.
The drone hardware incorporates on-board sensors and processing capabilities (2 processors) to fly autonomously. The MULTIDRONE platform software, which is the common interface for all modules, consists of ROS services and ROS message typesensuring system interoperability. Furthermore, the necessary flight supervisor and media (artistic) director GUIs have been designed. Shooting camera and navigation camera drone2ground video streaming transmitted over LTE/4G has been achieved successfully in media production scenarios.
The MULTIDRONE system was evaluated in three experimental media production meetings, two of them carried out in Germany (Bothkamp - Berlin) and one in Spain (Seville), in mock-up and real media production scenarios of a bike race, a rowing regatta and a parkour run. Several system integration issues were exposed and addressed on site or during consequent integration meetings. Overall, the results were very good; the system managed to fly autonomously and almost all system functionalities were successfully tested, while the system achieved to produce quality footage suitable for media production purposes.
MULTIDRONE dissemination and communication activities were diverse, multifaceted and can be considered very successful. They include publication (or acceptance) of 51 papers in high quality international scientific conferences and 29 papers in scientific journals, 27 keynote or invited speeches, 6 organized tutorias in prestigious conferences (e.g. ICCV) and a number of events (e.g. ERF, EBU, GMF) and a strong presence in social media. 7 exploitation planning events were held during the course of the project. 19 exploitable products and services (including SW code and binaries (mostly open SW), know-how, educational material, database/XML schemas and the drone platform design) were identified among the ones produced by the project and more than 150 accomplished industrial players were contacted.
