The project combines
- new research topics (rigid moving objects in Structure-from-Motion aka “multi-body”, register one image to untextured 3d model)
- new contributions to well-known research topics (uncertainty evaluation of the estimated parameters in Structure-from-Motion)
- high performance research on well-known computer vision (SIFT feature extraction on GPU, GPU feature matching, GPU depth map computation, PCIe-networking for multi-GPU)
- industrial transfer of academic results with engineering refinements to further improve speed and quality (high quality Structure-from-Motion and Multiple View Stereo pipeline, integration of Local Bundle adjustment from SLAM approaches into SfM, integration of 360° cameras and cameras rigs). The industrial and academic partnership has also delivered an open source nodal user interface that enables advanced users and academics to customize their 3D reconstruction pipeline. We hope that it will enable more reproducible research in the field of photogrammetry.
New research topics
New research has been done on rigid moving objects in Structure-from-Motion (aka “multi-body”). This work has delivered a new generic feature matching strategy that can be used beyond the multi-body case and has been integrated into the pipeline. Other work on multi-body are still ongoing. New research was conducted and published on the registration of RGB image to untextured 3d models, new camera solvers that deal with unknown focal length and/or significant distortion, and the selection of image subsets using machine learning as a new solution to improve the current use of the vocabulary tree approach. Also directly determining camera poses from image collection was explored.
New contributions to well-known research topics
A first solution for uncertainty evaluation of large scale 3D reconstruction has been delivered in 2017 and has been integrated into the pipeline. A new publication has been done in 2018 with a new approach that allows to deliver more accurate and faster results. New approaches have been implemented for 3d-3d registration with unknown scales, regarding point cloud alignment of an image-based SfM+MVS to LiDAR or Kinect point cloud.
High performance research
SIFT feature extraction on GPU provides a drop-in replacement for CPU implementations. Feature matching algorithms that win in terms of speed over the best approximate algorithm. GPU depth map computation was revised and low-latency PCIe-networking has been adapted to scale GPU computation transparently to several IOMMU-capable PCs.
Socio-economic impact
The LADIO data collection on set through the QuineBox and automated data delivery into the further production workflows provides an integrated production experience to QUI's customers. The QuineBox has a unique role for automated on-set data acquisition, both from recording devices and other sensors. The LADIO data model establishes relationships and maintains coherence between assets. It enables the development of frontends for assessing, annotating, importing and exporting all kinds of data for a modern production in the media and entertainment industry.
AliceVision (
https://alicevision.github.io(opens in new window)) is an open source framework that was set up by the LADIO partners to provide a free 3D reconstruction pipeline. By releasing AliceVision in open source, the LADIO partners have set up a collaborative framework with academic and industrial partners. It allows the partners to build a cutting-edge pipeline for Visual Effects based upon a state-of-the-art set of software libraries, and is an enabler for other communities. The platform showcases the LADIO partners' improvements to the state-of-the-art in 3D reconstruction and enables also for the future reproducible research on Structure from Motion and Multiple View Stereo for the entire research community.
