Executive Summary : | In this project we propose to investigate the role of network science in 3D reconstruction problems. Independent of the method used, there exists a graph or network of cameras (called viewgraph) in all contexts of 3D reconstruction from images or depth maps. While the existing methods do not account for this viewgraph, we argue that it essential to understand the role and impact of the viewgraph on the 3D reconstruction problem.
Our objective is two-fold: a) understand the statistical properties of viewgraphs in real-world 3D reconstruction contexts of both Structure-from-motion (SfM) and Simultaneous Localization and Mapping (SLAM) and b) principled application of this understanding towards developing algorithms for 3D reconstruction.
While network science is well-studied in science and engineering contexts, there has been no work in using network principles in 3D reconstruction. The viewgraphs or camera networks that exist in real-world 3D reconstruction datasets have specific statistical properties that emerge out of the manner in which the data is acquired. The nature of the 3D reconstruction problem and the difficulty in solving it is significantly influenced by the viewgraph. However most 3D reconstruction methods do not take the viewgraph into account and use a number of engineering heuristics. We believe that a principled study of real-world viewgraphs can yield important insights. First, these insights have value since viewgraphs are intrinsically interesting objects of study. Second, such insights can be used to develop principled algorithms in 3D reconstruction problems.
Additionally, we seek to apply our understanding of network principles to the context of motion averaging. The existing state-of-the-art approach to rotation averaging that was proposed by the PI of this proposal is extensively used by the 3D reconstruction community. It is significantly faster and more accurate than other approaches. However we seek to use network principles to make further advances in making the rotation averaging method more efficient. This will be of importance given the rapidly growing sizes of image or depth datasets in SfM and SLAM problems.
In SLAM, long range loop closures play an important role in accurate camera motion estimation. Existing methods re-solve the entire camera motion trajectory to take new loop closure information into account. We seek to develop a principled theoretical understanding of the implications of loop closure. Apart from understanding the impact of loop closure in terms of viewgraph properties, we also seek to develop efficient methods that re-solve camera motions only when it is expected to be useful.
In summary, developing network science principles to study viewgraphs in 3D reconstruction problems are of intrinsic interest and value. Moreover, our knowledge of these network principles in 3D reconstruction will play the role of an important and principled tool for developing novel algorithms. |