Minimizing user intervention in registering 2D images to 3D models

This paper proposes a novel technique to speed up the registration of 2D images to 3D models. This problem often arises in the process of digitalization of real objects, because pictures are often taken independently from the 3D geometry. Although there are a number of methods for solving the problem of registration automatically, they all need some further assumptions, so in the most general case the process still requires the user to provide some information about how the image corresponds to geometry, for example providing point-to-point correspondences. We propose a method based on a graph representation where the nodes represent the 2D photos and the 3D object, and arcs encode correspondences, which are either image-to-geometry or image-to-image point pairs. This graph is used to infer new correspondences from the ones specified by the user and from successful alignment of single images and to factually encode the state of the registration process. After each action performed by the user, our system explores the states space to find the shortest path from the current state to a state where all the images are aligned, i.e. a final state and, therefore, guides the user in the selection of further alignment actions for a faster completion of the job. Experiments on empirical data are reported to show the effectiveness of the system in reducing the user workload considerably.