Vincent Rabaud

Re-Thinking non-rigid structure from motion

Structure from motion (SFM) is the reconstruction in 3D of an object from a video sequence of tracked features. While the case of a rigid object is theoretically solved, the case of a non-rigid object (NRSFM) only knows one main approach. Several variations exist but they all rely on matrix decomposition and a purely geometrical interpretation of the problem. By interpreting the possible shapes an object can undergo as elements of a low-dimensional manifold, I will present the following two contributions: - when assuming the existence of a linear shape basis, the shape embedding can be recovered independently from the basis itself or the camera parameters. I will first present several MDS techniques before focusing on Generalized Non-metric Multi-Dimensional Scaling and how it can be used for NRSFM. - in the most general case, shapes can be assumed to belong to a low-dimensional non-linear manifold. If time permits, I will show how the manifold de-noising ability of Locally Smooth Manifold Learning (LSML) can be used to also solve for NRSFM.