Recovery of 3-D Motion from Time-Varying Image Flows

In this paper we deal with the problem of recovering 3-D motion and structure from a time-varying 2-D velocity vector field. A great deal has been done on this topic, most of which has concentrated on finding necessary and sufficient conditions for there to be a unique 3-D solution corresponding to a given 2-D motion. While previous work provides useful theoretical insight, in most situations the known algorithms have turned out to be too sensitive to be of much practical use. It appears that any robust algorithm must improve the 3-D solutions over time. As a step toward such algorithm, we present a method for recovering 3-D motion and structure from a given time-varying 2-D velocity vector field. The surface of the object in the scene is assumed to be locally planar. It is also assumed that 3-D velocity vectors are piecewise constant over three consecutive frames (or two snapshots of flow field). Our formulation relates 3-D motion and object geometry with the optical flow vector as well as its spatial and temporal derivatives. The linearization parameters, or equivalently, the first-order flow approximation (in space and time) is sufficient to recover rigid body motion and local surface structure from the local instantaneous flow field. We also demonstrate, through a sensitivity analysis carried out for synthetic and natural motions in space, that 3-D motion can be recovered reliably.