Ira Kemelmacher

3D Shape Reconstruction from gray-level and two-tone images

Ira Kemelmacher-Shlizerman

Weizmann Institute of Science

Lighting has a significant effect on the appearance of objects. At the same time lighting provides information that can be used to recover the 3-dimensional shape of objects. It is long known that recovering shape from shading information is ill-posed, yet people appear to have a good sense of the underlying three-dimensional world already from a single picture. In this talk I will show how explicit modeling of lighting, along with the incorporation of prior knowledge, can facilitate 3D reconstruction.

In the first part of my talk I will introduce a novel algorithm for 3D shape recovery of faces from single images using a single 3D reference model of a different person's face. The method uses the input image as a guide to mold the reference model to reach a desired reconstruction. Assuming Lambertian reflectance and rough alignment of the input image and reference model, we seek shape, albedo, and lighting that best fit the image while preserving the rough structure of the model. We demonstrate our method by providing accurate reconstructions of novel faces overcoming significant differences in shape due to gender, race, and facial expressions.

I will then discuss 3D reconstruction from two-tone images (Mooney images). These images are fascinating testimonial to the ability of biological vision systems to accurately handle and interpret impoverished data. Their ambiguous nature, face specificity, and sudden interpretability have fascinated psychologists and neurobiologists throughout the past half a century and led to a flurry of studies. Our analysis indicates that reconstruction from such images is very ambiguous even if we consider only reconstruction along the "Mooney transition curve," the boundary curve between black and white pixels. However with prior knowledge it is possible to successfully recover the 3D shape of faces from single Mooney images.

This is joint work with Ronen Basri. Parts of the work were done in collaboration with Boaz Nadler (Weizmann) and David W. Jacobs (Maryland).