Gallery of Results (Structured Light in Global Illumination)

Concave bowl on a marble slab. There are inter-reflections inside the concave bowl and sub-surface scattering on the translucent marble slab.

Image captured with a low-frequency projected pattern. Due to inter-reflections, scene points that are not directly lit have a large radiance. This results in structured light decoding errors.

Image captured with a high-frequency projected pattern. Due to sub-surface scattering on the marble slab, the high frequency pattern is blurred. Consequently, this image are not decoded accurately.

Using conventional Gray codes results in errors due to inter-reflections. Modulated phase shifting relies on explicitly separating the direct and the global illumination components. Hence, it suffers from low signal to noise ratio due to low direct component on the marble slab. Our result using an ensemble of codes has significantly fewer errors.

Interreflections result in errors for structured light patterns with low spatial frequencies (see paper for details). To prevent errors due to interreflections, structured light patterns with only high spatial frequencies must be used. Existing patterns (phase shifting, conventional Gray codes) have patterns with a range of spatial frequencies. We show that by using simple logical operations, codes with only high spatial frequencies can be constructed. Below we show an example with a V-groove scene (see paper for details).

Local effects such as sub-surface scattering and defocus result in blurring of incident illumination. For such effects, patterns with low spatial frequencies must be used. We used tools from combinatorial mathematics literature to design binary patterns with high minimum stripe width (low spatial frequencies) as shown below. Note the distribution of stripe widths for different codes.

For most scenes, we do not have a priori knowledge of the form of global illumination effects. Moreover, many scenes can have both interreflections (long-range) and local effects. For such scenes, we project an ensemble of codes and perform a simple consistency check.

For example, we project four codes - two sets of logical codes (optimized for interreflections) and two sets optimized for local effects.

For each scene point, we get four different depth values (as shown below). The key idea is that if the codes make errors, they are random errors. However, if any two agree, with high probability they will agree on the correct value.

Due to interreflections, the marked scene point is brighter when not directly lit (inverse pattern) as compared to when directly lit. This results in a binarization error as shown on the right. Such errors can happen for low-frequency projected patterns.

High frequency patterns are binarized correctly in the presence of interreflections. See paper for explanation

By using simple logical operations, codes with only high spatial frequencies can be constructed from conventional codes.

Since conventional codes make errors for low frequency patterns (high significance bits), the resulting error in depth maps is large. Our logical codes produce a nearly error free depth map while requiring the same number of images as conventional codes.

This scene has both inter-reflections (corner of the fruit-basket) and sub-surface scattering on the fruits. Conventional Gray codes and phase-shifting result in errors due to inter-reflections. On the other hand, modulated phase shifting produces errors on the translucent fruits due to low direct component. Our technique using an ensemble of codes results in significantly fewer errors.

Errors due to inter-reflections

Errors due to inter-reflections

Errors due to sub-surface

3D reconstructions using the code ensemble

Depth-maps

3D reconstructions using the code ensemble

Depth-maps

The goal here is to reconstruct the shape of the shown Ikea lamp. The lamp is made of thin translucent paper. Consequently, light diffuses inside the lamp, bounces around and comes back out. Because of this, conventional Gray codes result in errors near the periphery of the lamp.

Depth-maps

The paper cups have glazed white interiors, resulting in strong high-frequency inter-reflections. Consequently, conventional Gray codes result in significant errors. Because of high-frequency nature of inter-refletions, modulated phase-shifting also produces incorrect results. Depth map using our ensemble of codes has considerably fewer errors.

Depth-maps

Depth-maps

This object has points that receive both short and long range effects. Points inside the bowl receive interreflections and strong subsurface scattering. Since the interreflections are weak (the bowl is shallow), the code ensemble produces an accurate shape.

Depth-maps

This object illustrates a failure case. The object is a deep container made of highly translucent wax. Points inside the container receive both strong interreflections and strong subsurface scattering. Since none of the four codes compute the correct shape, the code ensemble fails to reconstruct the object.

Reconstructed Shapes

Light diffuses through the curtain and is reflected from the background, creating long-range optical interactions. Consequently, conventional Gray codes and phase shifting result in large errors and holes in the estimated shape. The correct shape of the curtain is nearly planar, with small ripples. Reconstruction using our logical XOR-04 codes is nearly error free, with the same number of input images as the conventional Gray codes.