Flexible Voxels for Motion-Aware Videography
Proc. ECCV 2010
Post capture control of spatial resolution and frame-rate of videos, depending on the scene content. Applications in consumer imaging, scientific imaging and microscopy.
The goal of this work is to build video cameras whose spatial and temporal resolutions can be changed post-capture depending on the scene. Building such cameras is difficult due to two reasons. First, current video cameras allow the same spatial resolution and frame rate for the entire captured spatio-temporal volume. Second, both these parameters are fixed before the scene is captured. We propose different components of video camera design: a sampling scheme, processing of captured data and hardware that offer post-capture variable spatial and temporal resolutions, independently at each image location. Using the motion information in the captured data, the correct resolution for each location is decided automatically. Our techniques make it possible to capture fast moving objects without motion blur, while simultaneously preserving high-spatial resolution for static scene parts within the same video sequence. Our sampling scheme requires a fast per-pixel shutter on the sensor-array, which we have implemented using a co-located camera-projector system.
Given a fixed voxel budget, a high spatial resolution (SR) camera results in large motion blur and aliasing. A high-speed camera results in low SR even for the static/slow-moving parts of the scene (drums). With our sampling and reconstruction scheme, the spatio-temporal resolution can be decided post-capture, independently at each location in a content-aware manner: notice the reduced motion blur for the hands and high SR for the slow-moving and static parts of the scene.
Proc. ECCV 2010