This work presents the first single-lens wide-field-of-view (FOV) light field (LF) camera. Shown below is a pan through a 138°, 72-MPix LF captured using the optical prototype. Shifting the virtual camera position over a circular trajectory during the pan reveals the parallax information in the LF. There is no complex post-processing or alignment between fields, this is the raw light field as measured by the camera.
Both wide-FOV and LF capture have been shown to simplify and enhance a range of tasks in computer vision, and we expect their combination to find application spanning autonomous vehicles, virtual and augmented reality capture, and robotics in general.
ABSTRACT
Light field (LF) capture and processing are important in an expanding range of computer vision applications, offering rich textural and depth information and simplification of conventionally complex tasks. Although LF cameras are commercially available, no existing device offers wide field-of-view (FOV) imaging. This is due in part to the limitations of fisheye lenses, for which a fundamentally constrained entrance pupil diameter severely limits depth sensitivity. In this work we describe a novel, compact optical design that couples a monocentric lens with multiple sensors using microlens arrays, allowing LF capture with an unprecedented FOV. Leveraging capabilities of the LF representation, we propose a novel method for efficiently coupling the spherical lens and planar sensors, replacing expensive and bulky fiber bundles. We construct a single-sensor LF camera prototype, rotating the sensor relative to a fixed main lens to emulate a wide-FOV multi-sensor scenario. Finally, we describe a processing toolchain, including a convenient spherical LF parameterization, and demonstrate depth estimation and post-capture refocus for indoor and outdoor panoramas with 15 x 15 x 1600 x 200 pixels (72 MPix) and a 138° FOV.
Collaborators
Joseph Ford and Glenn Schuster, Photonic Systems Integration Laboratory, UC San Diego.
External Links
Project page with more detail featuring panoramas, refocus, super-resolution, depth estimate, and hardware photo gallery.
FILES
- technical paper ( pdf )
- technical paper supplementary videos ( zip )
Citation
D. G. Dansereau, G. Schuster, J. Ford, and G. Wetzstein, “A wide-field-of-view monocentric light field camera”, in Computer Vision and Pattern Recognition (CVPR), in Press, 2017.
BibTeX
@inproceedings{dansereau2017wide,
title={A Wide-Field-of-View Monocentric Light Field Camera},
author={Donald G. Dansereau and Glenn Schuster and Joseph Ford and Gordon Wetzstein},
booktitle={Computer Vision and Pattern Recognition ({CVPR})},
year={2017},
month={Jul.},
organization={IEEE},
URL={http://dgd.vision/Papers/LFMonocentric.pdf}
}
Acknowledgments
We thank Kurt Akeley and Lytro for a hardware donation that enabled this work. This work is supported by the NSF/Intel Partnership on Visual and Experiential Computing (Intel #1539120, NSF #IIS-1539120). The authors thank Google ATAP for providing the Omnivision sensor interface, and Julie Chang and Sreenath Krishnan for their help with early optical prototypes. The monocentric lenses used in this work were fabricated within the DARPA SCENICC research program.
