The Light Field Stereoscope | SIGGRAPH 2015

Fu-Chung Huang, Kevin Chen, Gordon Wetzstein

Inspired by Wheatstone’s original stereoscope and augmenting it with modern factored light field synthesis, we present a new near-eye display technology that supports focus cues. These cues are critical for mitigating visual discomfort experienced in commercially-available head mounted displays and providing comfortable, long-term immersive experiences.

Abstract

Over the last few years, virtual reality has re-emerged as a technology that is now feasible at low cost via inexpensive cellphone components. In particular, advances of high-resolution micro displays, low-latency orientation trackers, and modern GPUs facilitate extremely immersive experiences. To facilitate comfortable long-term experiences and wide-spread user acceptance, however, the vergence-accommodation conflict inherent to all stereoscopic displays will have to be solved. We present the first factored near-eye display technology supporting high image resolution as well as focus cues: accommodation and retinal blur. To this end, we build on Wheatstone’s original stereoscope but augment it with modern factored light field synthesis via stacked liquid crystal panels. The proposed light field stereoscope is conceptually closely related to emerging factored light field displays, but it has very unique characteristics compared to the television-type displays explored thus far. Foremost, the required field of view is extremely small – just the size of the pupil – which allows for rank-1 factorizations to produce correct or nearly-correct focus cues. We analyze distortions of the lenses in the near-eye 4D light fields and correct them using the high-dimensional image formation afforded by our display. We demonstrate significant improvements in resolution and retinal blur quality over previously-proposed near-eye displays. Finally, we analyze diffraction limits of these types of displays along with fundamental resolution limits.

FILES

technical paper (pdf)
technical paper supplement (zip)
presentation slides (slideshare)

CITATION

F. Huang, K. Chen, G. Wetzstein. “The Light Field Stereoscope: Immersive Computer Graphics via Factored Near-Eye Light Field Displays with Focus Cues”, ACM SIGGRAPH (Transactions on Graphics 33, 5), 2015.

BibTeX

@article{Huang:2015:LightFieldStereoscope,
author = {F. Huang and K. Chen and G. Wetzstein},
title = {{The Light Field Stereoscope: Immersive Computer Graphics via Factored Near-Eye Light Field Displays with Focus Cues}},
journal = {ACM Trans. Graph. (SIGGRAPH)},
issue = {34},
number = {4},
year = {2015},
}

Emerging Technologies Demo

Since we submitted this work to the SIGGRAPH 2015 Technical Papers program, Fu-Chung has moved on and is now working at NVIDIA Research. We have continued to improve the prototype together with NVIDIA and will demonstrate the technology live at SIGGRAPH 2015 Emerging Technologies.

Additional Material

Schematic of factored near eye light field display. Two stacked, transparent liquid crystal displays modulate the uniform backlight in a multiplicative fashion. When observed through a pair of lenses, the display provides focus cues in addition to binocular disparity afforded by conventional VR displays.

Photographs of the prototype when focusing on virtual objects located on the front and rear panels as well as in between. The proposed rank-1 light field factorization provides high-quality imagery even for virtual objects that are not located on the physical panels. (3D scene courtesy of Bushmills Irish Whiskey)

Prototype I. These is a photograph of our first light field stereoscope prototype. The housing is 3D-printed and based on Adafruit’s 3D-printed Video Goggles (https://learn.adafruit.com/3d-printed-wearable-video-goggles/overview). You can download our .stl files in the technical paper supplement above and print it yourself. The LCD panels and lenses are ordered from ebay. Details on those can be found in the technical paper.

Prototype II. Different perspectives of our second prototype light field stereoscope. More details on individual parts can be found in the technical paper. The .stl files for the housing are included in the technical paper supplement, so you can 3D print it yourself.

Prototype III. This is the third-generation prototype that Fu-Chung designed and fabricated at NVIDIA Research. We will be demonstrating this prototype at ACM SIGGRAPH 2015 Emerging Technologies in collaboration with NVIDIA.

Technical illustration of the proposed near-eye light field display system. Two attenuating spatial light modulators (SLMs), such as liquid crystal displays, are mounted inside the physical device (right, black outline). The cascaded SLMs modulate the backlight (not shown) in a multiplicative manner. When observed through a magnifying lens, virtual and magnified images of the SLMs are created – one close to optical infinity and the other close to the observer. The viewing frustra used for rendering computer-generated content are vertically symmetric but horizontally asymmetric, as illustrated in the bottom.

table comparing alternative technologies

Overview of closely-related, focus-supporting display technologies. The light field stereoscope is the first device offering high image resolution and focus cues in a small form factor. The asterisk indicates diffraction limits.

Left view of reconstructed light fields showing a scene with bright and dark objects in background and foreground, respectively. Additive multi-focal-plane displays cannot accurately reproduce monocular occlusions in this case (top row), because the bright light emitted by farther objects always shines through the darker ones in the foreground. Multiplicative layers accurately reproduce occlusions over the eye box (bottom row), thereby providing a better approximation of retinal blur for complex scenes. (3D scene courtesy of Paul H. Manning)

Front and rear focus of a stereo light field photographed with a Lytro Illum and displayed on a light field stereoscope. Additional results showing captured, factored, and displayed stereo light fields can be found in the technical paper supplement.

Related Projects

You may also be interested in related projects from our group on next-generation near-eye displays and wearable technology:

Y. Peng et al. “Neural Holography with Camera-in-the-loop Training”, ACM SIGGRAPH 2020 (link)
R. Konrad et al. “Gaze-contingent Ocular Parallax Rendering for Virtual Reality”, ACM Transactions on Graphics 2020 (link)
B. Krajancich et al. “Optimizing Depth Perception in Virtual and Augmented Reality through Gaze-contingent Stereo Rendering”, ACM SIGGRAPH Asia 2020 (link)
B. Krajancich et al. “Factored Occlusion: Single Spatial Light Modulator Occlusion-capable Optical See-through Augmented Reality Display”, IEEE TVCG, 2020 (link)
N. Padmanaban et al. “Autofocals: Evaluating Gaze-Contingent Eyeglasses for Presbyopes”, Science Advances 2019 (link)
K. Rathinavel et al. “Varifocal Occlusion-Capable Optical See-through Augmented Reality Display based on Focus-tunable Optics”, IEEE TVCG 2019 (link)
N. Padmanaban et al. “Optimizing virtual reality for all users through gaze-contingent and adaptive focus displays”, PNAS 2017 (link)
R. Konrad et al. “Accommodation-invariant Computational Near-eye Displays”, ACM SIGGRAPH 2017 (link)
R. Konrad et al. “Novel Optical Configurations for Virtual Reality: Evaluating User Preference and Performance with Focus-tunable and Monovision Near-eye Displays”, ACM SIGCHI 2016 (link)
F.C. Huang et al. “The Light Field Stereoscope: Immersive Computer Graphics via Factored Near-Eye Light Field Display with Focus Cues”, ACM SIGGRAPH 2015 (link)