Holographic Near-Eye Displays Based on Overlap-Add Stereograms | SIGGRAPH Asia 2019

Nitish Padmanaban, Yifan (Evan) Peng, Gordon Wetzstein

The overlap-add stereogram (OLAS) algorithm uses overlapping hogels to encode the view-dependent lighting effects of a light field into a hologram achieving better quality than other holographic stereograms.

Holographic Near-Eye Displays Based on Overlap-Add Stereograms

ABSTRACT

Holographic near-eye displays are a key enabling technology for virtual and augmented reality (VR/AR) applications. Holographic stereograms (HS) are a method of encoding a light field into a hologram, which enables them to natively support view-dependent lighting effects. However, existing HS algorithms require the choice of a hogel size, forcing a tradeoff between spatial and angular resolution. Based on the fact that the short-time Fourier transform (STFT) connects a hologram to its observable light field, we develop the overlap-add stereogram (OLAS) as the correct method of “inverting” the light field into a hologram via the STFT. The OLAS makes more efficient use of the information contained within the light field than previous HS algorithms, exhibiting better image quality at a range of distances and hogel sizes. Most remarkably, the OLAS does not degrade spatial resolution with increasing hogel size, overcoming the spatio-angular resolution tradeoff that previous HS algorithms face. Importantly, the optimal hogel size of previous methods typically varies with the depth of every object in a scene, making the OLAS not only a hogel size–invariant method, but also nearly scene independent. We demonstrate the performance of the OLAS both in simulation and on a prototype near-eye display system, showing focusing capabilities and view-dependent effects.

Technical Paper
Supplemental PDF
Source Code
Light Field Dataset

CITATION

N. Padmanaban, Y. Peng, G. Wetzstein. “Holographic Near-Eye Displays Based on Overlap-Add Stereograms”, ACM SIGGRAPH Asia (Transactions on Graphics 38, 6), 2019.

BibTeX

@article{Padmanaban:2019:OLAS,
author = {N. Padmanaban and Y. Peng and G. Wetzstein},
title = {{Holographic Near-Eye Displays Based on Overlap-Add Stereograms}},
journal = {ACM Trans. Graph. (SIGGRAPH Asia)},
issue = {38},
number = {6},
year = {2019},
}

 


 

Example Scene: Focal stacks for a selected scene at two different distances. It can be seen in the zoomed views to the right of the simulated and captured results that near and far objects come into focus at the correct near and far focus distances.

Mirror Example: Experimental results demonstrating a non-Lambertian surface. The depth map of the mirror is rendered as a flat plane, as expected. However, using information in the light field, holographic stereograms like the OLAS can recover some depth for the hologram.

Parallax Example: Experimental results demonstrating parallax. We compare the Fresnel method against the OLAS. In general, evidence of parallax can be seen with both methods; however, the OLAS performs better.

Prototype: A photograph of the prototype holographic near-eye display system showing the optical path.

Stereograms, Light Fields, and the STFT: Stereograms approximate the phase profile of the holographic wavefront with discrete plane waves, with their direction varying based on object distance. The light field, which can be thought of the geometric equivalent to the hologram, can be visualized as an epipolar image by plotting each ray’s angle and position. It turns out that the slope of the lines in the epipolar image is related to the stereogram plane wave directions via an STFT.

Resolution Comparison: Comparison of the APAS and OLAS algorithms and hogel sizes for a simulated resolution chart at a small distance to the hologram plane. The hogel size is indicated to the left of each row, and the red square in the top left corner shows the hogel relative to the image for scale. For the APAS, the indicated hogel size corresponds to the output hogel size after cropping. The “APAS 1×1” uses the indicated hogel size as the input size, cropped to 1×1. The OLAS algorithm provides the best performance at all hogel sizes by making full use of the angular information when inverting the STFT. The overlapping hogels do not noticeably degrade resolution with larger hogels.

Related Projects

You may also be interested in related projects from our group on holographic near-eye displays:

  • Y. Peng et al. “Neural Holography with Camera-in-the-loop Training”, ACM SIGGRAPH 2020 (link)

and other next-generation near-eye display and wearable technology:

  • R. Konrad et al. “Gaze-contingent Ocular Parallax Rendering for Virtual Reality”, ACM Transactions on Graphics 2020 (link)
  • B. Krajancich et al. “Optimizing Depth Perception in Virtual and Augmented Reality through Gaze-contingent Stereo Rendering”, ACM SIGGRAPH Asia 2020 (link)
  • B. Krajancich et al. “Factored Occlusion: Single Spatial Light Modulator Occlusion-capable Optical See-through Augmented Reality Display”, IEEE TVCG, 2020 (link)
  • N. Padmanaban et al. “Autofocals: Evaluating Gaze-Contingent Eyeglasses for Presbyopes”, Science Advances 2019 (link)
  • K. Rathinavel et al. “Varifocal Occlusion-Capable Optical See-through Augmented Reality Display based on Focus-tunable Optics”, IEEE TVCG 2019 (link)
  • N. Padmanaban et al. “Optimizing virtual reality for all users through gaze-contingent and adaptive focus displays”, PNAS 2017 (link)
  • R. Konrad et al. “Accommodation-invariant Computational Near-eye Displays”, ACM SIGGRAPH 2017 (link)
  • R. Konrad et al. “Novel Optical Configurations for Virtual Reality: Evaluating User Preference and Performance with Focus-tunable and Monovision Near-eye Displays”, ACM SIGCHI 2016 (link)
  • F.C. Huang et al. “The Light Field Stereoscope: Immersive Computer Graphics via Factored Near-Eye Light Field Display with Focus Cues”, ACM SIGGRAPH 2015 (link)