Research Overview

Our research focuses on three core areas: neural rendering, computational imaging, and computational displays. We are particularly interested in developing next-generation imaging systems that combine advances in hardware and software, although many of us also focus on purely algorithmic work and applied mathematics that have practical applications computer graphics, vision, and imaging. Find a few of our recent projects below and on this YouTube video.

1. Neural Rendering

EG3D

3D GAN for photorealistic, multiview consistent, and shape-aware image synthesis.

pi-GAN

A 3D-aware GAN architecture using periodic implicit neural representations

SIREN

A neural signal representation for images, audio, shapes, and wavefields.

2. VR/AR Displays & Wearables

Event-based Near-eye Gaze Tracking

Ultra-low-latency and low-power eye tracking beyond 10,000 Hz

Neural Holography

AI-driven Holographic Near-eye Displays for VR/AR

Autofocals

Eye-tracked presbyopia correction that is better than progressives.

3. Deep Optics

Deep Optics for Depth Estimation

End-to-end optimization of optics and image processing for depth imaging.

Hybrid Optical-Electronic CNN

Hybrid optical-electronic convolutional neural networks for image classification.

End-to-end Optimization Optics and Image Processing

Achromatic EDOF and superresolution imaging.

4. Single-photon Time-of-Flight Imaging

Keyhole Imaging

Imaging 3D objects through the keyhole of a closed door.

Confocal Diffuse Tomography

Single-photon 3D imaging through densely scattering media

Wave-based Non-line-of-sight Imaging

Fast and robust wave-based NLOS imaging using f-k migration

5. Computational Microscopy

Deep Learning Cryo-EM

Fast Reconstruction of Poses and 3D Molecular Volumes for Cryo-EM.

Deep Learning Localization Microscopy

Deep learning multi-shot 3D localization microscopy using hybrid optical–electronic computing.

COSMOS

Fast, synchronous recording of cortex-spanning neural dynamics.

6. Computational Displays

Adaptive Color Displays

A projector with a dynamically-optimized color gamut.

Vision-correcting Displays

A display that corrects for the prescription of the user.

Tensor Displays

Compressive light field displays with stacked LCDs.