Near-eye light field display

A near-eye lightfield display is a kind of light field display that shows a clear image even when it is close to the eye. It can be part of a display headset. It is a type of near-eye display that can generate accurate natural 3D visuals with proper human accommodation.

Depending on the type, it can be part of a monocle.

It can be made with a lens array. Near-eye lightfield displays have been made in a variety of types, including microlens-based, sequential projection, and stacked LCD. An example of a sequential projection display is from CREAL.

A near-eye lightfield display may be paired with a computer system that automatically updates the rendering viewpoint according to positional and orientational sensor data, such as a virtual reality system or augmented reality.

Sharp images from out-of-focus display elements are depicted by synthesizing these light fields that correspond to virtual scenes located within the viewer’s natural accommodation range. Lanman and Luebke (2013) mentioned that “conventional displays are intended to emit light isotropically. In contrast, a light field display supports the control of tightly-clustered bundles of light rays, modulating radiance as a function of position and direction across its surface.”

Traditional HMDs only provide a single display plane; without a proper focus cue, the display decouples accommodations from the vergence of the eyes. Since there is a mismatch, the observer has to rely only on the binocular vision to perceive a 3D space. This can lead to visual discomfort, fatique, eye strain, and headaches .

There have been demonstrations that light field displays allow for small form factors of NEDs. This was made by placing a microlens array on a small screen close to the eye. In near-eye light field displays the image created appears to be floating outside the physical gadget enclose, and the observer can accommodate with a narrow range. However, the lens used in the studies have a tradeoff between achieved spatial resolution and the supported depth range . Another technique used to implement light field displays is to stack liquid crystal displays (LCDs). In this case, the image formation is multiplicative, allowing for correct or nearly-correct focus cues to be supported over larger depth ranges. Alternatively, it reduces the number of required display planes .

Correct or nearly correct focus cues significantly improve stereoscopic correspondence matching. 3D shape perception becomes more veridical, and people can discriminate different depths better. Vergence and accommodation cues are neurally coupled in the human brain; it seems intuitive that displays supporting all depth cues improve visual comfort and performance in long-term experiences.

There may be a human visual system response similar to the vestibulo-ocular reflex in a system without positional tracking that can be counteracted using a long focal distance for something that is displayed. This concept can be built into a software system that has an IMU attached by disabling the nearest parts of the image if the head moves, to avoid VOR discomfort.

Specifications

To manufacture near-eye microlens array-based LFDs, it is recommended to use a large panel of high PPI. The display panel size is recommended to be about 2.5 inches square, with a PPI of over 2000. This results in a recommended resolution of 5000x5000 per eye.

A field of view of 90 degrees or higher is recommended for a virtual reality scenario.

Examples

Figure 4. Images with front and rear focus produced by the light field stereoscope (Image: Huang et al., 2015)

Figure 5. Light Field Stereoscope schematic (Image: fudzilla.com)

During the 2013 SIGGRAPH (Special Interest Group on Computer Graphics and Interactive Techniques) conference, NVIDIA showed the product of its research sector: a near-eye light field display prototype (Figure 2). This was Doug Lanman's binocular light field display prototype. It consisted of a pair of Sony ECX332A OLED micro-displays with a pixel density of about 2100 ppi. The display panels measured 15.36 x 8.64 mm, with a resolution of 1280 x 720 through 24-bit color pixels, and were installed on a glasses-like frame with a small box of electronics on top.

The microlens arrays mounted in front of the displays are used to convert pixels to individual light rays, generating a light field in front of the eye. It allows the viewer to focus at multiple depths and create a field of view of approximately 70 degrees. Users who experimented the prototype during the conference confirmed both aspects. Furthermore, they reported that despite being situated close to the eye, the prototype still provided some sharp images. Nevertheless, the proximity caused some pixel loss due to a decreased spacial resolution. Another interesting aspect of this prototype is that adjustments can be made, at the level of software, to take into account the user’s glasses or contacts prescription. The software is powered by NVIDIA GPUs and OpenGL .

NVIDIA, in collaboration with Stanford Computational Imaging, presented a new near-eye display technology that supports focus cues (accommodation and retinal blur) and high image resolution during SIGGRPAH’s 2015 conference (Figure 3). The prototype was based on Wheatstone’s original stereoscope, augmented with modern factored light field synthesis via stacked liquid crystal panels.

Huang et al. (2015) explain that “the light field stereoscope is a near-eye display that facilitates immersive computer graphics via stereoscopic image synthesis with correct or nearly correct focus cues. As opposed to presenting conventional 2D images, the display shows a 4D light field to each eye, allowing the observer to focus within the scene. The display comprises two stacked liquid crystal displays (LCDs) driven by nonnegative light field factorization.” (Figure 4)

A light field is presented to each eye, providing a more natural viewing experience than conventional NEDs. The required field of view is very small (the size of the pupil), and it produces correct or nearly-correct focus cues. These cues are important for diminishing visual discomfort and contributing to comfortable, long-term immersive experiences. The developers of the light field stereoscope had the main goal of providing a practical, inexpensive display technology that supports focus cues in a wearable form factor .

NVIDIA showed a revision of its light field prototype during the Virtual Reality LA Expo of 2016, in Southern California. The unit was attached to a desktop PC equipped with a Maxwell-based GPU. To compute the light fields images in real-time, NVIDIA used an algorithm based on its CUDA parallel programming language for GPUs. Still in collaboration with Stanford University, the new prototype eliminates the headphone requirement with a VR headset design that does not produce motion sickness side effects. The light field stereoscope uses two layered displays with back-to-back LCD panels. These are in-between a backlight and two circular viewing lenses (Figure 5). The gadget creates a natural depth-of-field in each eye, combining it with a stereoscopic technique (showing images at slightly different angles in each eye) .

Variables

Reduced resolution is a limitation of some near-eye light field displays. Douglas Lanman and Luebke (2013) indicated that the resolution is proportional to the ratio of the microlens focal length to the distance of the display from the eye. This means that a compact form factor, while desirable, results in a lower resolution. To have a higher resolution means you would have a bigger, bulkier display. Lanman and Luebke emphasized that it is important to manufacture large displays that have very small pixel pitch, which can be used for a high field of view and high resolution.

The near-eye light field stereoscope prototype of 2015 also addresses the accommodation-convergence conflict that leads to motion sickness, resulting in a more natural and flexible experience. In this case, the gadget uses two layers of LCD separated by approximately 5 millimeters. These send an entire four-dimensional (4D) light field of images so the eye can set focus between objects more naturally .