Editing Near-eye light field display

A '''near-eye lightfield display''' is a kind of [[light field display]] that can be viewer near to the eye. It can be part of a [[display headset]].

A near-eye lightfield display ought to be made in pairs, as one is needed for each eye to have all depth cues.

Near-eye lightfield displays have been made in a variety of types, including microlens-based, sequential projection, and stacked LCD.

An example of a sequential projection display is from [[CREAL]].

A near-eye lightfield display may be paired with a computer system that automatically updates the rendering viewpoint according to positional and orientational sensor data, such as a [[virtual reality]] system or [[augmented reality]].

There may be a human visual system response similar to the [[vestibulo-ocular reflex]] in a system without positional tracking that can be counteracted using a long focal distance for something that is displayed. This concept can be built into a software system that has an IMU attached by disabling the nearest parts of the image if the head moves, to avoid VOR discomfort.

Near-eye light field displays introduce a light-field-based approach to NEDs. This allows for thinner and lighter HMDs capable of depicting accurate accommodation, convergence, and binocular-disparity [[depth cue]]s. The two human eyes perceive the world slightly differently. In the same way, light rays that enter the pupil at different location will encode a slightly different picture of the world being observed <ref name=”3”>Stanford University. Near-Eye Light Field Displays. Retrieved from https://talks.stanford.edu/douglas-lanman-near-eye-light-field-displays/</ref><ref name=”4”> Fattal, D. (2016). The ultimate guide to 3D technologies. Retrieved from https://thenextweb.com/insider/2016/04/23/guide-to-3d-tech/#</ref>. A light field is composed of all the light rays at every point in space travelling in every direction. It is a 4D data, since every point in three-dimensional space is attributed a direction. This concept came about in the 1990s as a solution to problems in computer graphics and vision <ref name=”5”> LightField Forum. Refocus your Eyes: Nvidia presents Near-Eye Light Field Display Prototype. Retrieved from http://lightfield-forum.com/2013/07/refocus-your-eyes-nvidia-presents-near-eye-light-field-display-prototype/</ref>. Near-eye light field displays must independently render light rays that are coming from every direction through every point in space in order to trigger accommodation. Sharp images from out-of-focus display elements are depicted by synthesizing these light fields that correspond to virtual scenes located within the viewer’s natural accommodation range (Figure 1). Lanman and Luebke (2013) mention that “conventional displays are intended to emit light isotropically. In contrast, a light ﬁeld display supports the control of tightly-clustered bundles of light rays, modulating radiance as a function of position and direction across its surface.”<ref name=”2”>Lanman, D. and Luebke, D. (2013). Near-Eye Light Field Displays. ACM Transactions on Graphics, 32(6)</ref><ref name=”3”></ref><ref name=”4”></ref>

Traditional HMDs only provide a single display plane; without a proper focus cue, the display decouples accommodations from the vergence of the eyes. Since there is a mismatch, the observer has to rely only on the binocular vision to perceive a 3D space. This can lead to visual discomfort, fatique, eye strain, and headaches <ref name=”6”> Stanford Computational Imaging Lab (2015). The Light Field Stereoscope - SIGGGRAPH 2015 [Video]. Retrieved from https://www.youtube.com/watch?v=YJdMPUF8cDM</ref>.

There have been demonstrations that light field displays allow for small form factors of NEDs. This was made by placing a microlens array on a small screen close to the eye. In near-eye light field displays the image created appears to be floating outside the physical gadget enclose, and the observer can accommodate with a narrow range. However, the lens used in the studies have a tradeoff between achieved spatial resolution and the supported depth range <ref name=”2”></ref><ref name=”7”> Huang, Fu-Chung, Chen, K. and Wetzstein, G. (2015). The Light Field Stereoscope: Immersive Computer Graphics via Factored Near-Eye Light Field Displays with Focus Cues. ACM Transactions on Graphics, 34(4)</ref>. Another technique used to implement light field displays is to stack liquid crystal displays (LCDs). In this case, the image formation is multiplicative, allowing for correct or nearly-correct focus cues to be supported over larger depth ranges. Alternatively, it reduces the number of required display planes <ref name=”7”></ref>.

With the [[Oculus Rift]], the commercial interest in HMDs increased. Indeed, over the last few years interest in VR has been increasing, both by researchers as well as consumers. NED technology has a vast range of applications besides gaming and entertainment. It can be applied in education, teleconferencing, scientific visualization, training and simulation, phobia treatment, and surgical training, for example. Immersive VR has also been shown to be effective in the treatment of post-traumatic stress disorder. For the continuing development of VR. It is essential to have a visually comfortable experience, such as diminishing the vergence-accommodation conflict that occurs in most HMDs. The improvement of light field displays is a path to the creation of better and more visually comfortable headsets <ref name=”2”></ref><ref name=”7”></ref><ref name=”8”> Huang, Fu-Chung, Chen, K. and Wetzstein, G. (2015). The Light Field Stereoscope | SIGGRAPH 2015. Retrieved from http://www.computationalimaging.org/publications/the-light-field-stereoscope/</ref>. Huang et al. (2015) wrote that “correct or nearly correct focus cues signiﬁcantly improve stereoscopic correspondence matching, 3D shape perception becomes more veridical, and people can discriminate different depths better. Vergence and accommodation cues are neurally coupled in the human brain; it seems intuitive that displays supporting all depth cues improve visual comfort and performance in long-term experiences.” <ref name=”7”></ref>
__NOTOC__
==Specifications==
To manufacture near-eye microlens array-based LFDs, it is recommended to use a large panel of high [[PPI]]. The display panel size is recommended to be about 2.5 inches square, with a PPI of over 2000. This results in a recommended resolution of 5000x5000 per eye.

90 degrees field of view or higher is recommended for a virtual reality scenario.

==Prototypes==
[[File:NE-LF prototype.png|thumb|NVIDIA’s near-eye light field display prototype (image: Lanman and Luebke, 2013)]]

[[File:Lightfield stereoscope.jpg|thumb|The [[light field stereoscope]] prototype (Image: Huang et al., 2015)]]

[[File:LFS images.jpg|thumb|Figure 4. Images with front and rear focus produced by the light field stereoscope (Image: Huang et al., 2015)]]

[[File:Nvidia-light-field-stereoscope-schematic.jpg|thumb|Figure 5. Light Field Stereoscope schematic (Image: fudzilla.com)]]

During the 2013 [[SIGGRAPH]] (Special Interest Group on Computer Graphics and Interactive Techniques) conference, NVIDIA showed the product of its research sector: a near-eye light field display prototype (Figure 2). It consisted of a pair of Sony [[ECX332A]] OLED micro-displays with a pixel density of about 2100 ppi. The display panels measured 15.36 x 8.64 mm, with a resolution of 1280 x 720 through 24-bit color pixels, and were installed on a glasses-like frame with a small box of electronics on top <ref name=”5”></ref><ref name=”9”> Steele, B. (2013). NVIDIA Research's near-eye light field display prototype eyes-on (video). Retrieved from https://www.engadget.com/2013/07/24/nvidia-research-near-eye-light-field-display-prototype/</ref>.

The microlens arrays mounted in front of the displays are used to convert pixels to individual light rays, generating a light field in front of the eye. It allows the viewer to focus at multiple depths and create a field of view of approximately 70 degrees. Users who experimented the prototype during the conference confirmed both aspects. Furthermore, they reported that despite being situated close to the eye, the prototype still provided some sharp images. Nevertheless, the proximity caused some pixel loss due to a decreased spacial resolution. Another interesting aspect of this prototype is that adjustments can be made, at the level of software, to take into account the user’s glasses or contacts prescription. The software is powered by NVIDIA GPUs and OpenGL <ref name=”5”></ref><ref name=”9”></ref>.

NVIDIA, in collaboration with Stanford Computational Imaging, presented a new near-eye display technology that supports focus cues (accommodation and retinal blur) and high image resolution during SIGGRPAH’s 2015 conference (Figure 3). The prototype was based on Wheatstone’s original stereoscope, augmented with modern factored light field synthesis via stacked liquid crystal panels <ref name=”8”></ref>. Huang et al. (2015) explain that “the light field stereoscope is a near-eye display that facilitates immersive computer graphics via stereoscopic image synthesis with correct or nearly correct focus cues. As opposed to presenting conventional 2D images, the display shows a 4D light ﬁeld to each eye, allowing the observer to focus within the scene. The display comprises two stacked liquid crystal displays (LCDs) driven by nonnegative light ﬁeld factorization.” (Figure 4) <ref name=”7”></ref> 

A light field is presented to each eye, providing a more natural viewing experience than conventional NEDs. The required field of view is very small (the size of the pupil), and it produces correct or nearly-correct focus cues. These cues are important for diminishing visual discomfort and contributing to comfortable, long-term immersive experiences. The developers of the [[light field stereoscope]] had the main goal of providing a practical, inexpensive display technology that supports focus cues in a wearable form factor <ref name=”7”></ref><ref name=”8”></ref>. 

NVIDIA showed a revision of its light field prototype during the Virtual Reality LA Expo of 2016, in Southern California. The unit was attached to a desktop PC equipped with a Maxwell-based GPU. To compute the light fields images in real-time, NVIDIA used an algorithm based on its CUDA parallel programming language for GPUs. Still in collaboration with Stanford University, the new prototype eliminates the headphone requirement with a VR headset design that does not produce motion sickness side effects. The light field stereoscope uses two layered displays with back-to-back LCD panels. These are in-between a backlight and two circular viewing lenses (Figure 5). The gadget creates a natural depth-of-field in each eye, combining it with a stereoscopic technique (showing images at slightly different angles in each eye) <ref name=”10”> Worrel, J. (2016). Nvidia shows off its 'Light Field' VR headset at VRLA 2016. Retrieved from http://www.fudzilla.com/news/graphics/39762-nvidia-shows-off-its-light-field-vr-headset-at-vrla-2016</ref>.

==Main characteristics of near-eye light field displays==

The main benefits of the near-eye light field displays are reduced thickness and weight. In the 2013 prototype this was achieved by substituting a compact microlens array for compound magnifying optics. Adding to this, the gadget addresses the accommodation-convergence conflict with binocular configurations by approximating retinal defocus blur. It also accounts for the user’s glasses or contact prescription through software <ref name=”2”></ref><ref name=”5”></ref>.

On the other hand, reduced spatial resolution was the primary limitation of this device. Lanman and Luebke (2013) indicated that the resolution is proportional to the ratio of the microlens focal length to the distance of the display from the eye. This means that a compact form factor, while desirable, results in a lower resolution. To have a higher resolution means you would have a bigger, bulkier display. Furthermore, they wrote that “practical applications will necessitate manufacturing larger microdisplays with smaller pixel pitches, enabling wide ﬁelds of view and high resolutions, respectively.” <ref name=”2”></ref>

The near-eye light field stereoscope prototype of 2015 also addresses the accommodation-convergence conflict that leads to motion sickness, resulting in a more natural and flexible experience. In this case, the gadget uses two layers of LCD separated by approximately 5 millimeters. These send an entire “4D” light field of images so the eye can set focus between objects more naturally <ref name=”11”> Pirzada, Usman (2015). Nvidia Working On Next Generation Virtual Reality Device; The Near Eye Light Field Stereoscope, Aims To Bring It To The Market by 2018. Retrieved from http://wccftech.com/nvidia-virtual-reality-device-light-field-2018/</ref>.

The resolution was also a limitation in this prototype. While the screen resolution was 1280x800, the HMD splits the screen into two halves, making the effective resolution decrease to 640x800 per eye. This is too low for a current setting, but it could be improved in the future since this is still in the prototype stage. Indeed, NVIDIA intends to bring this gadget into the market by 2018. While NVIDIA could be developing the new standard for HMDs, it could also mean that by the time the gadget is released to the market conventional VR makers may have solved the problems related to motion sickness in their own gadgets <ref name=”11”></ref>.

==References==
<references />

[[Category:Light field display]]
[[Category:Augmented reality]]