Video-based AR

Passthrough, also known as "video passthrough", is a capability of mixed reality/virtual reality headsets where they can display the outside world by using cameras to the user. This makes it a sort of pseudo-augmented reality.

One of the difficulties of getting this right is the positional difference between cameras on the outside of a headset and the eyes of the user. So the image has to go through a distortion process. On the Quest Pro and Quest 3, this uses a neural network. Other devices potentially also use a neural network.