Timewarp
- See also: Asynchronous Spacewarp
General Information[edit]
Timewarp / Time warping also known as Reprojection is a technique in VR that warps the rendered image before sending it to the display to correct for the head movement occurred after the rendering.[1] Timewarp can reduce latency and increase or maintain frame rate. Additionally, it can reduce judder caused missed frames (when frames take too long to render).[2] This process takes the already rendered image, modify it with freshly collected positional information from your HMD's sensors, then display it to your screen. Utilizing depth maps (Z Buffers) already present in the engine, Timewarp requires very little computation.
Timewarp is a feature of Oculus SDK. It was initially released in the version 0.3.1 of the SDK. Positional Timewarp was introduced in Oculus PC SDK 0.6.0[3].
Runtimes for various platforms such as the Oculus Runtime and SteamVR Runtime are responsible for Timewarp. Different runtimes perform timewarp differently.
Asynchronous Timewarp (ATW)[edit]
Asynchronous Timewarp or ATW is when timewarp occurs on another thread in parallel (asynchronously) with rendering. Before every vsync, the ATW thread generates a new timewarped frame from the latest frame completed by the rendering thread.[1] ATW fills in the missed frames and reduces judder.[4]
How Timewarp Reduces Latency[edit]
Without Timewarp, your HMD would capture the data about the position of your head, render the image based on this data (correct angle etc.), then display the image when the next scene is due to be on screen. In a 60 fps game, a new scene is displayed once every 16.7 milliseconds. With this process, each image you see is based on the head-tracking data from almost 17 milliseconds ago.
With Timewarp, the first 2 parts of the process is the same. your HMD would capture the data about the position of your head and render the image based on the data. Before this image is displayed, your HMD captures the position of your head again. Using this information, the rendered image is modified with a mathematical calculation to fit the latest data. Finally the modified image is displayed on screen. The resulting image is more recent and more accurately depict the position of your head at the time of display than the image initially rendered. Timewarp only works in very short distances and time intervals or the resulting image will look unrealistic or out of place.
How Timewarp Increases Frame Rate[edit]
Timewarp allows engines to increase or maintain frame rate when they are otherwise unable to do. It does this by artificially filling in dropped frames.
In a game engine limited to 50 frames per second, a new frame is displayed once every 20 milliseconds. To increase the game's frame rate to 60, you need to display a new frame once every 16.7 milliseconds. To increase the fps through timewarp, the last completely rendered frame, not the one that is currently rendering, is updated with the latest data about the position of your head. The modified frame is displayed before your eyes.[2] Theoretically, this method could increase frame rate from 1 to 60. Realistically, timewarp only works in very short distances and time intervals. Long distances or time intervals will make the image appear unrealistic or out of place.
Limitations of Timewarp[edit]
Timewarp works well in a rotational only tracking system. When positional / translational tracking is introduced, it does not work as well and presents a few problems.[1] When timewarp is used with positional tracking, only the rotational change is reflected in the resulting image. The positional change is ignored, causing judder. This is solved through the introduction of Positional Timewarp.
Moving objects such as an animated character can also cause judder. The notice-bility of the judder depends on the size, number and speed of the moving object.[1] Large and fast moving objects can cause very noticeable judder.
Positional Timewarp[edit]
Positional Timewarp, also known as Asynchronous Spacewarp (ASW), applies animation detection, camera translation, and head translation to previous frames in order to predict the next frame. ASW expends upon Asynchronous Timewarp (ATW), which only applies to the rotational tracking of the head.
Oculus Rift vs. SteamVR vs. PSVR[edit]
The Oculus Runtime uses asynchronous timewarp to interpolate frames from any frame rate to 90. I.e. the game has a massive explosion, fps drop to 70, async timewarp interpolates to 90.
The SteamVR Runtime uses reprojection when the target frame rate is not met. But instead of interpolating from an arbitrary frame rate to 90, it forces the game to render at 45fps and then doubles the frame rate via interpolation.
Both approaches have advantages and disadvantages. Asynchronous timewarp is great for very short dips below 90, but it's variability means frame times aren't that smooth. SteamVR's reprojection has very smooth and predictable frame times, however the fps hit is much higher, which could look weird if animations and physics are meant to run with 90Hz.
Sony's PSVR is using SteamVR style reprojection as one of their default modes. Games can render at 60fps and will be reprojected to 120.