Optimized pixel streaming for ultra-low latency cloud gaming and VR/XR applications.

Until recently, top-notch gaming and XR/VR experiences required costly hardware with powerful GPUs. Today's cloud-based solutions aim to make these experiences more accessible by rendering the experience in the cloud and then “pixel streaming” - i.e., with one video encoding per each user - the point of view, but doing so hinges on delivering high-quality, ultra-low latency video, so compression efficiency is a core component of these applications. — While traditional codecs like H.264 and HEVC are currently used, they have limitations in cloud gaming and VR/XR scenarios. Newer codecs like AV1 and VVC offer better compression but are computationally heavy and produce lower gains vs HEVC in real-time ultra-low-latency applications. Enter MPEG-5 Part 2 LCEVC, a low-complexity enhancement that improves compression up to 30% when used with existing codecs like HEVC, AV1 or VVC, without adding computational burden and with coding tools that maintain their efficiency also in low-latency scenarios. Compatible with existing devices and codecs like HEVC, AV1, and VVC, LCEVC enables scalable, high-quality pixel streaming within the bandwidth limits necessary for widespread deployment. — This paper provides data on LCEVC's performance in conjunction with HEVC, specifically focusing on its application in ultra-low latency streaming for cloud gaming and VR/XR, where quality and responsiveness are key.

Published

2023-10

Content type

Original Research

Keywords

cloud gaming, VR, XR, MPEG-5 LCEVC, AV1, VVC, HEVC, H.264

DOI

10.5594/M002023

ISBN

978-1-61482-964-5