SEI Message for Coding Video Containing Multiple Constituent Rectangles

The Constituent Rectangles Supplemental Enhancement Information (SEI) message is under development by the Joint Video Experts Team (JVET) for coding video formed from compositing multiple videos as constituent rectangles. Many applications require the use of multiple synchronized videos, for multi-view content or for different types of content, such as alpha channels, depth maps, objects masks, or image features. Use cases for the SEI message include VR/AR, 3D, free viewpoint video, game streaming, and cloud gaming. Pictures containing frame packed rectangular regions have previously been coded with existing single layer video codecs/profiles in an ad hoc non-standardized manner, without clear description within the video bitstream. The Constituent Rectangles SEI message provides a standardized way to describe a video containing pictures formed by compositing multiple rectangular videos of one or more types, using a single-layer video codec profile such as HEVC Main or VVC Main profile. The multiple videos otherwise could have been coded as separate pictures in layers of a multi-layer bitstream, requiring use of a multi-layer video codec profile. The Constituent Rectangles SEImessage is targeted for inclusion in a new version of the Versatile Supplemental Enhancement Information (VSEI) standard, a partner standard to the Versatile Video Coding (VVC) standard which defines SEI messages providing in-band metadata targeting a variety of use cases. SEI messages defined in VSEI may also be used with other video coding standards, including Advanced Video Coding (H.264/AVC) and High Efficiency Video Coding (HEVC).

Published

2024-10-21

Content type

Original Research

Keywords

supplemental enhancement information (sei), versatile supplemental enhancement information (vsei), versatile video coding (vvc), high efficiency video coding (hevc)

DOI

10.5594/MOO/3003

ISBN

978-1-61482-965-2