AV2 Codec ISO Media File Format Binding

1. Terminology

The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in BCP 14 [RFC2119] [RFC8174] when, and only when, they appear in all capitals, as shown here.

All of the text of this specification is normative except sections explicitly marked as non-normative, examples, and notes.

Examples in this specification are introduced with the words "for example" or are set apart from the normative text with class="example", like this:

Informative notes begin with the word "Note" and are set apart from the normative text with class="note", like this:

NOTE: This is an example of an informative note.

↑ Back to Table of Contents

2. Bitstream features overview

An AV2 bitstream is composed of a sequence of OBUs (Open Bitstream Units), grouped into Temporal Units. A Temporal Unit consists of a Temporal Delimiter OBU and all following OBUs up to, but not including, the next Temporal Delimiter OBU. Each Temporal Unit is associated with a single presentation time.

Every OBU begins with a 1- or 2-byte header that carries the obu_type and the temporal layer identifier obu_tlayer_id. When the OBU header extension is present (i.e. when obu_header_extension_flag is equal to 1 in that header), two additional layer identifiers are carried: the embedded layer identifier obu_mlayer_id and the extended layer identifier obu_xlayer_id. Depending on its type, an OBU can carry configuration information (e.g., a Sequence Header OBU or Layer Configuration Record OBU), metadata, or coded video data.

A Temporal Unit MAY contain multiple coded pictures but only one is output. AV2 defines a number of coded-frame OBU types, including Closed Loop Key OBUs, Open Loop Key OBUs, regular and leading tile-group OBUs, Switch OBUs, OBU_RAS_FRAME, Bridge Frame OBUs, and TIP (Temporally Interpolated Prediction) frames.

Frames that can be decoded without reference to other frames are Key Frames (carried in a Closed Loop Key OBU or an Open Loop Key OBU) and Intra frames. All other coded frames - including Inter Frames, Switch Frames, RAS frames, Bridge frames, TIP frames, and frames with show_existing_frame equal to 1 - have coding dependencies on other frames in the same or previous Temporal Units.

A Closed Random Access is the random access process that applies to an extended layer whose first coded frame is a Key Frame carried in a Closed Loop Key OBU. It starts a new coded video sequence for the extended layer; the layer’s reference frame buffers are invalidated and every subsequent frame of that layer can be decoded. This is the primary random-access mechanism.

An Open Random Access is the random access process that applies to an extended layer whose first coded frame is a Key Frame carried in an Open Loop Key OBU. During sequential decoding, an Open Random Access point does not start a new coded video sequence; however, when a decoder initiates decoding at that point, the decoding process is treated as if it were the start of a new coded video sequence for the extended layer. Leading frames that follow an Open Random Access point MAY reference frames that precede it, and are discarded when decoding begins at the Open Random Access point.

In addition to Key Frames, AV2 provides two further mechanisms that support random access or stream switching: a RAS frame (OBU_RAS_FRAME) is an inter-predicted frame that uses long-term reference frames and enables random access or switching without inserting a full key frame; a Switch Frame (OBU_SWITCH) is an inter-predicted frame that enables switching between representations at a given point, typically in adaptive streaming scenarios.

AV2 supports three scalability dimensions: temporal layers (identified by obu_tlayer_id), embedded layers (identified by obu_mlayer_id), and extended layers (identified by obu_xlayer_id). The reserved obu_xlayer_id value GLOBAL_XLAYER_ID (equal to 31, as defined in [AV2]) denotes global scope and is used by OBUs that do not belong to any specific extended layer (such as the Temporal Delimiter OBU). A bitstream that uses a single distinct non-global value of obu_xlayer_id is a Singlestream bitstream; a bitstream that uses two or more distinct non-global values is a Multistream bitstream.

AV2 defines two distinct Metadata OBU types for carrying metadata: OBU_METADATA_SHORT, using the metadata short OBU syntax, and OBU_METADATA_GROUP, using the metadata group OBU syntax. A separate OBU type carries film grain synthesis parameters; its presence is indicated by the film_grain_params_present flag in the Sequence Header OBU.

The Sequence Header OBU also carries the bitstream’s profile, level, and - when applicable - tier, as defined in Annex A of [AV2].

The Sequence Header OBU carries a monotonic_output_order_flag that defines the output mode of the associated coded video sequence.

When monotonic_output_order_flag is equal to 1, the output order of coded output frame equals their decoding order within the coded video sequence. Within a sample, the composition time (CTS) of the associated output frame is equal to the sample’s decoding time (DTS); the sample MAY additionally contain hidden frames.

When monotonic_output_order_flag is equal to 0, the output order of coded output frames MAY differ from their decoding order. The composition time of each sample is then derived from a ctts box that signals the offset between the sample’s decoding time (DTS) and the composition time of its associated output frame. As in the monotonic case, a sample MAY additionally contain hidden frames.

NOTE: Non-monotonic output mode is new in AV2; AV1 supports only the monotonic mode.

↑ Back to Table of Contents

3. Basic Encapsulation Scheme

This section describes the basic data structures used to signal encapsulation of AV2 bitstreams in [ISOBMFF] containers.

There are two locations where OBUs can be found in a file conformant to this specification:

• Some High-Level Syntax OBUs, such as the Sequence Header OBU, can be carried in an OutOfBandAV2CodecConfigurationBox as described in § 3.2 AV2 Sample Entry and Configuration. This corresponds to "availability through external means" as described in the [AV2] specification. These OBUs apply to all samples of the corresponding AV2SampleEntry.

• The other OBUs are stored in samples. This corresponds to "presence in the bitstream" as described in the [AV2] specification.

An AV2 sample is the union of a sample inclusive of the OBUs it carries, any OBUs carried in the AV2CodecConfigurationBox of the corresponding AV2SampleEntry, and an implicit Temporal Delimiter OBU.

NOTE: This specification refers to samples as defined in [ISOBMFF], which are not to be mistaken with samples or sample values as defined in the [AV2] specification.

3.1. General Requirements & Brands

A file conformant to this specification satisfies the following:

• It SHALL conform to the normative requirements of [ISOBMFF]
• It SHALL have the av02 brand among the compatible brands array of the FileTypeBox
• It SHALL contain at least one track using an AV2SampleEntry, possibly transformed by encryption as specified in § 5 Common Encryption
• It SHOULD indicate a structural ISOBMFF brand among the compatible brands array of the FileTypeBox, such as iso6
• It SHALL contain at least one AV2 sample
• Each AV2 sample it contains SHALL form exactly one [AV2] Temporal Unit if reconstructed as described in § 3.3.7 AV2 sample reconstruction

3.2. AV2 Sample Entry and Configuration

3.2.1. AV2 Sample Entry

The AV2SampleEntry extends VisualSampleEntry and signals a track carrying AV2 samples. There are two AV2SampleEntry flavours.

abstract class AV2SampleEntry(int codingname) extends VisualSampleEntry(codingname) {
}

class OutOfBandAV2SampleEntry extends AV2SampleEntry('av2E') {
    OutOfBandAV2CodecConfigurationBox config;
}

class InBandAV2SampleEntry extends AV2SampleEntry('av2e') {
    InBandAV2CodecConfigurationBox config;
}

An OutOfBandAV2SampleEntry contains an OutOfBandAV2CodecConfigurationBox and an InBandAV2SampleEntry contains an InBandAV2CodecConfigurationBox.

An InBandAV2SampleEntry MAY be used instead of an OutOfBandAV2SampleEntry, if the content of an OBU of a type that may be carried by an OutOfBandAV2CodecConfigurationBox changes during the sequence, and if that sequence cannot be split into multiple OutOfBandAV2SampleEntries. Otherwise, an OutOfBandAV2SampleEntry SHOULD be used.

An AV2SampleEntry SHALL contain exactly one AV2CodecConfigurationBox describing all extended layers, embedded layers, and temporal layers in the AV2SampleEntry. That AV2CodecConfigurationBox SHALL be a direct child box of the AV2SampleEntry.

Unless explicitly stated otherwise in the definition of a box, the direct child boxes of an AV2SampleEntry describe all extended layers, embedded layers, and temporal layers that can be output from the associated AV2 bitstream as specified in the Singlestream and Multistream Full bitstream decoding process from Annex F of [AV2].

An AV2SampleEntry SHALL contain at least one colr box with the nclx colour_type. Each layer in an AV2SampleEntry SHALL be described by at most one colr box with the nclx colour_type.

Each value among colour_primaries, transfer_characteristics, and matrix_coefficients carried in a colr box with the nclx colour_type, for the layers it applies to:

• SHOULD equal the corresponding effective value of the AV2 bitstream carried by the track if the latter is anything but 2 (Unspecified);
• SHALL take precedence over the corresponding effective value of the AV2 bitstream carried by the track.

The value of full_range_flag carried in a colr box with the nclx colour_type, for the layers it applies to, SHOULD equal the corresponding effective value of the AV2 bitstream carried by the track and SHALL take precedence over the corresponding effective value of the AV2 bitstream carried by the track.

An AV2SampleEntry SHALL contain at least one pixi box. Each layer in an AV2SampleEntry SHALL be described by at most one pixi box. pixi boxes in an AV2SampleEntry SHALL have the least significant bit of px_flags enabled. The constraints described in § 3.3.6 Layers apply to that AV2SampleEntry.

NOTE: An AV2SampleEntry may contain additional direct child boxes permitted by [ISOBMFF] for a VisualSampleEntry, such as colr with a colour_type value other than nclx, pasp, clli, mdcv or btrt. They may be located before and/or after the AV2CodecConfigurationBox.

NOTE: An AV2SampleEntry may contain indirect child boxes, for example colr when carried by the LayerPropertyContainerBox as defined in the draft of ISOBMFF 14496-12 10th Edition to describe the colour properties of the base track and the different colour properties of the gain map track.

3.2.2. AV2 Codec Configuration Box

The AV2CodecConfigurationBox carries the decoder initialisation data required to decode the AV2 samples of a track. There are two AV2CodecConfigurationBox flavours.

An AV2CodecConfigurationBox that is the direct child box of an AV2SampleEntry describes all layers in that AV2SampleEntry.

3.2.2.1. Description

An OutOfBandAV2CodecConfigurationBox MAY carry OBUs of the following types: OBU_MSDO, OBU_LAYER_CONFIGURATION_RECORD, OBU_OPERATING_POINT_SET, OBU_ATLAS_SEGMENT, OBU_METADATA_SHORT with obu_xlayer_id equal equal to GLOBAL_XLAYER_ID and metadata_is_suffix equal to 0, OBU_METADATA_GROUP with metadata_is_suffix=0, and OBU_SEQUENCE_HEADER.

An OutOfBandAV2CodecConfigurationBox SHALL NOT carry OBUs of the following types: OBU_TEMPORAL_DELIMITER, OBU_CONTENT_INTERPRETATION, OBU_MULTI_FRAME_HEADER, OBU_BUFFER_REMOVAL_TIMING, OBU_QUANTIZATION_MATRIX, OBU_FILM_GRAIN, OBU_CLOSED_LOOP_KEY, OBU_OPEN_LOOP_KEY, OBU_LEADING_TILE_GROUP, OBU_REGULAR_TILE_GROUP, OBU_SWITCH, OBU_LEADING_TIP, OBU_REGULAR_TIP, OBU_BRIDGE_FRAME, OBU_RAS_FRAME, OBU_METADATA_SHORT with obu_xlayer_id equal not equal to GLOBAL_XLAYER_ID or metadata_is_suffix equal to 1, OBU_METADATA_GROUP with obu_xlayer_id equal not equal to GLOBAL_XLAYER_ID or metadata_is_suffix equal to 1, and OBU_PADDING.

There SHOULD NOT be any Metadata OBU carried by an OutOfBandAV2CodecConfigurationBox if a codec-agnostic carriage solution exists for that metadata.

The OBUs carried by an OutOfBandAV2CodecConfigurationBox SHALL appear in the order specified by the ordering of OBUs in a temporal unit in [AV2], with the grouped high level syntax block being present.

3.2.2.2. Syntax

abstract class AV2CodecConfigurationBox(int codingname) extends Box(codingname)
{
}

aligned(8) class OutOfBandAV2CodecConfigurationBox extends AV2CodecConfigurationBox('av2C')
{
  unsigned int(8) reserved1 = 0;
  unsigned int(8) config_obus_count_minus1;
  for (i = 0; i < config_obus_count_minus1 + 1; ++i) {
    unsigned int(8) config_obu[];
  }
}

aligned(8) class InBandAV2CodecConfigurationBox extends AV2CodecConfigurationBox('av2c')
{
  unsigned int(3) reserved1 = 0;
  unsigned int(2) max_num_streams_minus_1;
  unsigned int(5) max_profile_idc;
  unsigned int(5) max_level_idx;
  unsigned int(1) max_tier;
}

Editor’s note: The exact fields to carry in InBandAV2CodecConfigurationBox should be studied further.

3.2.2.3. Semantics

reserved1 SHALL be set to 0. Readers SHALL ignore its value.

Editor’s note: Should reserved1 be renamed to reserved?

config_obus_count_minus1 is the number of config_obu minus one.

config_obu is a byte array. Each config_obu SHALL carry a single OBU preceded by a leb128() num_bytes_in_obu value giving the byte length of that OBU, as specified in the Length delimited bitstream format from Annex B of [AV2].

max_num_streams_minus_1 specifies the maximum number of independent streams in the bitstream.

• There SHALL NOT be any Multi Stream Decoder Operation OBU in the corresponding AV2 sample whose num_streams_minus_2 value is greater than or equal to max_num_streams_minus_1.
• If max_num_streams_minus_1 is not zero, there SHOULD be at least one Multi Stream Decoder Operation OBU in the corresponding AV2 sample.

NOTE: The above means there shall be no Multi Stream Decoder Operation OBU if max_num_streams_minus_1 is set to 0.

NOTE: It is a requirement of bitstream conformance that num_streams_minus_2 is not greater than 2 in [AV2], so max_num_streams_minus_1 fits in two bits. reserved1 can be repurposed once num_streams_minus_2 uses its three allocated bits in a future version of [AV2].

max_profile_idc specifies the maximum profile for the coded video sequence.

• There SHALL NOT be any Sequence Header OBU in the corresponding AV2 sample whose seq_profile_idc value is greater than max_profile_idc.
• There SHALL NOT be any Multi Stream Decoder Operation OBU in the corresponding AV2 sample whose multistream_profile_idc value is greater than max_profile_idc.
• There SHOULD be at least one Sequence Header OBU in the corresponding AV2 sample whose seq_profile_idc has the same value as max_profile_idc, or at least one Multi Stream Decoder Operation OBU in the corresponding AV2 sample whose multistream_profile_idc has the same value as max_profile_idc.

max_level_idx specifies the maximum level that the coded video sequence conforms to.

• There SHALL NOT be any Sequence Header OBU in the corresponding AV2 sample whose seq_level_idx value is greater than max_level_idx.
• There SHALL NOT be any Multi Stream Decoder Operation OBU in the corresponding AV2 sample whose multistream_level_idx value is greater than max_level_idx.
• There SHOULD be at least one Sequence Header OBU in the corresponding AV2 sample whose seq_level_idx has the same value as max_level_idx, or at least one Multi Stream Decoder Operation OBU in the corresponding AV2 sample whose multistream_level_idx has the same value as max_level_idx.

max_tier specifies the maximum tier that the coded video sequence conforms to.

• There SHALL NOT be any Sequence Header OBU in the corresponding AV2 sample whose seq_tier value is greater than max_tier.
• There SHALL NOT be any Multi Stream Decoder Operation OBU in the corresponding AV2 sample whose multistream_tier value is greater than max_tier.
• There SHOULD be at least one Sequence Header OBU in the corresponding AV2 sample whose seq_tier has the same value as max_tier, or at least one Multi Stream Decoder Operation OBU in the corresponding AV2 sample whose multistream_tier has the same value as max_tier.

Editor’s note: for a Multistream bitstream, the AV2CodecConfigurationBox contains a Sequence Header OBU for each extended layer. The impact on multi-layer use cases — including how a reader determines the active decoder configuration per extended layer from multiple Sequence Header OBUs — needs investigation. Also how profile / level is signalled to decoder API for layers. etc.

3.3. AV2 Sample Format

3.3.1. General

The AV2 sample boundary delimits the temporal unit.

Each OBU within a sample SHALL be framed using the Length delimited bitstream format specified in Annex B of [AV2]: each OBU is preceded by a leb128() num_bytes_in_obu field giving the byte length of the OBU that follows. Figure 3.1 illustrates this mapping.

3.3.2. OBUs in samples

The following applies to a sample associated with an AV2SampleEntry:

• The OBUs carried by a sample SHALL appear in the order specified by the ordering of OBUs in a temporal unit in [AV2].
• The sample SHALL NOT carry an OBU of a type among OBU_TEMPORAL_DELIMITER, OBU_CONTENT_INTERPRETATION and OBU_PADDING.

  NOTE: The AV2 sample boundary already delimits the temporal unit, and byte-level padding within a track is achieved using ISOBMFF mechanisms, e.g. free/skip boxes or sample size signalling.

• There SHOULD NOT be any Metadata OBU carried by a sample if a container-level carriage solution exists for that metadata.

The following applies to a sample associated with an AV2SampleEntry containing an OutOfBandAV2CodecConfigurationBox:

• The sample SHALL NOT carry an OBU of a type among the types allowed to be carried in an OutOfBandAV2CodecConfigurationBox as listed in § 3.2.2 AV2 Codec Configuration Box, unless there is a bit-exact copy of that OBU (type and content) in the associated OutOfBandAV2CodecConfigurationBox.
• The OBUs present in both the sample and the associated OutOfBandAV2CodecConfigurationBox, identified by the same type and content, SHALL be present in the same order.
• A decoder implementation MAY skip, in the sample, any OBU of a type among the types of the OBUs carried in the associated OutOfBandAV2CodecConfigurationBox.

  Editor’s note: Double-check that OBU skipping leads to the same output in all cases. Otherwise another assertion would be needed: "The sample SHALL contain all necessary OBUs so that any OBU skipped this way by a decoder implementation leads to the same output as if the OBU was not skipped".

• The sample SHOULD NOT carry an OBU of a type among the types allowed to be carried in an OutOfBandAV2CodecConfigurationBox.

3.3.3. Content Interpretation OBU

The following container-level signalling is available to replace the semantics of the following Content Interpretation OBU fields:

• ci_scan_type_idc: ilce sample group from ISO/IEC [23001-17]

• ci_color_description_idc, ci_color_primaries, ci_transfer_characteristics, ci_matrix_coefficients, and ci_full_range_flag: colr box from [ISOBMFF] with the nclx colour_type

• ci_chroma_sample_position_top: pixi box from [ISOBMFF]

• ci_aspect_ratio_idc, ci_sar_width, and ci_sar_height: iso3 brand, pasr sample group, or pasp box from [ISOBMFF]

• timing_info(): mvhd and stts boxes from [ISOBMFF]

3.3.4. Redundant information

The following table lists fields that can be found at the container level and in the bitstream.

3.3.5. Sync sample

An AV2 sample is a sync AV2 sample if, for every coded extended layer unit present in the AV2 sample, the first coded frame OBU of that coded extended layer unit has obu_type equal to OBU_CLOSED_LOOP_KEY. Such an AV2 sample is a Closed Random Access point for the carried bitstream and corresponds to a SAP of type 1 or 2 as defined in Annex I of [ISOBMFF].

A sync AV2 sample SHALL be self-sufficient for starting decoding at that point for all layers: every OBU referenced by the coded frame OBUs of the sync AV2 sample SHALL be available either by being carried in the AV2CodecConfigurationBox of the active AV2SampleEntry or by being present in the sample, as required by the availability of HLS OBUs in [AV2].

NOTE: In ISOBMFF terms, sync samples are signalled using the SyncSampleBox (stss) in unfragmented tracks, or by setting sample_is_non_sync_sample equal to 0 in fragmented tracks. When every sample in a track is a sync sample, the SyncSampleBox is omitted in unfragmented tracks (per [ISOBMFF]), and in fragmented tracks this is expressed through the default sample flags of the MovieExtendsBox (trex) or TrackFragmentHeaderBox (tfhd) with no per-sample override.

3.3.6. Layers

In an AV2 sample, the following applies to all extended layers and embedded layers associated with a pixi box:

• The channel at index i in the pixi box corresponds to the (i+1)th channel among the channels of the extended and embedded layers associated with the pixi box. This is the case for each temporal layer if any.

• The channels of the layers associated with that pixi box are considered to be ordered as if the channels of the extended and embedded layers were concatenated in increasing xlayer_id order, then in increasing mlayer_id order.

• If the pixi box describes a channel, the corresponding channel in the AV2 sample SHALL be present.
• If the pixi box describes a channel to be a color channel (channel_idc values such as 2, 3, 4, and 7), the corresponding channel in the AV2 sample SHALL be a color channel: if signalled, the lcr_layer_type field of its associated Layer Configuration Record OBU SHALL be set to 0 (TEXTURE_LAYER).
• If the pixi box describes a channel to be a color channel (channel_idc values such as 2, 3, 4, and 7), the layer carrying the corresponding channel in the AV2 sample SHALL be described by a colr box.
• If the pixi box describes a channel to be an alpha channel (channel_idc 5), the corresponding channel in the AV2 sample SHALL be an alpha channel: if signalled, the lcr_layer_type field of its associated Layer Configuration Record OBU SHALL be set to 1 (AUX_LAYER) and lcr_auxiliary_type to 0 (ALPHA_AUX).
• If the pixi box describes a channel to be a depth channel (channel_idc 6), the corresponding channel in the AV2 sample SHALL be a depth channel: if signalled, the lcr_layer_type field of its associated Layer Configuration Record OBU SHALL be set to 1 (AUX_LAYER) and lcr_auxiliary_type to 1 (DEPTH_AUX).
• If the pixi box describes a channel as used and specified (channel_idc values other than 0 and 1) with subsampling_flag set to 1, the corresponding channel in the AV2 sample SHALL have the value of the chroma_format_idc field of its associated Sequence Header OBU correspond to the same meaning as the value of the pixi box’s corresponding channel’s subsampling_type value.
• If the pixi box describes a channel, the corresponding channel in the AV2 sample SHALL have the value of the BitDepth variable computed from its associated Sequence Header OBU equal to the value of the pixi box’s channel’s bits_per_channel value.

  NOTE: Channels of different types of embedded layers, such as texture and alpha, may have different bit depths.

• The component_format of every channel in the pixi box SHALL indicate an unsigned integer format.

Editor’s note: pixi semantics follow the FDIS of [ISOBMFF] 9th edition (Tables 8 and 9) and need re-checking against the published text. Open issues: the component_format value for unsigned integer (ISO/IEC 23001-17); channel_idc for channels with no defined meaning (segmentation, gain map).

3.3.7. AV2 sample reconstruction

A byte sequence containing a conformant [AV2] Temporal Unit using the Length delimited bitstream format specified in Annex B of [AV2] MAY be reconstructed from the OBUs of an AV2 sample by applying the following steps:

1. Write a Temporal Delimiter OBU, preceded by its leb128() length prefix.

2. Write the OBUs carried in the OutOfBandAV2CodecConfigurationBox, if any, of the corresponding AV2SampleEntry, in the same order as present in the OutOfBandAV2CodecConfigurationBox, each preceded by its leb128() length prefix as present in the OutOfBandAV2CodecConfigurationBox.

3. Write the OBUs carried in the sample, in the same order as present in the sample, each preceded by its leb128() length prefix as present in the sample.

Note: AV2-ISOBMFF reader implementations do not have to reconstruct the contiguous byte sequence containing a conformant AV2 bitstream or Temporal Unit in memory upon decoding.

↑ Back to Table of Contents

4. CMAF AV2 track format

[CMAF] defines structural constraints on ISOBMFF files additional to [ISOBMFF] for the purpose of, for example, adaptive streaming or for protected files. Conformance to these structural constraints is signaled by the presence of the brands cmfc or cmf2 in the FileTypeBox.

NOTE: It is important to note that while the cmf2 brand does not introduce any new requirements compared to the previously defined cmfc brand, it further restricts the cmfc constraints.

For compatibility reasons, when the cmf2 brand is used, the cmfc brand SHOULD also be included in the FileTypeBox.

If a CMAF Video Track uses the brand av02, it is called a CMAF AV2 Track and the following constraints, defining the CMAF Media Profile for AV2, apply:

• it SHALL use an AV2SampleEntry, possibly transformed by encryption as specified in § 5 Common Encryption;
• it MAY use multiple sample entries, and in that case the following values SHALL not change in the track:

  • seq_profile_idc

  • single_picture_header_flag

  • seq_level_idx

  • seq_tier

  • chroma_format_idc

  • bit_depth_idc

  • seq_initial_display_delay_minus_1

When protected, CMAF AV2 Tracks SHALL use the signaling defined in [CMAF], which in turn relies on [CENC], with the provisions specified in § 5 Common Encryption.

↑ Back to Table of Contents

5. Common Encryption

CMAF AV2 Tracks and non-segmented AV2 files MAY be protected. If protected, they SHALL conform to [CENC]. Files SHOULD be protected using the cbcs protection scheme and MAY be protected using the cenc protection scheme.

When the protected scheme cenc is used, samples SHALL be protected using subsample encryption and SHALL NOT use pattern encryption.

• crypt_byte_block = 1 and skip_byte_block = 9,
• crypt_byte_block = 5 and skip_byte_block = 5,
• crypt_byte_block = 10 and skip_byte_block = 0.

Version 1 and version 2 of the saiz box as defined in [ISOBMFF] SHALL be supported.

5.1. General Subsample Encryption constraints

Within protected samples, the following constraints apply:

• Protected samples SHALL be exactly spanned by one or more contiguous subsamples.
  • An OBU MAY be spanned by one or more subsamples, especially when it has multiple ranges of protected data. However, as recommended in [CENC], subsample encryption entries SHOULD be as compactly represented as possible. For example, only one subsample encryption entry should be created for multiple consecutive unprotected OBUs as well as the first unprotected and protected parts of the following protected OBU, if such a protected OBU exists.
  • A large unprotected OBU whose data size is larger than the maximum size of a single BytesOfClearData field MAY be spanned by multiple subsamples with zero size BytesOfProtectedData.
• All num_bytes_in_obu fields and OBU Headers SHALL be unprotected.
• Tile Group OBUs (the collection of OBU types identified by the is_tile_group() helper function) SHALL be protected as specified below. All other OBU types SHALL be unprotected.

• Within Tile Group OBUs, the following applies:

  • for the protected scheme cbcs:

    • BytesOfProtectedData SHALL start on the first byte and end on the last byte of the decode_tile structure (including any trailing bits).

    NOTE: As a result of the above, BytesOfProtectedData is not necessarily a multiple of 16 bytes and partial blocks are included in the BytesOfProtectedData range, but per [CENC] the data is not encrypted.

    • A subsample SHALL be created for each tile, even if its size is less than 16 bytes.
    • All other parts of Tile Group OBUs SHALL be unprotected .

  • for the protected scheme cenc:

    • BytesOfProtectedData SHALL be a multiple of 16 bytes.
    • BytesOfProtectedData SHALL end on the last byte of the decode_tile structure (including any trailing bits).
    • BytesOfProtectedData SHALL span all complete 16-byte blocks of the decode_tile structure (including any trailing bits).

    NOTE: As a result of the above, partial blocks are not used and it is possible that BytesOfProtectedData does not start at the first byte of the decode_tile structure, but some number of bytes following that.

    • A subsample SHALL be created for each tile that has a decode_tile structure whose size (including any trailing bits) is greater than or equal to 16 bytes. If it is less than 16 bytes, per the rules above, the decode_tile structure is not encrypted and the corresponding bytes SHOULD be included in the BytesOfClearData field of a surrounding subsample, if any.
    • All other parts of Tile Group OBUs SHALL be unprotected .

↑ Back to Table of Contents

6. Codecs Parameter String

DASH and other applications require defined values for the Codecs parameter specified in [RFC6381] for ISO Media tracks. The codecs parameter string for the AOM AV2 codec is as follows:

<sample entry 4CC>.<profile>.<level><tier>.<bitDepth>.<channelCount>.<chromaSubsampling>.
<colorPrimaries>.<transferCharacteristics>.<matrixCoefficients>.<videoFullRangeFlag>

All fields following the sample entry 4CC are expressed as double digit decimals, unless indicated otherwise. Leading or trailing zeros cannot be omitted.

• The profile parameter value SHALL equal the value of seq_profile_idc in the first Sequence Header OBU.
• The level parameter value SHALL equal the level value indicated by seq_level_idx in the first Sequence Header OBU.
• The tier parameter value SHALL be equal to M when the seq_tier value in the first Sequence Header OBU is equal to 0, and H when it is equal to 1.
• The bitDepth parameter value SHALL equal the value of the BitDepth variable as defined in [AV2] derived from the first Sequence Header OBU.
• The channelCount parameter value is represented by 1, 2, 3, or 4. If the pixi box associated with the first Sequence Header OBU has a channel_idc set to 5 (Alpha), then the channelCount parameter value SHALL equal one plus the value of the NumPlanes variable as defined in [AV2] derived from that Sequence Header OBU, otherwise it SHALL match NumPlanes..

• The chromaSubsampling parameter value, represented by a two-digit decimal:

  1. SHALL have its first digit equal to the subsampling_type value of the channels with channel_idc set to 3 (Second colour channel) and 4 (Third colour channel) in the pixi box associated with the AV2 sample if that subsampling_type value is signaled, and 0 otherwise.
  2. SHALL have its second digit equal to the subsampling_location value of the channels with channel_idc set to 3 (Second colour channel) and 4 (Third colour channel) in the pixi box associated with the AV2 sample if that subsampling_location value is signaled, and 0 otherwise.
• The colorPrimaries, transferCharacteristics, matrixCoefficients, and videoFullRangeFlag parameter values SHALL equal the values of matching fields in the colr box with the nclx colour_type.

  NOTE: A colr box with the nclx colour_type is mandated to be present in § 3.2.1 AV2 Sample Entry.

• The videoFullRangeFlag is represented by 0 or 1.

If the codecs parameter string ends with ".3.22.01.01.01.0" (containing all the default values below), that trailing part of the string SHOULD be omitted.

The parameters sample entry 4CC, profile, level, tier, and bitDepth are all mandatory fields. If any of these fields is empty, or not within its allowed range, the processing device SHOULD treat it as an error. All the other fields (including their leading '.') are optional, mutually inclusive (all or none) fields. If not specified then the values listed in the table below are assumed.

If any character that is not '.', digits, part of the AV2 4CC, or a tier value is encountered, the string SHALL be interpreted ignoring all the characters starting from that character.