MPEG_audio_spatial

Contributors

Status

Draft

Dependencies

Written against the glTF 2.0 spec.

Overview

The MPEG audio extension adds support for spatial audio. This extension is identified by MPEG_audio_spatial, which may be included at top level or attached to any node in the scene. The MPEG_audio_spatial extension supports three different node types: * source: an audio source that provides input audio data into the scene. Mono object and HOA sources are supported in this version of the document. * Type: ‘Object’ or ‘HOA’ * HOA audio sources shall ignore the parent node’s position and be rendered only in 3DoF. * reverb: A reverb effect can be attached to the output of an audio source. Several reverb units can exist and sound sources can feed into one or more of these reverb units. An audio renderer that does not support reverb shall ignore it if the bypass attribute is set to true. If the bypass attribute is set to false, the audio renderer shall return an error message * listener: An audio listener represents the output of audio in the scene. A listener should be attached to a camera node in the scene. By being a child node of the camera, additional transformations can be applied to the audio listener relative to the transformation applied to the parent camera.

glTF Schema Updates

MPEG_audio_spatial

Spatial audio is defined by adding the MPEG_audio_spatial extension to the camera node or as a separate node.

The following table lists the semantics of the properties of an MPEG_audio_spatial extension:

Type Description Required
sources array An array of source items No
listener array An audio listener item No
reverbs array An array of reverb items No

MPEG_audio_spatial.sources

MPEG_audio_spatial.listener

MPEG_audio_spatial.reverb

source

Type Description Required
id integer An identifier of the audio source Yes
type string A type of the audio source Yes
pregain number A level-adjustment of the audio source No
playbackSpeed number Playback speed of the audio source No
attenuation string A function used to calculate the attenuation of the audio source No
attenuationParameters object[1-*] An array of parameters that are input to the attenuation function No
referenceDistance number A distance in meters for which the distance gain is implicitly included in the audio source after application of pregain No
accessors integer[1-*] An array of accessors that describe the audio source Yes
reverbFeed integer[1-*] An array of pointers to reverb units No
reverbFeedGain integer[1-*] An array of gain values [dB] No
source.id
source.type

Specifies an identifier of the audio source. Type value equal to "Object" indicates mono object. Type value equal to "HOA" indicates HOA object.

source.pregain

Specifies a level-adjustment in dB for the signal associated with the audio source.

source.playbackSpeed

Specifies the playback speed of the audio source. A value of 1.0 corresponds to playback at normal speed.

source.attenuation

Specifies the function used to calculate the attenuation of the audio signal based on the distance to the source.

The attenuation functions, and their parameters are defined in ISO/IEC 23090-14:2021:Annex C.

source.attenuationParameters

Specifies an array of parameters that are input to the attenuation function. The semantics of these parameters depend on the attenuation function itself.

EDITORIAL NOTE: It should be elaborated and provide some references

source.referenceDistance

Specifies the distance in meters for which the distance gain is implicitly included in the source signal after application of pregain. The value should be disregarded for "HOA" audio sources.

source.accessors

An array of indices of the accessors. Provides a references to the accessors, by specifying the glTF 2.0 accessor’s indices in accessors array, that describes the buffers where the data will be made available.

source.reverbFeed

Specifies one or more pointers to reverb units. It may be optionally extended by a floating point scaling factor.

source.reverbFeedGain

provides an array of gain [dB] values to be applied to the source’s signal(s) when feeding it to the corresponding reverbFeed.

listener

Type Description Required
id integer A unique identifier of the audio listener Yes
listener.id

reverb

Type Description Required
id integer A unique identifier of the audio reverb unit in the scene Yes
bypass boolean Specifies whether the reverb unit can be bypassed or not No
properties array An array of property items Yes
predelay number Delay [seconds] from onset of audio source to onset of late reverberation for which DSR is provided. No
reverb.id
reverb.bypass

Specifies whether the reverb unit can be bypassed or not if an audio renderer does not support it.

reverb.properties

Array of items that specifies reverbProperties items describing reverb unit specific parameters.

reverb.predelay

Specifies delay in seconds from onset of source to onset of late reverberation for which DSR is provided.

reverb.property

Type Description Required
frequency number Frequency for RT60 and DSR values Yes
RT60 number RT60 values Yes
DSR number Diffuse-to-Source Ratio value Yes
property.frequency

Specifies frequency for provided RT60 and DSR values.

property.RT60

Specifies RT60 value in second for the frequency provided in the property.frequency field.

property.DSR

Specifies Diffuse-to-Source Ratio value in dB for the frequency provided in the property.frequency field. For details see ISO/IEC 23090-14:2021:5.4.1.4.

Interaction with other extensions

The MPEG_audio_spatial works with the MPEG_accessor_timed to reference audio sources.

JSON Schema

MPEG_audio_spatial.schema.json

MPEG_audio_spatial.listener.schema.json

MPEG_audio_spatial.reverb.schema.json

MPEG_audio_spatial.reverb.property.schema

MPEG_audio_spatial.source.schema.json

Known Implementations

Resources