Skip to content

Draft: decoder: add support for Sub ES tracks

Thomas Guillem requested to merge tguillem/vlc:sub-es into master

Hello,

Here is a DRAFT MR of Sub ES tracks handling. I would like your opinion before continuing my work on this subject.

Context: some upcoming codecs/format, like AC4 and ADM, support the following:

  • Multiple presentation (language) in one stream
  • Multiple profile/channels choice (stereo vs 5.1 vs spatial) in one stream

A Sub ES track represent a presentation and/or a profile. A decoder module can send a list of Sub ES tracks from the module probe function or midstream (from pf_decode).

A Sub ES is represented by the struct decoder_sub_es_t from the vlc_codec.h API. An es_out_id_t will be created by es_out.c for each Sub ES. For the end user (vlc_player_t user, like interface module), a Sub ES will be represented by a classical vlc_player_track.

A Sub ES track preference choice can also be saved (from the medialib) since its will use the following id scheme:

str_id: audio/0/sub-es/audio/0
str_id: audio/0/sub-es/audio/1
str_id: audio/0/sub-es/audio/2
str_id: audio/0/sub-es/audio/3
str_id: audio/0/sub-es/audio/4

str_id: audio/0/sub-es/audio/3 is the 4th Sub ES track of the first audio track.

Known limitations:

  • A Sub ES track is only known when the parent track is selected. This can be problematic with media having AC3 (as compat codec) and AC4 tracks. We could modify the autoselect algorithm in es_out.c to prioritize modules handling Sub ES tracks to work around this issue.
  • Unselecting a Sub ES track will hide all Sub ES tracks. We could work around this issue by keeping Sub ES tracks in es_out.c, even if the parent ES is unselected.

TODO:

  • split WIP: decoder: add Sub ES handling
  • Improve documentation
  • Add a parent track pointer in the vlc_player_track to allow UIs to represent Sub tracks in a tree
  • Test it via the new mock sub_es_count option.
  • Fix asserts/crash with the clock

Thanks for reading me, any comment is welcome.

Edited by Thomas Guillem

Merge request reports