[SOUT Workshop 2023] Technical Discussions summary

Milsetone

We agreed that we should focus on testing and hls/chromecast for 4.0

The other SOUT improvements mentioned in this document could wait for 5.0.

To test before 4.0

Remuxing

The output formats we care about mostly are the following:

• mp4
• fragmented mp4 (mp4frag or mp4stream)
• ts
• avi
• MKV
• RTP

Discontinuity

• ts
• core
• positive discontinuity ? Make a sample ?
• Negative discontinuity

Sout plugins

• Mosaic
• Start-time: test it with transcode

Seek/Preroll

• the transcode module should handle preroll flag:
  • decoder modules (inside transcode) should handle it (and drop it)
  • otherwise, the transcode module should drop them before passing to encoders (using the PTS)
  • ideally it should be handled in the common core layer shared with transcode, but it would require difficult refactoring.

Core → Flag Random Access vs Flag Discontinuity vs Corrupted

4 cases

• Flag Random Access (Seek of DVD VM) → always drain
• User Random Access (normal seek) → The core call pf_flush
• Discontinuity → Always a block flag, set by the demuxer/packetizer
• Corrupted → How to handle them ? Cc. @fcartegnie

TODO

• Add a stream_out continuity module (submodule of gather?) : for non-TS case, do a join of PTS/DTS/PCR in case of discontinuity
  • Use a mux/sout module capability to enable it or not automatically (disabled for TS)
• Remove ENABLE_SOUT from the core.
• Cycle: stitch PCR (seamless)

Metadata: DATA_ES vs new es_out callbacks ?

Issues

• #11875
• #15964
• #7556
• #6364 (closed)
• #5530
• #982
• #27392
• #26217
• #21576
• #18608

Solution

• Keep legacy DEMUX_GET_META
  • es_out.c: EsOutMeta() receive all legacy Metas (taglib, DEMUX_GET_META)
  • TODO: Send Meta to decoder.c/sout/sout_stream
• Add a way to send timed Metas from demuxers modules to es_out.c, they can be tied to a program id.
  • Use the same call to forward such Meta, but the callback needs to handle:
    • group_id
    • PTS
    • enum meta_type (for now, only VLC, but can add ID3 in a future)
    • union/void * for the data (for now, a vlc_meta_t pointer)
    • enum source (medialib, cue, playlist, demux, etc cf. TBD)

Draft: HLS stream output

!3764 (merged)

• !3764 (comment 389681): Check for max size before OOM, value configurable and a good default, in function of video res/bitate.
• For chromecast: always use memory, not FS
• !3764 (comment 389641): add const char str_id * in pf_add
• str_id for slaves issue: md5sum not good: use spawn order ID
  • but the spawn order is not reproducible (BUG in 3.0 but 4.0 ?): test it !
  • The order will change if the user removes some subtitles: we are OK with this drawbacks
  • To mitigate ^^, we can add a prefix before the ID: the iso code language or SUB if lang is unknown.

Mux API (maybe 5.0)

• Bad API: add()/mux(void): only really work with ts, other module struggle to handle large data fifo.
• Replace with send()
• Handle private data fifo in each module (helper ?)

Slaves PCR selection

Now that we have a way to differentiate demux source, we can select a PCR value between them (easiest is the minimum). This would remove some boilerplate in the subtitles demux (avoiding sending PCR in slave mode) and potentially fix video/audio capture cases where the slave PCR is actually below the main PCR.