Discontinuities propagation and effects changes

Problem

Past 2.1, playback of unreliable media has degraded.

As for 3.0 and 4.0, unreliable transport will mostly trigger frame drop and GOP resync, creating sometimes unwatchable conditions. This get worse when you cumulate with the Windows UDP issue.

Depending on the context, packet loss is notified using BLOCK_FLAG_DISCONTINUITY and/or BLOCK_FLAG_CORRUPTED in the following cases:

• Packet loss within aggregation
• Packet loss out of aggregation
• Full sample or sequence loss

More globally the BLOCK_FLAG_DISCONTINUITY is used to notify any data stream discontinuity or temporal discontinuity. This lead to some additional specific cases:

• Virtual machine passive seek (sequence change)
• Timeline jump within a demuxer

BLOCK_FLAG_DISCONTINUITY tags the block beginning the new sequence, or past drop. BLOCK_FLAG_CORRUPTED should tag the block truncated or containing the corruption.

When notifying a DISCONTINUITY on block N, logically the CORRUPTED block should be the N-1th.

Within an aggregation unit, we should able to tell that a discontinuity is truncating content and tag properly. With delegated aggregation (ie packetizer) we usually already have sent previous blocks, and notifying corruption is impossible. Full sample or full sequence loss make also impossible to notify corruptions.

The pipeline should also be able to differentiate from temporal jumps, passive seeks and packet loss. All those should have different consequences (ignore, drain, flush...) , but today these are mostly unified.

History and consequences

Since introduction of CORRUPTED flag, the pipeline has been changing behaviour upon CORRUPTED or DISCONTINUITY flags.

70ee5fbf

     * block: added
        - BLOCK_FLAG_CORRUPTED : signal corrupted data (do not use anymore
     BLOCK_FLAG_DISCONTINUITY in that case)

At packetizer/decoder level, due to the N or N+1 tagging of packet loss, BLOCK_FLAG_DISCONTINUITY blocks were mostly erroneously dropped until removed (5e36cb2c)

BLOCK_FLAG_DISCONTINUITY|BLOCK_FLAG_CORRUPTED on a dummy block was used (9e6e2eeb) at input level to trigger decoders flush, until it was replaced with proper flush callback (2b38ec86 and 8c4fff2f)

At packetizer/decoder level, BLOCK_FLAG_DISCONTINUITY|BLOCK_FLAG_CORRUPTED was unified, mostly ending in a Flush (ce6db786 and 477abfab)

Later, packetizer/decoder drain fixes have been applied to the same case, ending triggering a drain on every propagated packet loss.

Current state

The usual behaviour and tagging is as below:

	packet drop	truncation	random access (vm seek/chapter)	flush (input seek)	FLAG_CORRUPTED	FLAG_DISCONTINUITY
access_demux	FLAG_DISCONTINUITY	FLAG_DISCONTINUITY FLAG_CORRUPTED	FLAG_DISCONTINUITY	FLAG_DISCONTINUITY
demux	FLAG_DISCONTINUITY FLAG_CORRUPTED	FLAG_DISCONTINUITY FLAG_CORRUPTED		FLAG_DISCONTINUITY		FLAG_DISCONTINUITY FLAG_CORRUPTED
packetizer		FLAG_CORRUPTED		flush() reset()	drain() ¹ reset() ¹	drain() reset() FLAG_DISCONTINUITY
decoder				flush() reset()	drop() drain() ¹ reset() ¹	drain()

¹ - Candidate to be removed by merge request !2876

Handling of discontinuities and corruptions

At packetizer level, temporal jumps should have effect on interpolated dates, and pure data loss should not require much specific cases when relying on start codes. Processing corruption is a plus, but as for now it was never done.

At decoder level the corruption becomes more trickier because depending on the codec it can have mixed consequences (mostly raw audio static noise vs video glitches)

Some corruptions could be ignored if the bitstream or the codec is resilient.