Skip to content

Fix seek when using hw-dec: use the same pic/surface count between decoder core and hw modules

Thomas Guillem requested to merge tguillem/vlc:dec-pic-count into master

This is a second proposal of the patch set that was sent by @robUx4, cf. discussion here:

https://mailman.videolan.org/pipermail/vlc-devel/2020-July/135708.html

@Courmisch said

There are two ways to do it:

  1. (This patch) The decoder tells the owner, and the owner takes care of it. This only works for normal (i.e. software) picture buffers.
  2. The decoder allocates its own pool of the correct size. This works for both normal and opaque picture buffers.

And based on that, we need to support the second mechanism anyway, so the first mechanism seems rather vain. The second mechanism is also more in line with the earlier design directions/decisions.

Implementing the solution 2) require a lot of work and rework, I don't think it is reasonable to do it for VLC 4.0. Besides, the solution 2) won't solve the issue #25694 (closed). We would still have to add a way to notify the decoder module to cancel its internal pool.

That is the reason I propose again this MR.

Edit: More explanations: cf. gdb back traces of all involved threads

Note: it seems I found the true problem while writing this more detailed explanation, cf the bold paragraph

VLC Decoder Thread (src/input/decoder.c: DecoderThread) (Thread 31)
  | pf_decode()
    \ avcodec_send_packet()
     \ ff_thread_decode_frame()
      \ submit_packet()
      | if (prev_thread) wait for prev_thread->state == STATE_INPUT_READY
      | set cur_thread->state = STATE_SETTING_UP
  | if (dec_flushing) { pf_flush(); vout_Flush() }

FFmpeg pthread frame (frame_worker_thread()) (Thread 29)
  | wait for thread->state == STATE_SETTING_UP
  | h264_decode_frame()
   \ lavc_GetFrame()
    \ vlc_va_Get()
    | wait for va_pool (vout)
  | set cur_thread->state = STATE_INPUT_READY

VLC input thread (src/input/input.c: Run) (Thread 25)
 | vlc_input_decoder_Flush()
  | dec_flushing = true; signal();
  | vout_Cancel()

The avcodec pf_decode() callback ends in submit_packet() that wait for the previous avcodec worker thread to decode its input buffer. The frame_worker_thread() can take a long time decoding its input buffer. Indeed, it first waits for an output buffer, that depends on the VLC vout.

During normal playback, a new output buffer should be available every 40ms (25fps) or 16ms (60fps) (or every 1sec in the worst-case scenario: 1fps). Blocking frame_worker_thread() and pf_decode() for such delay is negligible.

While seeking, the clock is reset but the vout continue to display its fifo of output buffers with a completely new sync point, hence the delay of few seconds.

Therefore, it can take a long time for the DecoderThread to process the seek command since it's waiting for pf_decode, that is waiting for the vout to display its next picture using the reset sync point.

Edited by Thomas Guillem

Merge request reports