Commits on Source (16)
-
Missing part from 5140b25e.
f7be6711 -
A unit-test for the transcoder code currently is hard to achieve, as it involves decoders, encoders, filters, sout_stream_t object, and general transcode code. This commit introduces an integration test which mocks the extern components (filters, decoders, encoders) used by the transcode pipeline and starts this pipeline using the usual input item properties. The main goals are to: - check whether no part of the stream is dropped. - check whether push model with video context is supported everywhere. - check the different kind of decoder behaviour, in particular those for which decode is asynchronous with the decoder_QueueVideo call, those for which Open() is asynchronous with the decoder_UpdateVideo* call and their synchronous variants. - check memory leaks and use-after-free in the pipeline (through asan). - check threading within the pipeline (through tsan). - check decoder, encoder and filter loading failure. - check format adaptation through filters before the encoder. - check format conversion (called final_conv_static) before the encoder, needed for encoders requesting a conversion.
11ee3d5f -
39fd9245
-
The object is needed to forward the elementary streams and data to the sout stream components following the transcode. Note that having the stream object here means that we must ensure we don't call it after the transcode is supposed to be closed, so we must ensure every asynchronous components are drained and destroyed before leaving the close functions. The constraint was already there though, given p_obj actually pointed to the p_stream object too.
f84b3212 -
They will be reused differently when refactoring the initialization later at the format update site.
b1278274 -
If transcode failed, we want to forward the error to the stream output pipeline, so we need to store it for later and to abort the transcoding process.
7ed34d69 -
In future commits, we'll create the filter chain and encoder as soon as the decoder outputs the format it will generate through decoder_UpdateVideoOutput. It means that if the decoder doesn't report any format, we won't have an encoder anymore.
387a691a -
Some decoders are pushing format update synchronously during their activation but others have an asynchronous behaviour and may push it before their activation function has finished, or even after the first decode call when the decoder is asynchronously configured and ready. Thus, previously, the decoder format from the decoder was not necessarily known when opening the filters and the encoder. This patchset shifts the initialization of previously mentioned objects to the decoder update callback. In addition, encoders will now need to get the final video context to configure and it cannot open correctly without it, so the encoder test typically makes no sense anymore for video. In this patch, since we create the encoder only when the final format is finally supplied by the decoder, we can directly create the encoder that will be used for transcoding. In addition, since format update would still happen at activation for the synchronous decoders, the format and encoders will already be available, effectively replacing the encoder test afterwards.
8177a1d6 -
This test scenario for the transcode checks that the error state is correctly reported back to the stream output pipeline even if it happened asynchronously.
79902c34 -
When the encoder is opened, it cannot change format anymore, and we cannot change it behind its back.
b04216b2 -
9722e11c
-
Some decoders have the decoder_QueueVideo called asynchronously wrt. the decode() call, which is allowed and expected by the API. The MediaCodec decoder also has a limited number of input buffers and output buffers when mediacodec-dr is enabled. This creates a strong backpressure linking the output buffers to the input ones, meaning that no input buffer can be decoded unless the already submitted output buffers are processed and released. By running the decode() call followed by the processing of output buffers at the same location, we're linking the output buffers processing to the execution of the decode() call: no processing of outputs can happen until decode() hasn't returned. Except that decode() can be called to saturate the input buffers while the output was already saturated, leading it to a non deterministic deadlock. By processing the buffer synchronously with the decoder_QueueVideo() call, we ensure that the processing of the picture happens regardless of the decode() blocking state. It will typically call the filter and encode callbacks from the thread of the asynchronous module or from the decode callback itself for synchronous decoders, leading to either the same behaviour or difference wrt. depth-first vs breadth-first processing at the decoder level (do we decode all the pictures we can or do we process every new picture before decoding the others). The refactor is made so that the actual processing function is isolated and can be reused in a latter refactor involving a new thread if needed to bring those difference in execution. The previous transcode function is still handling the decoder and encoder drain(), waiting for the decoder to queue all the video frames and then waiting for the encoder to output every packets. It fixes a systematic non-deterministic deadlock happening when having the MediaCodec decoder in direct rendering mode within the transcode pipeline, when implementing #25153.
d8c883e2 -
Pictures are now pushed towards the encoder directly and not queued for encoding.
41600043 -
The encoder test is not used anymore, we'll try to open the encoder at the moment we have something to encode and using the output of filters.
77cfe38a -
A unit-test for the transcoder code currently is hard to achieve, as it involves decoders, encoders, filters, sout_stream_t object, and general transcode code. This commit introduces an integration test which mocks the extern components (filters, decoders, encoders) used by the transcode pipeline and starts this pipeline using the usual input item properties. The main end goals are to: - check whether no part of the stream is dropped. - check whether push model with video context is supported everywhere. - check the different kind of decoder behaviour, in particular those for which decode is asynchronous with the decoder_QueueVideo call, those for which Open() is asynchronous with the decoder_UpdateVideo* call and their synchronous variants. - check memory leaks and use-after-free in the pipeline (through asan). - check threading within the pipeline (through tsan). - check decoder, encoder and filter loading failure. - check format adaptation through filters before the encoder. - check format conversion (called final_conv_static) before the encoder, needed for encoders requesting a conversion. It's currently written in an asynchronous state way spread across multiple feature functions to handle the different stage of the pipeline without interaction, like it was done for the video output test, but we might want to find a way to rewrite those tests in a more sequential way if it were to get more complex even.
a685b7ea -
The endpoint checks that the encoder is correctly closed when reaching the end of the test. Such failure typically happened during the refactor of encoders function pointers into a separate virtual table.
5e202180
Showing
- modules/stream_out/transcode/encoder/encoder.c 11 additions, 11 deletionsmodules/stream_out/transcode/encoder/encoder.c
- modules/stream_out/transcode/encoder/video.c 4 additions, 52 deletionsmodules/stream_out/transcode/encoder/video.c
- modules/stream_out/transcode/transcode.c 8 additions, 2 deletionsmodules/stream_out/transcode/transcode.c
- modules/stream_out/transcode/transcode.h 3 additions, 1 deletionmodules/stream_out/transcode/transcode.h
- modules/stream_out/transcode/video.c 214 additions, 286 deletionsmodules/stream_out/transcode/video.c
- test/Makefile.am 7 additions, 0 deletionstest/Makefile.am
- test/modules/stream_out/transcode.c 381 additions, 0 deletionstest/modules/stream_out/transcode.c
- test/modules/stream_out/transcode.h 50 additions, 0 deletionstest/modules/stream_out/transcode.h
- test/modules/stream_out/transcode_scenarios.c 381 additions, 0 deletionstest/modules/stream_out/transcode_scenarios.c
test/modules/stream_out/transcode.c
0 → 100644
test/modules/stream_out/transcode.h
0 → 100644