nvdec: fix multiple call to cuInit

/!\ Disclamer: because of the usage of vlc_once_t, I'm not sure whether the device pointer used for initialization needs to be atomic or not. Feel free to indicate whether this incorrect, I haven't checked yet.

Each cuInit is leaking memory, and cuInit is supposed to be called only once. Ensure it through a vlc_once_t wrapper, but since the function table needs to be loaded in order to call into cuda functions, forward some available device for the initialization.

Below, there is two leaks because of the double decoder device allocation in debug mode. The first leak is leaked once, and is still there. The second and third leaks are now reduced to a single one.

Direct leak of 65536 byte(s) in 1 object(s) allocated from:
    #0 0x7febb6bf3652 in __interceptor_realloc /build/gcc/src/gcc/libsanitizer/asan/asan_malloc_linux.cpp:164
    #1 0x7feb8b56828f  (<unknown module>)

Direct leak of 56 byte(s) in 1 object(s) allocated from:
    #0 0x7febb6bf3459 in __interceptor_calloc /build/gcc/src/gcc/libsanitizer/asan/asan_malloc_linux.cpp:154
    #1 0x7feb89dc586a  (/home/alexandre/workspace/videolabs/vlc-meson/build-native/modules/.libs/libskins2_plugin.so+0x2f1086a)
    #2 0x7febb5f195c1 in decoder_device_Open ../../src/input/decoder_helpers.c:174
    #3 0x7febb5e67106 in vlc_module_load ../../src/modules/modules.c:243
    #4 0x7febb5f196e8 in vlc_decoder_device_Create ../../src/input/decoder_helpers.c:188
    #5 0x7febb60a7435 in vout_GetDevice ../../src/video_output/video_output.c:2244
    #6 0x7febb5ef7f26 in ModuleThread_GetDecoderDevice ../../src/input/decoder.c:608
    #7 0x7feba02ee139 in decoder_GetDecoderDevice ../../include/vlc_codec.h:304
    #8 0x7feba02fb51e in lavc_UpdateVideoFormat ../../modules/codec/avcodec/video.c:286
    #9 0x7feba0313ff7 in ffmpeg_GetFormat ../../modules/codec/avcodec/video.c:1682
    #10 0x7feb9f0bfe12  (/usr/lib/libavcodec.so.58+0x29ae12)

Direct leak of 56 byte(s) in 1 object(s) allocated from:
    #0 0x7febb6bf3459 in __interceptor_calloc /build/gcc/src/gcc/libsanitizer/asan/asan_malloc_linux.cpp:154
    #1 0x7feb8b5d386a  (<unknown module>)
    #2 0x7febb5f195c1 in decoder_device_Open ../../src/input/decoder_helpers.c:174
    #3 0x7febb5e67106 in vlc_module_load ../../src/modules/modules.c:243
    #4 0x7febb5f196e8 in vlc_decoder_device_Create ../../src/input/decoder_helpers.c:188
    #5 0x7febb60a7435 in vout_GetDevice ../../src/video_output/video_output.c:2244
    #6 0x7febb5ef7f26 in ModuleThread_GetDecoderDevice ../../src/input/decoder.c:608
    #7 0x7feba02ee139 in decoder_GetDecoderDevice ../../include/vlc_codec.h:304
    #8 0x7feba02fb51e in lavc_UpdateVideoFormat ../../modules/codec/avcodec/video.c:286
    #9 0x7feba0313ff7 in ffmpeg_GetFormat ../../modules/codec/avcodec/video.c:1682
    #10 0x7feb9f0bfe12  (/usr/lib/libavcodec.so.58+0x29ae12)

changed milestone to %4.0

added Component::Decoders Hw label

changed the description

added 1 commit

• e0962f34 - nvdec: fix multiple call to cuInit

Compare with previous version

resolved all threads

AddresdSanitizer has become totally unusable for me (Debian Unstable), due to what looks a lot like internal sanitizer bugs :( Not challenging the analysis, bit I wonder how you get this.

vlc_once() is supposed to have non-first calls wait for completion of the first call. It would be pretty useless elsewise.

Now that we no longer depend on pthread_once(), we could improve the interface, so that it would take a data pointer and avoid relying on globals, but that wouldn't help much at all in this particular case.

Note that this fix won't work if the procesd uses CUDA through some other path, outside LibVLC or underneath a contrib (maybe FFmpeg could?).

It also will continue to leak if LibVLC is destroyed and recreated multiple times, as the nvdec module would be unloaded, depending on the platform.

resolved all threads

added 43 commits

• e0962f34...8a0834af - 42 commits from branch videolan:master
• d837851b - nvdec: fix multiple call to cuInit

Compare with previous version

marked this merge request as ready

added 1 commit

• 1c37c0f1 - nvdec: fix multiple call to cuInit

Compare with previous version

resolved all threads

added MRStatus::NotCompliant label

added MRStatus::Acceptable label and removed MRStatus::NotCompliant label

added MRStatus::Accepted label and removed MRStatus::Acceptable label

MR Acceptance result

This MergeRequest has been Accepted! Congratulations.

MR acceptance checks details:

• MR should be considered mergeable by Gitlab
• Last pipeline should be successful
• MergeRequest should have at least one external review and/or vote
• All threads should be resolved, and score >= 0
• MergeRequest should have no activity (threads/votes) for (24h/24h)

This message was automatically generated by homer-bot.

added 30 commits

• 1c37c0f1...53ec7adc - 29 commits from branch videolan:master
• 2dfb7d84 - nvdec: fix multiple call to cuInit

Compare with previous version

merged