Skip to content

avcodec: fix crash in FFMPEG when VAAPI is initialized but can't be used

Pierre Lamot requested to merge chub/vlc:vaapi/hwcctxbug into master

the typical case is an application that use vgl callback with a GLX context, vaapi decoder will initiliaze properly and provide a hw_frames_ctx context, but due to the lack of interop, the decoder won't be usable and fallback to the sw decoder, but hw_frames_ctx will still be set and will cause a crash later on.

note that I'm not sure this is the right fix for the issue, as the documentation of hw_frames_ctx states that

If the default get_buffer2() is used with a hwaccel pixel format, then this AVHWFramesContext will be used for the frame buffers.

But if we're fallbacking to SW decoding, shouldn't we not have a hwaccel pixel format?

Merge request reports

VideoLAN code repository instance