Shader error playing 4k video encoded with hevc and yuv420p10le on Kubuntu 20.10
After upgrading to Kubuntu 20.10 from Kubuntu 20.04, VLC is unable to play some video files that are 2160p, encoded with HEVC and with yuv420p10le colors.
VLC is unable to output the video content to egl_x11, regular x11 output (non-accelerated) works fine, the sound also works fine. This bug does not occur with every 2160p, HEVC, yuv420p10le video file, only some of them, but I can't determine what particular element causes this bug.
I can provide a sample of video files, which cause this bug. The bug did not occur with previous version of VLC that came with Kubuntu 20.04, the very same files were played without a problem, on the very same hardware.
Using a vanilla VLC with default settings still causes this bug.
For brevity I'm only providing the relevant parts of these outputs:
VLC version:
VLC media player 3.0.11.1 Vetinari (revision 3.0.11.1-0-g52483f3ca2)
vlc -vvv example.mkv
[00007f3814003f10] egl_x11 gl debug: EGL version 1.5 by Mesa Project
[00007f3814003f10] egl_x11 gl debug: extensions: EGL_ANDROID_blob_cache EGL_ANDROID_native_fence_sync EGL_CHROMIUM_sync_control EGL_EXT_buffer_age EGL_EXT_create_context_robustness EGL_EXT_image_dma_buf_import EGL_EXT_image_dma_buf_import_modifiers EGL_EXT_swap_buffers_with_damage EGL_KHR_cl_event2 EGL_KHR_config_attribs EGL_KHR_create_context EGL_KHR_create_context_no_error EGL_KHR_fence_sync EGL_KHR_get_all_proc_addresses EGL_KHR_gl_colorspace EGL_KHR_gl_renderbuffer_image EGL_KHR_gl_texture_2D_image EGL_KHR_gl_texture_3D_image EGL_KHR_gl_texture_cubemap_image EGL_KHR_image EGL_KHR_image_base EGL_KHR_image_pixmap EGL_KHR_no_config_context EGL_KHR_reusable_sync EGL_KHR_surfaceless_context EGL_KHR_swap_buffers_with_damage EGL_EXT_pixel_format_float EGL_KHR_wait_sync EGL_MESA_configless_context EGL_MESA_drm_image EGL_MESA_image_dma_buf_export EGL_MESA_query_driver EGL_NOK_texture_from_pixmap EGL_WL_bind_wayland_display
[00007f3814003f10] main gl debug: using opengl module "egl_x11"
[00007f3814003f10] gl gl: Initialized libplacebo v2.72.0 (API v72)
[00007f381416e050] main generic debug: looking for glconv module matching "any": 4 candidates
libva info: VA-API version 1.8.0
libva info: Trying to open /usr/lib/x86_64-linux-gnu/dri/radeonsi_drv_video.so
libva info: Found init function __vaDriverInit_1_8
libva info: va_openDriver() returns 0
mesa: for the --simplifycfg-sink-common option: may only occur zero or one times!
mesa: for the --global-isel-abort option: may only occur zero or one times!
mesa: for the --amdgpu-atomic-optimizations option: may only occur zero or one times!
mesa: for the --structurizecfg-skip-uniform-regions option: may only occur zero or one times!
[00007f381416e050] main generic debug: using glconv module "glconv_vaapi_x11"
[00007f3814003f10] gl gl error: shader 0: 0:19(13): error: no matching function for call to `mix(vec3, vec3, bvec3)'; candidates are:
0:19(13): error: float mix(float, float, float)
0:19(13): error: vec2 mix(vec2, vec2, float)
0:19(13): error: vec3 mix(vec3, vec3, float)
0:19(13): error: vec4 mix(vec4, vec4, float)
0:19(13): error: vec2 mix(vec2, vec2, vec2)
0:19(13): error: vec3 mix(vec3, vec3, vec3)
0:19(13): error: vec4 mix(vec4, vec4, vec4)
[00007f3814003f10] gl gl error: shader program: error: linking with uncompiled/unspecialized shader
[00007f3814003f10] gl gl error: Unable to use program
[00007f381416e050] main generic debug: removing module "glconv_vaapi_x11"
[00007f3814003f10] gl gl warning: could not init tex converter for VAO0
[00007f3814003f10] main gl debug: removing module "egl_x11"
[00007f38140013b0] main vout display debug: no vout display modules matched
[00007f381c5c4c20] main video output error: video output creation failed
lspci
29:00.0 VGA compatible controller: Advanced Micro Devices, Inc. [AMD/ATI] Ellesmere [Radeon RX 470/480/570/570X/580/580X/590] (rev ef)
29:00.1 Audio device: Advanced Micro Devices, Inc. [AMD/ATI] Ellesmere HDMI Audio [Radeon RX 470/480 / 570/580/590]
uname -a
Linux kubuntu-pc 5.8.0-33-generic [#36](https://code.videolan.org/videolan/vlc/-/issues/36)-Ubuntu SMP Wed Dec 9 09:14:40 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux
ffprobe example.mkv
ffprobe version 4.2.4-1ubuntu0.1 Copyright (c) 2007-2020 the FFmpeg developers
built with gcc 9 (Ubuntu 9.3.0-10ubuntu2)
configuration: --prefix=/usr --extra-version=1ubuntu0.1 --toolchain=hardened --libdir=/usr/lib/x86_64-linux-gnu --incdir=/usr/include/x86_64-linux-gnu --arch=amd64 --enable-gpl --disable-stripping --enable-avresample --disable-filter=resample --enable-avisynth --enable-gnutls --enable-ladspa --enable-libaom --enable-libass --enable-libbluray --enable-libbs2b --enable-libcaca --enable-libcdio --enable-libcodec2 --enable-libflite --enable-libfontconfig --enable-libfreetype --enable-libfribidi --enable-libgme --enable-libgsm --enable-libjack --enable-libmp3lame --enable-libmysofa --enable-libopenjpeg --enable-libopenmpt --enable-libopus --enable-libpulse --enable-librsvg --enable-librubberband --enable-libshine --enable-libsnappy --enable-libsoxr --enable-libspeex --enable-libssh --enable-libtheora --enable-libtwolame --enable-libvidstab --enable-libvorbis --enable-libvpx --enable-libwavpack --enable-libwebp --enable-libx265 --enable-libxml2 --enable-libxvid --enable-libzmq --enable-libzvbi --enable-lv2 --enable-omx --enable-openal --enable-opencl --enable-opengl --enable-sdl2 --enable-libdc1394 --enable-libdrm --enable-libiec61883 --enable-nvenc --enable-chromaprint --enable-frei0r --enable-libx264 --enable-shared
libavutil 56. 31.100 / 56. 31.100
libavcodec 58. 54.100 / 58. 54.100
libavformat 58. 29.100 / 58. 29.100
libavdevice 58. 8.100 / 58. 8.100
libavfilter 7. 57.100 / 7. 57.100
libavresample 4. 0. 0 / 4. 0. 0
libswscale 5. 5.100 / 5. 5.100
libswresample 3. 5.100 / 3. 5.100
libpostproc 55. 5.100 / 55. 5.100
Input [#0](https://code.videolan.org/videolan/vlc/-/issues/0), matroska,webm, from 'example.mkv':
Metadata:
encoder : libebml v1.4.0 + libmatroska v1.6.0
creation_time : 2020-11-07T15:22:14.000000Z
Duration: 00:54:18.08, start: 0.000000, bitrate: 20334 kb/s
Chapter [#0](https://code.videolan.org/videolan/vlc/-/issues/0):0: start 0.000000, end 590.960000
Metadata:
title : Chapter 1
Chapter [#0](https://code.videolan.org/videolan/vlc/-/issues/0):1: start 590.960000, end 1255.800000
Metadata:
title : Chapter 2
Chapter [#0](https://code.videolan.org/videolan/vlc/-/issues/0):2: start 1255.800000, end 1860.920000
Metadata:
title : Chapter 3
Chapter [#0](https://code.videolan.org/videolan/vlc/-/issues/0):3: start 1860.920000, end 2466.560000
Metadata:
title : Chapter 4
Chapter [#0](https://code.videolan.org/videolan/vlc/-/issues/0):4: start 2466.560000, end 2930.400000
Metadata:
title : Chapter 5
Chapter [#0](https://code.videolan.org/videolan/vlc/-/issues/0):5: start 2930.400000, end 3258.080000
Metadata:
title : Chapter 6
Stream [#0](https://code.videolan.org/videolan/vlc/-/issues/0):0: Video: hevc (Main 10), yuv420p10le(tv, bt2020nc/bt2020/arib-std-b67), 3840x2160 [SAR 1:1 DAR 16:9], 25 fps, 25 tbr, 1k tbn, 25 tbc (default)
Metadata:
BPS-eng : 18597593
DURATION-eng : 00:54:18.080000000
NUMBER_OF_FRAMES-eng: 81449
NUMBER_OF_BYTES-eng: 7574056055
_STATISTICS_WRITING_APP-eng: mkvmerge v48.0.0 ('Fortress Around Your Heart') 64-bit
_STATISTICS_WRITING_DATE_UTC-eng: 2020-11-07 15:22:14
_STATISTICS_TAGS-eng: BPS DURATION NUMBER_OF_FRAMES NUMBER_OF_BYTES
Stream [#0](https://code.videolan.org/videolan/vlc/-/issues/0):1(eng): Audio: dts (DTS-HD MA), 48000 Hz, 5.1(side), s16p (default)
Metadata:
BPS-eng : 1733591
DURATION-eng : 00:54:17.856000000
NUMBER_OF_FRAMES-eng: 305424
NUMBER_OF_BYTES-eng: 705974092
_STATISTICS_WRITING_APP-eng: mkvmerge v48.0.0 ('Fortress Around Your Heart') 64-bit
_STATISTICS_WRITING_DATE_UTC-eng: 2020-11-07 15:22:14
_STATISTICS_TAGS-eng: BPS DURATION NUMBER_OF_FRAMES NUMBER_OF_BYTES
Stream [#0](https://code.videolan.org/videolan/vlc/-/issues/0):2(eng): Subtitle: subrip
Metadata:
BPS-eng : 44
DURATION-eng : 00:53:49.720000000
NUMBER_OF_FRAMES-eng: 614
NUMBER_OF_BYTES-eng: 17994
_STATISTICS_WRITING_APP-eng: mkvmerge v48.0.0 ('Fortress Around Your Heart') 64-bit
_STATISTICS_WRITING_DATE_UTC-eng: 2020-11-07 15:22:14
_STATISTICS_TAGS-eng: BPS DURATION NUMBER_OF_FRAMES NUMBER_OF_BYTES
Stream [#0](https://code.videolan.org/videolan/vlc/-/issues/0):3(eng): Subtitle: subrip
Metadata:
title : SDH
BPS-eng : 47
DURATION-eng : 00:53:49.720000000
NUMBER_OF_FRAMES-eng: 697
NUMBER_OF_BYTES-eng: 19168
_STATISTICS_WRITING_APP-eng: mkvmerge v48.0.0 ('Fortress Around Your Heart') 64-bit
_STATISTICS_WRITING_DATE_UTC-eng: 2020-11-07 15:22:14
_STATISTICS_TAGS-eng: BPS DURATION NUMBER_OF_FRAMES NUMBER_OF_BYTES