Skip to content

opengl: render YUY2 at full definition

Romain Vimont requested to merge rom1v/vlc:yuy2_opengl into master

Some YUV 4:2:2 formats are packed: two consecutive pixels are stored using 4 values (for example [Y1 U Y2 V]).

But in OpenGL we cannot have both:

  • a single texture, and
  • a correct native interpolation for both Y and UV components.

To avoid the problem, the current implementation just dropped the Y2 value, so the pictures were rendered at half the horizontal resolution.

To render at full definition, upload the single plane into two separate OpenGL textures:

  • one in GL_RG, to read each Y value in a different texel;
  • one in GL_RGBA, to read both U and V values in a single texel.

As a consequence, pic->i_planes is not necessarily equal to interop->tex_count anymore (there might be 1 picture plane but 2 textures).

Fixes #26712 (closed)


From #26712 (closed):

Alternatively, we could upload the same picture twice (😱), once in GL_RG to access the Y components, once in GL_RGBA to access the UV components, and keep the native OpenGL interpolation.

I choose this solution because it is the most straightforward to implement (with a limited impact on source code and architecture) and it provides correct rendering of YUV 4:2:2 pictures "easily".

More details from this comment:

Here is the storage of the original plane:

Y0 U0 Y1 V0 Y2 U2 Y3 V2
Y4 U4 Y5 V4 Y6 U6 Y7 V6

When uploading to a GL_RG texture, the texels are split as follow:

+-----+-----+-----+-----+
|Y0 U0|Y1 V0|Y2 U2|Y3 V2|
+-----+-----+-----+-----+
|Y4 U4|Y5 V4|Y6 U6|Y7 V6|
+-----+-----+-----+-----+

So we can access Y values from the shader using .r. For example, if we call texture2D() with coordinates inside the square formed by Y0-Y1-Y4-Y5, the resulting first component will be an interpolation of Y0 Y1 Y4 and Y5 values.

When uploading to a GL_RGBA texture, the texels are split as follow:

+-----------+-----------+
|Y0 U0 Y1 V0|Y2 U2 Y3 V2|
+-----------+-----------+
|Y4 U4 Y5 V4|Y6 U6 Y7 V6|
+-----------+-----------+

This allows to access U and V values together using .g and .a. For example, if we call texture2D() with coordinates inside the square formed by Y0-Y2-Y4-Y6, the resulting second component will be an interpolation of U0 U2 U4 and U6, and the resulting forth component will be an interpolation of V0 V2 V4 and V6.

EDIT: to test a YUYV 4:2:2 stream: vlc v4l2:///dev/videoX:chroma=YUYV (comment)

Edited by Romain Vimont

Merge request reports