opengl: simplify TexCoordsMaps in sampler
The short-term goal is simplify the code (both C and GLSL).
But in the end, I'd like to modify the OpenGL filters API to expose the input textures to the filters, and let them compute the texture coordinates in CPU (so that the matrix multiplications are not executed from the fragment shader). By making the coordinates transformation identical for all planes, this will allow to simplify the API.
The input picture may contain padding, for two reasons:
- the rectangle defined by the picture_t offset and visible width and height may not cover the whole picture size;
- if the OpenGL API does not support NPOT textures, textures are created with power-of-two dimensions.
As a consequence, for sampling, the coordinates (between 0 and 1 in both dimensions) must be transformed to take the padding into account.
These transformations were computed and performed per-plane. Therefore, there were n "TexCoordsMaps" matrices for a picture with n planes.
However, these transformations are necessarily the same for all planes:
- the picture_t offset/sizes are defined for the whole picture (then scaled appropriately for each plane);
- since the ratio between different planes of the same pictures is always a power of two (e.g. in YUV 4:2:0, the size of U and V planes are half the size of the Y plane in both dimensions), this property holds even if we align the dimensions of each plane texture to the next power-of-two.
Therefore, use a single transformation matrix.
Concretely, before the change, the TexCoordsMaps
were applied separately for each plane (this is an fragment shader with an YUV 4:2:0 input):
uniform mat4 TransformMatrix;
uniform mat4 OrientationMatrix;
uniform sampler2D Textures[3];
uniform mat3 TexCoordsMaps[3];
uniform mat4 ConvMatrix;
vec4 vlc_texture(vec2 pic_coords) {
vec3 pic_hcoords = vec3((TransformMatrix * OrientationMatrix * vec4(pic_coords, 0.0, 1.0)).st, 1.0);
vec2 tex_coords;
vec4 texel;
vec4 pixel = vec4(0.0, 0.0, 0.0, 1.0);
tex_coords = (TexCoordsMaps[0] * pic_hcoords).st;
texel = texture2D(Textures[0], tex_coords);
pixel[0] = texel.r;
tex_coords = (TexCoordsMaps[1] * pic_hcoords).st;
texel = texture2D(Textures[1], tex_coords);
pixel[1] = texel.r;
tex_coords = (TexCoordsMaps[2] * pic_hcoords).st;
texel = texture2D(Textures[2], tex_coords);
pixel[2] = texel.r;
vec4 result = ConvMatrix * pixel;
result = _main_0_0(result);
return result;
}
After this change, it is applied once for the whole picture:
uniform mat4 TransformMatrix;
uniform mat4 OrientationMatrix;
uniform sampler2D Textures[3];
uniform mat3 TexCoordsMap;
uniform mat4 ConvMatrix;
vec4 vlc_texture(vec2 pic_coords) {
vec2 tex_coords = (TexCoordsMap * (TransformMatrix * OrientationMatrix * vec4(pic_coords, 0.0, 1.0)).xyz).xy;
vec4 texel;
vec4 pixel = vec4(0.0, 0.0, 0.0, 1.0);
texel = texture2D(Textures[0], tex_coords);
pixel[0] = texel.r;
texel = texture2D(Textures[1], tex_coords);
pixel[1] = texel.r;
texel = texture2D(Textures[2], tex_coords);
pixel[2] = texel.r;
vec4 result = ConvMatrix * pixel;
return result;
}
Merge request reports
Activity
changed milestone to %4.0
added Component::Core: Video output label
- Resolved by Romain Vimont
Tip for testing: turn of NPOT support.
diff --git a/modules/video_output/opengl/gl_api.c b/modules/video_output/opengl/gl_api.c index 92db2206f1..f15878efe6 100644 --- a/modules/video_output/opengl/gl_api.c +++ b/modules/video_output/opengl/gl_api.c @@ -191,5 +191,7 @@ vlc_gl_api_Init(struct vlc_gl_api *api, vlc_gl_t *gl) vlc_gl_StrHasToken(api->extensions, "GL_APPLE_texture_2D_limited_npot"); } + api->supports_npot = false; + return VLC_SUCCESS; }
This MR has been tested successfully on Linux and Android.
I'd be grateful if someone could test on Windows and macOS/iOS (just play a video with the opengl vout and npot support disabled). Thank you :)
added MRStatus::Reviewable label
- Resolved by Romain Vimont
added MRStatus::Acceptable label and removed MRStatus::Reviewable label
added MRStatus::InReview label and removed MRStatus::Acceptable label
mentioned in issue #26105
added MRStatus::Acceptable label and removed MRStatus::InReview label
added MRStatus::Accepted label and removed MRStatus::Acceptable label
MR Acceptance result
This MergeRequest has been Accepted! Congratulations.MR acceptance checks details:
-
MR should be considered mergeable by Gitlab -
Last pipeline should be successful -
MergeRequest should have at least one external review and/or vote -
All threads should be resolved, and score >= 0 -
MergeRequest should have no activity (threads/votes) for (24h/24h)
-
added MRStatus::InReview label and removed MRStatus::Accepted label
added 162 commits
-
4882eada...48d50cdf - 160 commits from branch
videolan:master
- 893aa567 - opengl: simplify TexSize multiplication
- 499b7ef7 - opengl: simplify TexCoordsMaps in sampler
-
4882eada...48d50cdf - 160 commits from branch