Skip to content
Snippets Groups Projects

opengl: simplify TexCoordsMaps in sampler

Merged Romain Vimont requested to merge rom1v/vlc:opengl_texcoordsmaps into master

The short-term goal is simplify the code (both C and GLSL).

But in the end, I'd like to modify the OpenGL filters API to expose the input textures to the filters, and let them compute the texture coordinates in CPU (so that the matrix multiplications are not executed from the fragment shader). By making the coordinates transformation identical for all planes, this will allow to simplify the API.


The input picture may contain padding, for two reasons:

  1. the rectangle defined by the picture_t offset and visible width and height may not cover the whole picture size;
  2. if the OpenGL API does not support NPOT textures, textures are created with power-of-two dimensions.

As a consequence, for sampling, the coordinates (between 0 and 1 in both dimensions) must be transformed to take the padding into account.

These transformations were computed and performed per-plane. Therefore, there were n "TexCoordsMaps" matrices for a picture with n planes.

However, these transformations are necessarily the same for all planes:

  1. the picture_t offset/sizes are defined for the whole picture (then scaled appropriately for each plane);
  2. since the ratio between different planes of the same pictures is always a power of two (e.g. in YUV 4:2:0, the size of U and V planes are half the size of the Y plane in both dimensions), this property holds even if we align the dimensions of each plane texture to the next power-of-two.

Therefore, use a single transformation matrix.


Concretely, before the change, the TexCoordsMaps were applied separately for each plane (this is an fragment shader with an YUV 4:2:0 input):

uniform mat4 TransformMatrix;
uniform mat4 OrientationMatrix;
uniform sampler2D Textures[3];
uniform mat3 TexCoordsMaps[3];

uniform mat4 ConvMatrix;
vec4 vlc_texture(vec2 pic_coords) {
 vec3 pic_hcoords = vec3((TransformMatrix * OrientationMatrix * vec4(pic_coords, 0.0, 1.0)).st, 1.0);
 vec2 tex_coords;
 vec4 texel;
 vec4 pixel = vec4(0.0, 0.0, 0.0, 1.0);
 tex_coords = (TexCoordsMaps[0] * pic_hcoords).st;
 texel = texture2D(Textures[0], tex_coords);
 pixel[0] = texel.r;
 tex_coords = (TexCoordsMaps[1] * pic_hcoords).st;
 texel = texture2D(Textures[1], tex_coords);
 pixel[1] = texel.r;
 tex_coords = (TexCoordsMaps[2] * pic_hcoords).st;
 texel = texture2D(Textures[2], tex_coords);
 pixel[2] = texel.r;
 vec4 result = ConvMatrix * pixel;
 result = _main_0_0(result);
 return result;
}

After this change, it is applied once for the whole picture:

uniform mat4 TransformMatrix;
uniform mat4 OrientationMatrix;
uniform sampler2D Textures[3];
uniform mat3 TexCoordsMap;

uniform mat4 ConvMatrix;
vec4 vlc_texture(vec2 pic_coords) {
 vec2 tex_coords = (TexCoordsMap * (TransformMatrix * OrientationMatrix * vec4(pic_coords, 0.0, 1.0)).xyz).xy;
 vec4 texel;
 vec4 pixel = vec4(0.0, 0.0, 0.0, 1.0);
 texel = texture2D(Textures[0], tex_coords);
 pixel[0] = texel.r;
 texel = texture2D(Textures[1], tex_coords);
 pixel[1] = texel.r;
 texel = texture2D(Textures[2], tex_coords);
 pixel[2] = texel.r;
 vec4 result = ConvMatrix * pixel;
 return result;
}
Edited by Romain Vimont

Merge request reports

Merge request pipeline #140869 passed

Merge request pipeline passed for 499b7ef7

Merged by Romain VimontRomain Vimont 3 years ago (Sep 23, 2021 12:11pm UTC)

Loading

Pipeline #140875 passed

Pipeline passed for 499b7ef7 on master

Activity

Filter activity
  • Approvals
  • Assignees & reviewers
  • Comments (from bots)
  • Comments (from users)
  • Commits & branches
  • Edits
  • Labels
  • Lock status
  • Mentions
  • Merge request status
  • Tracking
  • Romain Vimont added 1 commit

    added 1 commit

    • a5505968 - opengl: simplify TexCoordsMaps in sampler

    Compare with previous version

  • Romain Vimont added 1 commit

    added 1 commit

    • 1793a329 - opengl: simplify TexCoordsMaps in sampler

    Compare with previous version

  • Romain Vimont resolved all threads

    resolved all threads

  • Romain Vimont mentioned in issue #26105

    mentioned in issue #26105

  • Romain Vimont resolved all threads

    resolved all threads

  • Romain Vimont added 1 commit

    added 1 commit

    • 4882eada - opengl: simplify TexCoordsMaps in sampler

    Compare with previous version

  • MR Acceptance result

    :tada: This MergeRequest has been Accepted! Congratulations.

    MR acceptance checks details:

    • :white_check_mark: MR should be considered mergeable by Gitlab
    • :white_check_mark: Last pipeline should be successful
    • :white_check_mark: MergeRequest should have at least one external review and/or vote
    • :white_check_mark: All threads should be resolved, and score >= 0
    • :white_check_mark: MergeRequest should have no activity (threads/votes) for (24h/24h)

    This message was automatically generated by homer-bot.

  • Romain Vimont added 162 commits

    added 162 commits

    Compare with previous version

  • Romain Vimont resolved all threads

    resolved all threads

  • Loading
  • Loading
  • Loading
  • Loading
  • Loading
  • Loading
  • Loading
  • Loading
  • Loading
  • Loading
  • Please register or sign in to reply
    Loading