GitLab

The version and precision header strings are different for OpenGL and
OpenGL ES.

Insert them directly into the shader string literal, instead of
initializing renderer fields and inserting them at runtime.

The renderer format is, by definition, the one of its sampler. There is
no need to copy it.

Uploading a picture_t to OpenGL flips it vertically.

To display the video in the correct orientation, the renderer reversed
the transformation: each vertex received as attribute texture
coordinates (x, 1-y) instead of (x, y), in order to flip the image
vertically.

This caused several problems.

Firstly, the renderer must be independent of the input picture storage.
Otherwise, when it will receive an input in the correct orientation, it
will wrongly apply a vertical flip.

Secondly, since the vflip was applied on the input coordinates, it
occurred before the orientation transform:

    OrientationMatrix * VFlipMatrix * input_coords  (semantically)

whereas the correct transformation is:

    VFlipMatrix * OrientationMatrix * input_coords

(for reasons explained in e329c4bb).

Since matrix multiplication is not commutative, it resulted in a
wrong orientation in some cases (for example if OrientationMatrix
contains a rotation by 90°).

(In fact, in pratice, this case still worked, because the initialization
of OrientationMatrix was also wrong, see the two previous commits).

To fix all these problems, initialize the texture coordinates in the
normal orientation in the renderer, and apply the vertical flip using
the TransformationMatrix in the sampler.

This also allows to simplify the Android interop, which can just provide
its own transform matrix without compensating for the vertical flip
that was applied in the renderer.

The purpose of this callback is to load sampler data (bind textures,
load uniforms, etc.) to be used for the OpenGL draw call.

The name "load" also allows to possibly add an "unload" function later
(to reset bindings).

This makes explicit that these functions are set by the sampler
implementation and must be called by the user of sampler.

If GetUniformLocation() returns -1, then this is an implementation
error.

Assert instead of reporting the error to the caller.

Only the renderer dumps its shaders depending on the verbose level.

The function vlc_gl_renderer_Draw() (almost) just called
DrawWithShaders(). There is no need for a separate function.

The renderer is expected to be independant of the input picture format.
Instead, update the sampler directly.

Remove the remaining usage of interop from renderer.

Now, the renderer only need to use the sampler.

The sampler were owned by the renderer.

Move it to vgl, so that the renderer never have to handle an interop
instance directly.

The renderer should not know the interop, so it may not be responsible
to upload the textures.

Move the pictures update to sampler.

The orentation matrix is managed by the sampler. Move its initialization
to opengl_fragment_shader_init() (called by the sampler).

Move opengl_fragment_shader_init() call from renderer to sampler.

The sampler handles the input pictures, so it is also responsible for
generating the necessary GLSL extensions code (if any).

The function opengl_fragment_shader_init() both initializes the
and creates the fragment shader.

To be able to move its initialization outside the renderer, store it in
the sampler instead of returning it. This will also allow to provide the
"extensions" part of the fragment shader as a separate string.

The renderer instance is not needed anymore.

Remove the (weak) reference to the interop from the renderer structure.

The interop is still accessed explicitly from the renderer via the
sampler structure for now.

The fact that the sampler use internal function pointers (instead of
if-blocks for example) is an internal detail.

Remove tex_width and tex_height parameters, as they are either unused or
accessible from the sampler itself.

The alpha value was used to render subpictures, which are now drawn by
the subpictures renderer using its own fragment shader.

Load uniforms related to sampler from sampler->pf_prepare_shader.

Fetch locations related to sampler from sampler->pf_fetch_locations.

The sampler is responsible to fetch its location and initialize its
textures and uniforms.

This will allow to use them from several files.

Extract sampler creation and destruction to separate functions.

This allows to use an interop instance without a renderer.

The interop initializes its own video_format_t from the input format.

In the renderer initialization, always use this updated format. This
avoids an additional format parameter, which may be confusing.

The interop may modify its own copy of the video_format_t.

These changes must be reported to the core (by writing to the provided
video_format_t), except for the orientation: it is managed internally by
OpenGL, so from the core point of view, the orientation does not change.

Handle the format to report to the core outside of the renderer. This
paves the way to pass only the interop to the renderer, without an
additional video_format_t instance.

The interop instances were owned by the renderers.

To prepare for making the renderers independant of the input picture
(in particular, independant of the interop), make the vgl own the
interop instances.

Move the fields necessary to retrieve a picture pixel from its
coordinates.

Make the renderer create the fragment shader, using the vlc_texture()
function generated by fragment_shader.c.

Romain Vimont authored Jan 31, 2020 and Jean-Baptiste Kempf committed Mar 23, 2020

The video_format_t parameter of vout_display_opengl_Display() is not
used anymore. Remove it.
Signed-off-by: Jean-Baptiste Kempf <jb@videolan.org>

Romain Vimont authored Jan 31, 2020 and Jean-Baptiste Kempf committed Mar 23, 2020

Texture paddings were computed in _Display(), while the picture format
is known in _Prepare().

This will allow to remove the video_format_t parameter of _Display().
Signed-off-by: Jean-Baptiste Kempf <jb@videolan.org>

Romain Vimont authored Jan 31, 2020 and Jean-Baptiste Kempf committed Mar 23, 2020

It will be modified to depend on static functions. Move it below them.

This will make the diff more readable.
Signed-off-by: Jean-Baptiste Kempf <jb@videolan.org>

Romain Vimont authored Feb 04, 2020 and Jean-Baptiste Kempf committed Mar 23, 2020

After the previous changes, two successive loops iterating tex_count
times can now be merged into one.
Signed-off-by: Jean-Baptiste Kempf <jb@videolan.org>

Romain Vimont authored Feb 04, 2020 and Jean-Baptiste Kempf committed Mar 23, 2020

The multiview mode (stereo) was applied by changing the texture
paddings to crop only the left eye.

The problem is that these texture paddings are specific to the input
picture, while the stereo mode is specific to the renderer, which should
be independent of the input picture.

To separate these concerns, apply the stereo mode using a matrix to
transform the texture coordinates from the renderer.
Signed-off-by: Jean-Baptiste Kempf <jb@videolan.org>

Romain Vimont authored Feb 04, 2020 and Jean-Baptiste Kempf committed Mar 23, 2020

Apply all texture coordinates transformation in the fragment shader.

This is necessary to be able to implement a renderer independent of the
input picture (i.e. without even knowing the details) by using a single
GLSL function "vlc_texture(vec2 pic_coords)".
Signed-off-by: Jean-Baptiste Kempf <jb@videolan.org>

Romain Vimont authored Jan 31, 2020 and Jean-Baptiste Kempf committed Mar 23, 2020

Now that the coordinates are independant of paddings (which are
transmitted via a separate transform matrix), there is no need to
recompute them when the paddings change.
Signed-off-by: Jean-Baptiste Kempf <jb@videolan.org>

Romain Vimont authored Jan 29, 2020 and Jean-Baptiste Kempf committed Mar 23, 2020

A picture is stored in OpenGL textures (one per plane), possibly with
padding (the texture may be larger than the actual picture).

The conversion from picture coordinates to texture coordinates (which
takes the padding into account) was applied on the input coordinates,
before the vertex shader. As a consequence, the vertex shader received
one vector of input texture coordinates per plane (the padding is not
necessarily the same for all the planes):

    (before this commit)

   picture   texture
   coords    coords        (attributes)      (varyings)
          (1 per plane)

             (x0, y0) --> MultiTexCoord0     TexCoord0     fragment
   (x,y) --> (x1, y1) --> MultiTexCoord1 --> TexCoord1 --> shader
             (x2, y2) --> MultiTexCoord2     TexCoord2

This poses a problem to separate chroma conversion from rendering: the
renderer should be able to retrieve a pixel color in picture
coordinates, regarless of the input format or padding.

To solve this issue, pass the picture coordinates instead of the texture
coordinates as attribute, and initialize uniform matrices to convert
from picture to texture coordinates for each plane directly in the
fragment shader:

    (after this commit)

   picture
   coords    (attribute)     (varying)

   (x,y) --> PicCoordsIn --> PicCoords --> fragment shader
                                             ^^^
                                             |||
                             TexCoordsMap0 --'||
                (uniforms)   TexCoordsMap1 ---'|
                             TexCoordsMap2 ----'

Note that this also changes the multiplication order of
(non-commutative) matrices, from (semantically):

    TexCoords = Orientation * TexCoordsMap * PicCoords

to:

    TexCoords = TexCoordsMap * Orientation * PicCoords

The latter is the correct one: the orientation defines how the input
picture is rotated, so it must apply to picture coordinates, regardless
of the actual coordinates in the texture.

As a side effect, BuildRectangle, BuildSphere and BuildCube are now
independant of both the number of planes and any texture padding.

For now, TexCoordsMap is computed from the renderer, but the goal is to
move it to a separate component (a "sampler").
Signed-off-by: Jean-Baptiste Kempf <jb@videolan.org>