Skip to content

Refactor OpenGL filters API and implementation

Romain Vimont requested to merge rom1v/vlc:opengl_filters_apiv2 into master

TL;DR: This patchset moves what we call a sampler out of the filter engine, to expose it as a simple (optional) helper for OpenGL filters. This will allow filters to be able to access the input texture directly, or use their own intermediate textures as input for the sampler. In practice, it will make possible to use libplacebo upscalers from an OpenGL filter.


An OpenGL filter (vlc_gl_filter) could only access its input picture using the GLSL function vlc_texture(), injected into their fragment shader. The picture storage was abstracted, so that the filters need not care about interop, number of planes, paddings, orientation, etc. Instead, it could use vlc_texture() to retrieve RGBA pixels for given picture coordinates.

It was convenient, but had some drawbacks.

Firstly, all the linear transformations (orientation, paddings…) were applied from the fragment shader, which is not optimal. As a first step, !707 (merged) removed these transformations from vlc_texture() and exposed them to the filters via a separate API, so that they could transform either in CPU or via their vertex shader.

Secondly, some filters might need to access individual input planes, before chroma conversion. To bypass this constraint for deinterlace filters, a plane filtering feature had been implemented, but it is quite limited (planes may only be filtered independently, the output format must necessarily be the same as the input format, etc.).

Thirdly, some filters might need to a sampler to handle texture access and chroma conversion automatically for pictures others than the input picture of the filter (typically resulting of an intermediate rendering pass).

To remove these constraints for OpenGL filters, the goal of this patchset is to split the sampler into two parts:

  1. a part which imports a VLC picture_t (using an interop) to an "OpenGL picture" (i.e. textures and some meta), handling the necessary coordinates transformations (importer);
  2. a part which exposes an "OpenGL picture" via vlc_texture(), handling texture access, swizzle and chroma conversion (sampler).

This will allow OpenGL filters to:

  • bind and read the raw input textures directly;
  • generate chroma conversion GLSL code for any vlc_gl_picture with a given vlc_gl_format (not only for input VLC pictures).

As a consequence, an OpenGL filter now receives an input format (vlc_gl_format) on its module Open() function, and a (vlc_gl_picture) on draw(). It may decide to create a sampler or not, it's now local to the filter and independent of the filter engine. (As a side effect, it also simplifies some quirks in the filter engine to allow a filter to access its sampler from its Open() function, whereas the sampler depends on the format normally known after Open() has returned.)

The existing filters are adapted (the API has changed), but the impact is minimal and the adaptation quite straightforward.

In practice, it paves the way to implement OpenGL upscalers using libplacebo. In particular, it allows to pass textures of input planes directly to libplacebo.

Refs #26066 (closed)

Edited by Romain Vimont

Merge request reports