shaders: add custom shader support

Merged Niklas Haas requested to merge custom_shaders into master

This finally gives us feature parity with a long-awaited and long-requested part of mpv, namely the ability to use custom user shaders (in "mpv hook" style).

Semantically, we split up the implementation into two halves, with the pl_hook abstraction mediating between them:

  1. The component that parses mpv user shaders into pl_hook objects, available as pl_mpv_user_shader_parse, and
  2. The interaction API between the pl_hook objects and the pl_renderer, available via pl_hook_params.

To ensure this abstraction is as clean as possible, most of the mpv-specific logic (such as "saving" and "binding" textures) is handled inside the mpv_user_shader half of the abstraction, with the general API being very abstract (you can specify what stages to hook, and run an arbitrary function, with the only bit of messiness being a single callback needed to allocate new intermediate FBOs).

The long-term plan is to support hook types other than mpv user shaders, such as ones provided by any of the widely available postprocessing shader libraries, but in the short term, I decided to target the mpv hook system first because the code is mostly written (just needed to be ported from mpv), a lot of popular video shaders are available for this format, and I already know its design is compatible with pl_renderer without major changes.

It's worth pointing out that we have one important API break with mpv:

  • MAIN has been merged with MAINPRESUB, since libplacebo draws overlays at a later stage (after linearization and sigmoidization) for efficiency, and having MAIN imply linearization etc. does not make sense, semantically. Unfortunately, the new stage PL_HOOK_PRE_OVERLAY does not map to any current mpv hook stage.

The code contains a FIXME, but it's for a hypothetical scenario that's nontrivial to work out and could be deferred to this limitation actually being hit in practice.

Tested by going through a number of easily accessible user shaders (RAVU, SSimSuperRes, FilmGrain, antiringing etc.). This is not yet tested for planar content, so testing remains to be done to ensure e.g. LUMA and CHROMA continue being aligned even after prescalers.

Closes #19 (closed) Closes #21 (closed)

Merge request reports