Convenience function, we don't care about pl_frame here.

Also deprecate pl_frame_map_avdovi_metadata to reduce API surface in
future, both functions do the same thing. Prefer the one with relaxed
API.

Needed to ensure the extended capabilities required by vf_libplacebo
(i.e. rendering) are actually supported for a given format.

See-Also: https://github.com/haasn/libplacebo/issues/173

To avoid the trappings of users directly accessing `frame->user_data`,
which led to the current mess. Fix the problem cleanly by introducing a
new function.

Also clarify the exact policy here to avoid further misunderstandings.

And actually, exposing this implementation detail was a bad idea in the
first place.

Some integrators may need this for IP reasons.

Kacper Michajłow authored 2 years ago and Niklas Haas committed 2 years ago

Fixes #255

As a helper for adding DOVI metadata to a `pl_frame`.

This not only exposes the previous logic to users, but also adds support
for AVDynamicHDRPlus. Use this function internally in place of the
previous duplicated code, and also get rid of some sanity masking logic
that was of dubious importance (especially now that `nominal_min/max`
are decoupled from the HDR metadata).

For HDR10, we only map the fields relevant/interesting for us, ignore
the rest. (Which is also unused in practice)

Now that we bundle <vulkan.h>, PL_HAVE_VULKAN is no longer a "reliable"
method of detecting whether vulkan is enabled. To be fair, it never was,
but this change revealed this bug.

The real issue here is that <vulkan.h> cannot be unconditionally assumed
to be available, so we need to test for it somehow. Re-introduce the
dynamic check (works on GCC/clang) and leave a scary note hoping to get
MSVC users to include <vulkan.h> manually.

Alongside the basic/trivial deinterlacing modes, this contains an
implementation of the raw yadif deinterlacing kernel, based on
vf_yadif_cuda.cu by @philipl. Currently only supports fragment shaders,
with no compute shader speedup (although such a thing is conceptually
possible, to share duplicate sampling work between adjacent pixels).

Design the API in such a way that it permits field-merging (PAFF) should
the need should arise in the future. Also adds a few related helpers.

Closes: videolan/libplacebo#20

Lynne authored 2 years ago and Niklas Haas committed 2 years ago

Modified-by: Niklas Haas <git@haasn.dev>

See upstream.

Reintroduces dolby vision mapping, in a way that allows users to control
the process, and also allows us to extend this function in the future
should we run into anything similar again.

This is actually a regression for vf_libplacebo. We need to handle this
in a better way. Naively auto-applying it is nice, but causes too many
surprises.

I will think up something different. In the meantime, avoid breaking the
status quo.

Not bothering to bump the API version for this because it's a very
recent change and doesn't affect the public API.

pl_map_avframe / pl_unmap_avframe can take care of this automatically,
but users of the pl_frame_from_avframe or pl_upload_avframe APIs
unfortunately have to do it themselves the hard way.

That's the price they pay for using old APIs, I suppose.

The old `sig_peak`, `sig_floor` etc. values were getting increasingly
confusing, useless, and clumsy to interact with. As the tone mapping
grows in complexity, it's clear that this functionality needs to start
scaling its complexity.

Accomplish this by merging the full HDR metadata struct into
`pl_color_space` itself, and removing/deprecating the `sig_peak` etc.
fields entirely. Since this struct is growing, turn some call-by-value
functions into call-by-ref, for consistency.

Also simplify everything by removing `light`, and just treating the OOTF
as (conceptually) part of the EOTF. In addition, fundamentally refactor
the way SDR curves are handled, by always converting them to
linear-stretched absolute output scale.

This is a pretty major refactor of the way pretty much all tone mapping
is done in libplacebo, and the new design should hopefully be more
robust, more understandable, and generally more useful.

This is installed unconditionally on recent enough versions of FFmpeg.
It just results in runtime link failure if you try using it, which is an
FFmpeg bug (and not one that can be worked around by the previous
solution either).

Requires a refactor of the API to split up `pl_upload_frame` into
separate `pl_map_frame` and `pl_unmap_frame` callbacks. Overall, I'm
happy with the way this API turned out, although I'm starting to rethink
whether or not we really want `tex` to be provided by the user or just
create/destroy textures unconditionally every time we map a frame.

VAAPI support is provided by using ffmpeg's own facilities for mapping
vaapi frames to dmabufs. Could probably be extended to any other format
FFmpeg can successfully map.

Vulkan support is provided using the new timeline semaphore API, which
requires a fairly recent version of FFmpeg.

Apparently FFmpeg does not forward all stream-level properties (in
particular, side data) to individual AVFrames. This is especially
noticeable for the displaymatrix side data, which in practice I can
*only* find as stream-level metadata.

Annoying but necessary.

This reduces the sizeof(struct pl_film_grain_data), and anything that
depends on it (e.g. sizeof(struct pl_frame) or sizeof(struct
pass_state)) by around 10 kB, bringing pl_frame down to only 512 bytes.

The massive size of these structs was never intended. This change only
affects the AVFrame utilities, which can simply reference the existing
side data because the pl_frames generated from them were already
specified to depend on the lifetime of the corresponding AVFrame.

These set the exact HDR metadata from the content tagging.

In retrospect, I think I really regret not making these part of
`pl_frame` to begin with. But oh well, that bullet is swallowed for now.
I'm not breaking that API for a while.

This is common practice for interoperability with C++ code.

Niklas Haas authored 3 years ago and Niklas Haas committed 3 years ago

Apart from significantly cutting down amount the typing the user needs
to perform, I decided the past justification for keeping these as `const
struct` rather than typedef'd names no longer makes sense due to the way
libplacebo objects tend to have private parts hidden from the user.

Bite the bullet and refactor all the things. Should be backwards
compatible, but I decided to bump the major version to signal the
significance of this API change.

Similar to Dav1dPicAllocator, this allocates from DMA-visible GPU
memory. Especially useful on platforms that don't support arbitrary host
pointer imports.

Make `plplay` use this, for testing purposes.
Update the CI image to pull in avcodec headers.

Turns out I actually made this very mistake. Oops.

These helpers are useful when preparing render targets based on
AVFrames, such as when feeding the output of libplacebo into a further
filter chain (or encoder) with an already existing AVFrame.

This can speed up pl_upload_avframe() significantly, approaching zero
overhead (direct rendering) under favorable circumstances.

As clarified by @cyanreg, FFmpeg filters are supposed to ignore
obviously "wrong" combinations, in particular RGB + limited range. We
also make an effort to clean up broken sig_peak tagging, since this
exists in practice.

I decided not to sanitize the other paramaters until I encounter a
broken file that demand it, because it's hard to draw the line
otherwise, although we could certainly come up with a few more ideas for
things that are potential candidates. (ICtCp with non-HDR curve? XYZ?)

Also clean up some whitespace because I can't be bothered making a
separate commit for it.

This is motivated by these two structs, essentially, having
conglomerated into unity. Drop the deprecated fields from each and just
go ahead with the big search & replace.

This also allows us to unify a couple of helper functions, quite
pleasingly. The only really annoying thing is that `target.av1_grain`
makes zero sense, but whatever.

We also provide a new helper function to bridge the "functionality gap"
in the `pl_render_target_partial` / `pl_tex_clear` checks.

An attempt was made to preserve backwards compatibility, for what it's
worth.

An array of pointers to pointers? Complete rubbish and overkill. An
array of pointers is fine. So what if users have to group their plane
textures together, big deal. It's way more convenient this way, than
having to explicitly separate the plane texture by plane and somehow
provide them separately as an array.

Not even gonna bump the API version for this because 1) nobody uses this
API yet, and 2) it's single-header garbage anyway, so no risk of ABI
break.

These structs and enums are designed in not-quite-compatible ways, and
the mapping is not at all intuitive or obvious. This should hopefully
make life a billion times easier for people wanting to use libplacebo
together with FFmpeg/libav* (i.e., basically the default use case).

Also includes an ultra-high-level wrapper that just takes an AVFrame *
and straight up uploads it to the GPU, in `pl_image` format, ready to be
consumed by `pl_renderer`.

This entire set of helpers is done as a single-header library to avoid
depending on libavutil, which would be a *really* terrible idea since
libavfilter also wants to depend on libplacebo. This single-header
nonsense is the easiest way to avoid a cyclic dependency. I promise I
won't make it a habit. Besides, it's annoying having to do things this
way, because it means I can't re-use any of the fancy logging functions
from pl_ctx. Oh well. I hope the user knows what they're doing.