colorspace: add support for Dolby Vision

This covers Profiles 5 and 8.X. After playing around with the implementation quite a lot, I eventually settled on the current state, which requires no LUTs or state and is implemented entirely in pure GLSL. This is slightly slower than a pure 1D LUT approach, but has the benefit of not needing to add an extra pl_shader_obj to the API at all, and thus being transparently backwards compatible.

The decoding metadata is assumed to be passed in via a new struct pl_dovi_metadata, which contains both the colorspace matrices (especially important for profiles 8.X) and reshaping coefficients.

TODO:

• test MMR code
• clean up debug crap
• optimize/benchmark MMR
• add GLSL version checks for floatBitsToUint (or work around?)

Closes #113 (closed)

added 1 commit

• d03ff0cd - TMP: alternate implementation (maybe faster?)

Compare with previous version

added 4 commits

• d03ff0cd...f25699c3 - 3 commits from branch master
• 02373ca6 - (WIP) colorspace: add support for Dolby Vision

Compare with previous version

changed title from colorspace: add Dolby Vision Profile 5 (IPT-PQ) to (WIP) colorspace: add support for Dolby Vision

changed the description

added 1 commit

• bc5f7584 - (WIP) colorspace: add support for Dolby Vision

Compare with previous version

added 1 commit

• ff6ea7b5 - (WIP) colorspace: add support for Dolby Vision

Compare with previous version

added 1 commit

• 2d188994 - (WIP) colorspace: add support for Dolby Vision

Compare with previous version

Seems to work on my side with polynomial reshaping. Patch here: quietvoid/libplacebo@957e702e
Will look into MMR later.

BTW. Considering IPTPQc2 is full range how are we going to present it? LG C9 has direct access to ~~ 40 bits panel that is RGB with W for every pixel... While if we will try to output to 10 bit surface in PQ we will lose some data. Like, here we decode to linear RGB (not R'G'B', since there is NO R'G'B', but L'M'S') of BT.2020 primaries and D65 whitepoint, okay, and then encode with PQ which is going to be less quality because 10 bit IPTPQc2 can preserve more that 10 bit YCbCr. WE ALSO lose a lot of stuff going to limited range (that includes Y from 941 to 1019 and Cb, Cr from 961 to 1019 but also some details inside YCbCr 64-940 and 64-960 since a lot of YCbCr is outside R'G'B'), so support for YCbCr full range is badly needed, but Nvidia does not like full range Y'CbCr (while it is normal on consoles).

Also, what is superwhite in Dolby vision? Even though it is full range, some I'P'T' may go outside BT.2020 RGB. So... I suppose linear (float to help out-of-gamut) RGB can be used with 16 bits per component... But then we still need to choose the HDMI format. I suppose if we will use 12 bit PQ it should be enough? Chrome already supports 12 bit PQ but Nvidia driver is crashing often in that codepath.

Also, is NLQ to be implemented?

Also, is scene change bug fixed now with @quietvoid patch (can you push it here, Gitlab allows it if @haasn put a flag to allow others to push code).

added 2 commits

• ecc36b2b - (WIP) colorspace: add support for Dolby Vision
• 16f14b38 - demos/plplay: integrate libdovi to parse frame RPU

Compare with previous version

marked the checklist item test MMR code as completed

marked the checklist item clean up debug crap as completed

Updated with fixed MMR code, included @quietvoid's patch for testing (also includes MMR coefs). Tested working on every P5 and P8.1 clip available to me. Still need to benchmark/cleanup the MMR code.

What I'm extremely unhappy about is the need to recompile the shader if the number of pivots changes, since the pivots are hard-coded. I normally wouldn't worry about this but because it can happen in the middle of playback it might very well cause stuttering during playback especially for something like d3d11. I could get around this by just always including the code for 9 pivots for the luma plane specifically but it feels like a huge hack.

I don't know what to do about it.

Tested working on every P5 and P8.1 clip available to me

Does it include a lot of samples from rarbg? I would check some Marvel (Falcon) and Star wars, also Pixar stuff (like last dragon, Luka) and IMAX 1:1.90 stuff from Disney+ and of course some originals from Apple TV+. There you just need to download trailers in DV, no need to even use rarbg, though you can. See my comment on how to do it: https://github.com/yt-dlp/yt-dlp/issues/771#issuecomment-918161614

I don't know what to do about it.

That is far from over though, right? You still need to add dynamic metadata of v4, trims, NLQ, etc? It is absolutely normal to precompile stuff. mpeg2 and mpeg4 use it in ffmpeg. A lot of others. Also, precompiled stuff can be further optimised by the compiler.

You should try the open encodes from Netflix in PQ and IPTPQc2 (Miridian) and some leaks from Netflix (Night on Earth, The Witcher, Alterned Carbon, and 1 more?).

Tested with profile 7 and 8.1, both resulted in a much brighter tonemapped output.
Sometimes clipping detail. Maybe some code path is only evaluated for PL_COLOR_SYSTEM_BT_2100_PQ when tonemapping?

It also gets very bright with peak detection enabled.

The profile 7 has both polynomial and MMR reshaping.
And the profile 8.1 shouldn't be reshaped at all, since it's HDR10 already.

shouldn't be reshaped at all, since it's HDR10 already.

But what if you put profile 7 in profile 8? BTW, IPTPQc2 in HLG profile 8 may be also possible :) Just need to change matrices for it, but Dolby does not want it. Also profile 7 may have FEL stuff and NLQ.

added 2 commits

• 291857b8 - (WIP) colorspace: add support for Dolby Vision
• 7dbbcb82 - demos/plplay: integrate libdovi to parse frame RPU

Compare with previous version