patch for ffmpeg affects d3d11va/dxva2 hwaccels for AVC

FYI https://trac.ffmpeg.org/ticket/9771

Sample is here (I remuxed it to .mp4):

coow1212l and here is another sample

https://github.com/wang-bin/QtAV/files/1132086/example.zip (remux to mp4)

ffplay.exe -vcodec h264_cuvid hbcwasdecaw.mp4 is not affected, so is -hwaccel nvdec

changed the description

added Component::Build system: Contribs Platform::Windows desktop labels

So it's a bug in ffmpeg, why opening an issue here ?

I though you will be able to find better solution, like recent dxva patch, since bigger MAX_SLICES is more calculations in Lavfilters. Also close to this issues were present before.

The problem here if you will check the second file (remux it first to mp4) can be only happening on part of the image and only for a couple of seconds.

Who knows what it does affect.

assigned to @robUx4

ping @robUx4

I can confirm we have this bug with the sample from https://github.com/wang-bin/QtAV/issues/923

Setting the value of MAX_SLICES to 256 fixes the issue (there's still a glitch that is also present in software decoding, not sure it's related as we don't fail anymore due to MAX_SLICES).

there's still a glitch that is also present in software decoding

SW decoder does not fail in ffmpeg even with only 32 slices, it is bitperfect with 256. Also consider that 64 is enough as QtAV shows. Nev argued that it should be 256, but I dunno! In fact it very hard to generate such samples in x264 and some 32 samples were done by SIP phone PC app as a joke, I am sure. I think the first sample is not actually using any of those slices, while the second sample may use more than 32 but that somehow does not warn.

I think there should be a way to make d3d11va to decode it without changes to h264 native decoder.

Also consider that #define can be changed per file!

I checked the return of this call in dxva2_h264_decode_slice()

    if (ctx_pic->slice_count >= MAX_SLICES)
        return -1;

It had some slice_count higher than 128 but none higher than 256. So 256 should be enough, for this sample. I don't know the logic behind though. Maybe even higher values are possible. MPEG2 and VC1 use 1024.

And yes, the value may be changed only in dxva2_h264.c.

Any high values are possible. That is the whole point! Sigh!! Here is a link to doom9: https://forum.doom9.org/showpost.php?p=1320333&postcount=226

It had some slice_count higher than 128

If it first sample it is fake. That data is lying! Checking second sample is more interesting...

Comment from nev as to how 256 slices will affect memory... https://forum.doom9.org/showthread.php?p=1950303&highlight=max_slices#post1950303

changed the description

Interresting. I wonder why the fix was not pushed to ffmpeg. It's still not fixed in master.

Nev did not push it or even submitted to mailing list. Also hwaccel cuda is not affected and so is older -c:v h264_cuvid decoders. That alone is confusing, since at least first still uses h264 code to preparse. I did not check linux or macos... dxva code and d3d11va defines should be both modified too. And of course there is a problem that SW decoder does not care there is more than 32 slices... Why? New samples created by x264 -slices option are needed.

Oh and also, how many slices does Nvidia Turing support?? It cannot be 8100, can it?

changed the description

Also this patch may cause strange verbose warnings in dxva decoder.

I also verified in JM 19.0 that I compiled with Visual Studio 2022 by changing #define MAX_NUM_SLICES 500

that ffmpeg decoder is bitperfect on both samples (md5 of first yuv FILE is E47FDCC046210054BA90953B95A62ADF and second 31486A6EBD8A98EFE8FC54F13667A504). Checking second sample with ffplay.exe -video_size 1280x720 -pixel_format yuv420p -f rawvideo oldexa.yuv no artifacts.

In fact WHAT I DID I modified define MAX_NUM_SLICES 32 so that it will be same as ffmpeg.

The second sample then prints "Maximum number of supported slices exceeded." on 96th frame!! (Counting from 0.) That is why HW decoding does fail not immediately. I then modifed to 64 and that is enough to decode bitperfectly. 48 is enough. 40 is not enough it fails in the middle... 44 is enough, 42 is enough, 41 is enough! Lets just say this is hillarious. You need 41 SLICES for second sample. FFmpeg does not permit 41 slice though, only a power of 2.

Okay, so dxva2 does not support the first sample, too many slices.

Lav\ffmpeg.exe -hwaccel dxva2 -i error_4942.264 -f md5 -

[h264 @ 000001fe7ab6d9c0] Buffer for type 5 was too small
[h264 @ 000001fe7ab6d9c0] Failed to add bitstream or slice control buffer
[h264 @ 000001fe7ab6d9c0] hardware accelerator failed to decode picture
Error while decoding stream #0:0: Operation not permitted
MD5=b79d3517ca63de456208b5eb00559297

While both nvdec:

Lav\ffmpeg.exe -hwaccel cuda -i error_4942.264 -f md5 -

Lav\ffmpeg.exe -hwaccel d3d11va -i error_4942.264 -f md5 -

print

MD5=ab07a0a0a82a2e9a6486a8b6994eb505

There is also a problem with this sample: https://avc.hhi.fraunhofer.de/mantis/file_download.php?file_id=166&type=bug

That one is completly different in all cases :)

So any updates? BTW, your patches are still not applied https://patchwork.ffmpeg.org/project/ffmpeg/list/?series=527 that is not nice, those work perfectly in MPC-BE. You can also resend those. See https://github.com/Aleksoid1978/MPC-BE/commit/685e76b023b90a9a06c9d647b7a3d6c650798251

BTW if you wonder why it is loading Video Decode of 2080 Ti to 100% it is because the decoder is not real time, it is much faster than that.

Any updates?? There is now another dup https://trac.ffmpeg.org/ticket/10374

Well code doesn't write itself, when someone has a MR ready referencing the bug. You'll be automatically notified :)

The code here needs to be written for ffmpeg, not VLC. And it is already there in Lavfilters. https://git.1f0.de/gitweb/?p=ffmpeg.git;a=blobdiff;f=libavcodec/h264dec.h;h=97951281d01ed7330c4e5c112e0496fe2ac64a3d;hp=851c6e400e3e8a848699e70853d4da185f928d94;hb=550cf548b546d386a6c634351ad0c250a3e47f3b;hpb=c3c783db001e31269c4addddaff318600a05db7f;js=1

Well then pinging VideoLAN for a ffmpeg bug seems a bit weird

Anyway the point stands, when the bug will be closed because ffmpeg pushed a patch and we update our contribs you'll be notified. But keep in mind that despite the overwhelming charisma of our great president VideoLAN cannot force FFMPEG to do anything :).

The cool news is that apparently you can just modify dxva2_h264 file (and d3d11va file)! And that will just work. You do not need to change the main file like done above by Nev.

Well then pinging VideoLAN for a ffmpeg bug seems a bit weird

You all write code for ffmpeg. So not really.

After IRC ffmpeg discussion looks like you really want to leave this bug forever in the codebase.

Again, one line in dxva2_h264 file.

Again this is VLC's bugtracker not FFMpeg's. If it's one line feel free to propose a patch on ffmpeg-devel instead of posting passive aggressive comments here.

Little bit of background here but I think OP was banned from FFmpeg-devel due to trolling.

I you call political hatred from libtards against Russia that, then sure.

Calling people "libtards" is violation of VideoLAN's code of conduct. Consider this an official warning.

Considering the escalation despite the warning this issue is now locked. And comments were removed. FFMpeg IRC channel doesn't depends on VideoLAN's code of conduct and there's nothing related to a bug in VLC.