major memleaks in h264 hardware-accelerated decoding with VDA

After every seek to another video position, VLC allocates more memory (for full HD video sometimes +300 MB per seek). So, after you seeked around for a while, you end up with a vlc which needs 2 GB RAM or more.

We already discussed about this issue in #8440 (closed) and came to the conclusion that this needs to be fixed within libav. So I created a bug in libav bug tracker: https://bugzilla.libav.org/show_bug.cgi?id=532

But as we are advertising VLCs new hardware-accelerated decoding capabilities and VDA is enabled by default, IMO this should be fixed before 2.1 release, hence I open this ticket here. Any ideas welcome, for fixing this issue.

added Component::Decoders Severity::major Type::bug Version::master git labels

changed milestone to %2.1.0 release

changed the description

VDA is not enabled, or at least, it should not be. As things currently stand, enabling VDA by default will break decoding of unsupported codec profiles on VDA-capable systems, and turn off threaded decoding on VDA-incapable systems...

If there is a bug in the contrib version of Libav/FFmpeg, please reassign to contrib and provide a reference to the correct version or patch. As long as there is no upstream fix, assigning this to a VLC milestone makes no sense.

changed milestone to %Bugs paradize

I especially opened this ticket because I noticed that VDA got somehow enabled by default on my machine.

To be more precise, after deleting prefs, VDA is off by default, but after I changed ffmpeg settings (in my case threads to 1 and back to 0), VDA gets enabled somehow. This might be some bug in the module.

ffmpeg should be already fixed.

libav patch: http://lists.libav.org/pipermail/libav-devel/2013-June/047401.html

vlc patch: http://pastie.org/8133796

Replying to [comment:7 dfuhrmann]:

ffmpeg should be already fixed.

libav patch: http://lists.libav.org/pipermail/libav-devel/2013-June/047401.html

That seems impossible since this patch was committed six weeks ago but you tested current FFmpeg two weeks ago and the bug was still reproducible: #8440 (closed) comment 16

Nothing is impossible. :-)

2 weeks ago, I was not aware of the vlc patch with is needed for the other patches to have effect.

Thank you for clarifying!

commit e7c2758e Author: Rémi Denis-Courmont remi@remlab.net Date: Thu Jul 18 19:26:05 2013 +0300

vda: drop priority to zero (refs [#8927](https://code.videolan.org/videolan/vlc/-/issues/8927))

Original author: remi@remlab.net

assigned to @dfuhrmann and unassigned @MigrationBot

changed milestone to %2.1.0 release

commit c7a2a17a Author: David Fuhrmann david.fuhrmann@googlemail.com Date: Wed Sep 11 20:47:27 2013 +0200

vda: user ref buffer mode

This fixes memory leaks when seeking or dropping frames for other reasons.

close [#8927](https://code.videolan.org/videolan/vlc/-/issues/8927)

Original author: david.fuhrmann@googlemail.com

added Status::fixed label

closed

removed Status::fixed label

changed milestone to %2.2.x maintenance branch

reopened

closed

Fixed by enforcing the ffmpeg api in b5452bad.

If someone wants vda with libav, then this really needs to be fixed within libav before we can do anything on the vlc side.

added Status::fixed label

Moved to 2.1.x :)

changed milestone to %2.1.x maintenance bugs

changed milestone to %2.1.3