Memory usage grows when switching to new Media in MediaPlayer

changed the description

Hi,

Thanks for the issue. We'll try to have a look soon.

Can you check using VS if the memory increase is in native or managed memory? Do you reproduce this leak on Android or iOS?

I have not tested on Android or iOS, only macOS. I have been unable to find how to do memory diagnostics (using visual studio for mac).

https://github.com/siggi90/video_test/blob/49a6c34020a9e268f71cf739aae6b3237aae5f94/video_test/MainPage.xaml.cs#L81 This line likely increments the refcount which means your media is never released. Don't access the media with mediaplayer.Media if you already have a reference to it in a variable.

I get what you're saying, i only added that line of code from suggestions i was getting at stackoverflow, or my understanding of those suggestions. Memory usage increases regardless of that line of code. For example my application which uses LibVLCSharp has the following code: if(web.MediaPlayer == null) { reset = false; }

            using(Media media = new Media(this.lib_vlc, uri)) {
                if(!reset) {
                    web.MediaPlayer = new MediaPlayer(media) { EnableHardwareDecoding = true };
                    set_end = true;
                } else {
                    //web.MediaPlayer.Media.Dispose();
                    web.MediaPlayer.Media = media;
                }
            }

so it never calls the web.MediaPlayer.Media getter so it should not be increasing the reference count, nevertheless memory usage increases. And I've tried calling dispose as well and that seems to make no difference.

Maybe this gives some clue to what is going wrong: System.Diagnostics.Debug.WriteLine("perform dispose"); System.Diagnostics.Debug.WriteLine("dispose: "+this.media.NativeReference.ToString()); this.media.Dispose(); System.Diagnostics.Debug.WriteLine("dispose: "+this.media.NativeReference.ToString()); web.MediaPlayer.Media.Dispose(); System.Diagnostics.Debug.WriteLine("dispose: "+web.MediaPlayer.Media.NativeReference.ToString());

this prints: perform dispose dispose: 140602379657648 dispose: 0 dispose: 140602379657648

so the MediaPlayer.Media is not being disposed of. And storing the media in a variable and disposing of that variable doesn't dispose of the MediaPlayer's Media.

Be careful. MediaPlayer.Media is not a simple property getter, buts creates a new instance and increments the native counter every time it's called. Every time you call this getter, you should dispose the instance you are getting

I know that's what I've been doing so far, but if i store the Media i set as the MediaPlayer's Media in a variable and dispose of that instance, shouldn't the NativeReference for the MediaPlayer's Media also be disposed of? Is there some way to print the native counter?

The "MediaPlayer's media" is not saved in C# memory, it's retreived when requested from the native player (which still has a reference to it, until you set MediaPlayer.Media= null)

I've updated the video_test github project, to my current understanding of that proper way to handle switching MediaPlayer's Media. If you could take a look at that and tell me if it's not in accordance with how you think it should be.

I can verify that the media is never freed with the MediaFreed event handler:

            media.StateChanged += (s, e) => {
                System.Diagnostics.Debug.WriteLine(e.State.ToString());
            };
            media.MediaFreed += (s, e) => {
                System.Diagnostics.Debug.WriteLine("media_Freed");
            };

This code prints the following:

Playing Ended Stopped SET NEXT VIDEO: 0 NothingSpecial Thread started: #7 (closed) Opening Playing Ended Stopped SET NEXT VIDEO: 1 etc...

it never prints "media_freed"

MediaFreed is deprecated in libvlc 4, probably broken in libvlc 3. Never used it personally.

Alright, but it still doesn't diminish the fact that the memory usage grows, have you looked at my updated code and seen if you agree that it is a proper way of switching the media?

From what I've seen you're trying to dispose the media twice (with using and with an explicit Dispose call).

I have removed the redundant Dispose call, for any future references.

Can confirm, this memory leak also occurs on Ubuntu 20.04.

OS: Ubuntu 20.04.1 LTS x86_64
Device: Intel NUC i3 (Intel i3-3217U)
LibVLC version: libvlc 3.0.9.2-1 and 3.0.12.1
VA-API version 1.7.0

I can also reproduce this issue with VLC Player (3.0.9.2-1) on Ubuntu 20.04.1.

Steps to reproduce:

Start VLC
Add 2-3 videos to the media library (ideally with 5-10 seconds length)
Configure Loop/Repeat
Play and wait some hours

RAM consumption increases from ~200MB to 1GB
OS: Ubuntu 20.04.1 LTS x86_64
Device: Intel NUC i3 (Intel i3-3217U)
Used Media Driver: i965
VA-API version 1.7.0

Please post to the vlc bug track for this, this is unrelated to libvlcsharp

This should help https://github.com/videolan/libvlcsharp/pull/208

But until we know exactly what is leaking (managed and/or native memory), this likely won't be fixed.

Information from Activity Monitor:

Real memory Size: 313.7 MB Virtual Memory SIze: 8.05 GB Shared Memory Size: 2.00 GB Private Memory Size 153.7 MB

I've just downloaded Xamarin Profiler will hopefully have this information soon.

Edit: Xamarin Profiler requires Visual Studio Enterprise which costs $250 dollars a month. Which i can in no way afford, is there any chance anyone here has Visual Studio Enterprise and has the time to download Xamarin Profiler on mac and test this video_test project out?

VS Community on Windows has some memory analysis tools AFAIK.

What about XCode?

Taken from xcode leaks instrument. I'm not really an expert on this type of thing so if you can help me interpret this screenshot. Leaks are definitely occurring, i don't know how to tell if it's managed or native memory, but there is a stack trace there for the leak.

I have never used the XCode memory profiler so also a noob here. If I believe what I see from your screenshot, either mono is leaking, or any of the .NET code running.

There is a line with libvlc.dylib in there. Feel free to share the stacktrace if any :)

Here is the stacktrace for libvlc.dylib. If it's .NET code that's leaking wouldn't it most likely be the LibVLCSharp.Forms code since the leak occurs when switching media, or would that be apparent from the stack trace? If not, should i report this error to Xamarin? All of the leaks have the same stack trace except for the libvlc.dylib one that I've sent with this comment.

or would that be apparent from the stack trace? If not, should i report this error to Xamarin?

Honestly I don't know, and I don't have free time to spend on this right now.

Something I'm noticing now, (don't know if it's after the update to 3.5.0 or if it was always like this but it seems that memory also increases when playing just a single media file, slowly over time. Maybe this is normal if it would decrease after playback has stopped or when switching to next media. I was wondering you mentioned this: "This should help https://github.com/videolan/libvlcsharp/pull/208" I was wondering if this will be released soon or if there is a pre-release channel for version 3?

I doubt this fixes the problem honestly. Preview builds are available at https://feedz.io/org/videolan/repository/preview/packages/LibVLCSharp, see the docs https://code.videolan.org/videolan/LibVLCSharp/-/blob/3.x/docs/libvlc_preview.md

I've tried the latest preview build and the issue is unfixed. Just a thought, would it be possible to add something like a ClearMemory function to a LibVLC instance? That way i could i could clear memory each time before switching to new media since leaks seem to be occurring when media is playing as well. And since i have an application with tabs and multiple VideoViews it would probably be necessary for me to have multiple LibVLC instances for each VideoView, this function would be a godsend. I've tried creating a new instance for VideoView, MediaPlayer and LibVLC and wrap them in using statements to simulate this process but that doesn't work, probably because the memory is somehow left hanging somewhere despite Managed code being cleared up.

would it be possible to add something like a ClearMemory function to a LibVLC instance?

What would that function clear exactly?

it would probably be necessary for me to have multiple LibVLC instances

This is against documented guidelines. Never use multiple libvlc instance, create multiple mediaplayers from a single libvlc instance.

I'm not using multiple libvlc instances currently but since the idea for the function would be to clear libvlc instance completely from memory and create a new instance (clearing all unmanaged code and restarting a new instance) so if that is at all possible then i would have to use multiple libvlc instances because i have multiple videoviews plaing videos at the same time and if i would clear memory of the libvlc instance and it would be powering multiple videoviews then it would obviously not work. The idea comes from the fact that restarting the application obviously clears the memory so the leaked memory can be garbage collected if enough restarting is done. So even though it might be a little bit of a crummy solution if it would work it would be good enough for me for the time being. I'm not really clear on exactly how libvlc works regarding unmanaged code/memory so this is just a rough idea. As i said in the comment above I've tried creating a new LibVLC instance per each media being played and that doesn't clear memory or result in proper garbage collection. But i was hoping if it would be reset from unmanaged code somehow it would result in proper garabage collection of the entire unmanaged code.

That workaround is unlikely to work but feel free to try it...

Would be better to pinpoint and fix the real leak though.

Yes that would be better, i was just thinking of it as a solution in the meantime since i would love the have some temporary solution. But obviously this is far from optimal. I wish i had more insight into the actual problem so i could do something about it.

Same problem. Ubuntu 20.04, Raspberry PI.

Tried latest 3.5.1 preview, seems leak not fixed. On Windows no problemo.

Tried running version 3.6.0 in the video_test in Xcode Leaks, and i thought this information might be relevant. It seems that memory is pretty stable except for IOSurface which continually grows.

Could you try to repro in a tiny swift app using the latest vlckit cocoa pod please? IOSurface potential leaks is not related to libvlcsharp, and likely not Mono's fault. Either a libvlc or videotoolbox issue. With a small vlckit swift repro, we could reach out to the macOS team.

Thanks

It seems the latest version of vlckit on the cocoapods website is 3.3 but the nuget I'm using in my C# mac project is 3.1.3.1, and it seems to be the latest version. Are these versions consistent with each other? Is it ok if the test project would be in Objective-C?

Are these versions consistent with each other?

Not really, they are built a bit differently and the nuget kinda has its own versioning scheme. And continuous delivery would be nice. Just use the latest stable pod available.

Is it ok if the test project would be in Objective-C?

Yes, obj-c or swift both fine.

I've tried emulating a sample for vlckit but I am unable to get to work properly. Is there possibly someone else more experienced with vlckit and building native mac apps that could volunteer to do a small test. I don't think i'm doing anything special so a simple app that plays videos in sequence would be enough to demonstrate the leak. Also vlckit seems very different from LibVLCSharp so i don't really understand how i would make a project that would reflect the same methods i'm using on the C# side. I thought this screenshot might provide further information that could be helpful pinpoint the issue with IOSurface:

vlckit seems very different from LibVLCSharp so i don't really understand how i would make a project that would reflect the same methods i'm using on the C# side

libvlcsharp uses vlckit under the hood. And both use the same underlying LibVLC C code. Their public APIs may seem different but its very much similar.

I've made an objective-c test project, but it is obviously not exactly the same as the other project, and you have to change media manually but memory usage grows in the same manner, although in the Xcode leaks tools the memory leaks are categorised differently (in the objc project the leaks are not categorised as IOSurface but i assume that's because the applications are built differently, i think it the same underlying leak that causes both memory increases. https://github.com/siggi90/video_test_objc/tree/main/BasicPlayerWithPlaylist

Thanks, we're having a look.

I've managed to reproduce the leak, however it is just a very tiny one and most of it seems to be somewhere in the graphics driver code, so probably nothing we can do something about. Overall the memory usage of the example you shared with three different video files switching between them constantly stays around 50MB for me.

I guess i misread the information fromt the Leaks program, i hadn't checked the memory usage in activity monitor, it's the same for me, stays around 50MB. I'm sorry i wasted your time, this project apparently doesn't demonstrate the leak i've been trying to find.

No problem. Yes the Instruments leaks tool shows all allocations, even those that are not leaks (which can be useful to track large allocations). You can switch it to show just leaks in the breadcrumb navigation when clicking on "Allocation summary".

Had the same issue in winforms. Resolved after following developer's comment and dereferenced MediaPlayer.Media

The "MediaPlayer's media" is not saved in C# memory, it's retreived when requested from the native player (which still has a reference to it, until you set MediaPlayer.Media= null)

Didn't even call MediaPlayer.Media.Dispose() before MediaPlayer.Media = null because like the guy said, you're probably only disposing the object on C# side, not releasing the native reference.

Then create new media object for the next file.

MediaPlayer.Media = new Media(vlcLib, url, FromType.FromLocation);
MediaPlayer.Play();

Interestingly, you will still get memory leaks if you call MediaPlayer.Stop() before dereferencing

MediaPlayer.Stop();
MediaPlayer.Media = null

Doesn't seem to work on macOS, (memory still grows quickly). I'm never calling dispose and never calling the getter of Media, and am setting the media as null before setting next media.

            if(!reset) { //on first use
                web.MediaPlayer = new MediaPlayer(new Media(this.lib_vlc, uri)) {
                    //FileCaching = 0,
                    EnableHardwareDecoding = true
                };
                set_end = true;
            } else { //after media player is initialized
                web.MediaPlayer.Media = null;
                web.MediaPlayer.Media = new Media(this.lib_vlc, uri);
            }

            web.MediaPlayer.Play();
            web.MediaPlayer.Position = 0.999f; //for testing purposes
            web.MediaPlayer.Volume = 0; //for testing purposes    
            if(set_end) {
                web.MediaPlayer.EndReached += (s, e) => ThreadPool.QueueUserWorkItem(_ => {
                    this.video_index++;
                    this.init_video(); //load next video
                });
            }

Memory usage grows when switching to new Media in MediaPlayer

Summary

Minimal project and steps to reproduce

What is the current bug behavior?

What is the expected correct behavior?

Does it work on other plaforms? Does it work with the official VLC apps?

Relevant logs and/or screenshots

Environment

Possible fixes

Child items ...

Activity