Memory Leak
I finally figured out how to loop RTSP using VLC for Unity on Windows. A few interesting things have come to light while testing RTSP in general and looping that I want to share my experience and observations.
-
For looping, after a stop is detected via _mediaPlayer.Stopped += mpStopped, you have to dispose of the media, dispose of the media player, dispose of libVLC and start the initialization process all over again. Simply doing _mediaPlayer.Play() does not work.
-
You need to check the fps within the UseRenderingPlugin.cs Unity script. You can tally this within the updated conditional statement and you need a separate timer outside of the conditional statement for the time. Your video should be updating based on the fps of the source. For testing, my videos are 24/25 or 30 fps. If you know the stream itself is reliable and the video in Unity isn’t updating (video looks frozen or choppy) or the image is distorted, there’s a problem “somewhere.”
- If you successfully connected to the stream and the fps is zero for GE 5 seconds at anytime, there’s an issue and you need to disconnect (dispose of everything)/reconnect.
- If the fps is GT 10 and LT 19 for GE 5 seconds, there’s an issue and you need to disconnect/reconnect.
- If the fps is LE 10, either the video is just starting or finishing, or some other type of blip occurred. Just ignore it, because either the video will play fine or enter one of the above states.
- If the fps is GE 20, make sure you reset the timers to zero for zero fps and fps between 10 and 19.
- The video itself can cause issues. There are some videos that play without issues and others that always have an issues. Make sure to test different types of content sourced from different places.
IMO, the reason you need to disconnect/reconnect is because libVLC doesn’t know how to correct what we see as issues by itself. Once it enters a certain stream processing state, it just continues on as if there’s nothing wrong. It detects no error. The frame rate is not going to get better. You’ll find that disconnecting/reconnecting resolves the issue if there is network connectivity to the stream and the stream isn’t dead. It may have to disconnect/reconnect more than once, but the conditional statements above will sort things out and eventually the stream will play.
- There’s a nasty memory leak when looping RTSP. My app starts with a few hundred MB and if I just let it run, it has gone as high as 7.5 GB before I stopped it. My system has 32GB and I have not tried to max that out to see what happens. The Unity profiler doesn’t see this memory consumption, but the task manager does. In the Unity editor, pressing stop does not free up the memory. Reloading the scene does not free the memory. Just for clarity, this happens when the app is an executable as well.
In theory, within UseRenderingPlugin.cs, this should be freeing up the memory:
_mediaPlayer.Media?.Dispose();
_mediaPlayer?.Dispose();
_mediaPlayer = null;
_libVLC?.Dispose();
_libVLC = null;
What’s responsible for this memory leak? Is it Unity, libVLCSharp or Windows 10? I can’t say. I’ve read a few cases about memory leaks in libVLCSharp (you’re responded to these Martin), but not specifically involving RTSP, so this may or may not be a known issue. Is Unity not doing proper garbage collection with respect to the plugin? Is it something happening in Windows? What you need to identify the root cause of the memory leak is a system debugger and I don’t know how to use one.
IMO, this is a Unity, libVLCSharp or Windows issue and not specifically VLC for Unity, so I'm not submitting a bug report at this time.
There’s an interesting blog post on avoiding memory leaks in C# posted here:
Find, Fix, and Avoid Memory Leaks in C# .NET: 8 Best Practices
An interesting side notes:
At present I'm using the official VLC player to stream videos via RTSP and play the video locally. I often just close the player using the close button at the top right when I want to stop streaming. When I check the task manager, the process is still running, even though the interface is long gone. For VLC to terminate properly, I have to press the stop button and then close VLC.