Video freeze on set Time. Windows 10. NET6.0; libVLCSharp 3.6.6; libvlc 3.0.17.4
Summary
I'm very new to trying to use this library. Per the title, I've compiled V3.6.6 with library 3.0.17.4 and am referencing from a VB.NET (Net6.0-windows) application.
The application uses various mechanisms to scrub through the loaded video (trackbar, buttons, keyboard, fast forward) both when playing and when paused. Two issues arise during scrolling operations:
- Occasional/Intermittent "freezing" of video output. Image stays on a single frame but video still "plays" (TimeChanged events fire, audio can be heard, etc). This is inconvenience as the video output will pick up if I use the .Play method again.
- Occasional/Intermittent total freezing of the app (process needs to be killed externally). The entire app locks up.
CPU is i7-4790 CPU @ 3.60GHz 16Gb RAM; Windows 10 Home 64Bit and GPU is NVIDIA GeForce 1050Ti with latest drivers.
Minimal project and steps to reproduce
I have put a minimal version of the app on GitLab https://github.com/rcbroughton/ScrollIssue
What is the current bug behavior?
I am working on two types of video. Both are H.265, either 1920x1080@60 or 3840x2160@30. The problem seems worse (but not exclusive to) the 4k video.
Video loads fine. Video plays fine. Normal FFW (.SetRate) works fine up to 4x speed - thereafter I use my own process to simulate higher rates. I have a scroll-bar (and buttons and capturing arrow-keys - the latter of which exhibit similar behaviour) to scrub through the video to different time codes.
The scrollbar is active when I have the video playing and when I have it paused. When it's playing, it scrubs to a location and the video resumes from there. When it's paused it repositions the pause point to the new location, where I can single step (or multiple step) frames back and forward. To ensure I always pause on a frame, I create a frame reference table when the video loads that holds the time in ms of each frame and, to avoid seeking to arbitrary locations, I use this to "quantize" the step to individual frames.
It works very well - most of the time.
However, there were times when the video "freezes" - I can still scroll back and forward - the time code change, sometimes I hear audio - but the video stays on a single frame and won't move. This is usually cleared by cycling pause/play and the video then resumes from where it had actually been scrolled to (i.e. not necessarily the frame on which it had been frozen) Other times it is just completely locked and I have to kill the program.
I've tried various options, currently I have :file-caching=1000 :acodec-hw=any :avcodec-fast I've tried with :avcodec-hw=none but the performance sucks when I try to do a fast forward at more than 2x speed (I've implemented a funky workaround for "faster" ffwd up to 16x by using a timer to reposition a paused video which works OK)
The larger caching seems to alleviate but not eradicate the problems.
I tried working around the problem by putting in some extra code to prevent the app from trying to reposition too quickly - I've done this by setting a flag when the "Buffering" event fires at less than 100% and only clearing the flag when this reports 100%. Any scrolling is discarded if the flag is set. This seems to further alleviate the problem but it's not totally gone away.
With the minimal project, I can load the video - play/scroll/etc. It might work fine for a while but occasionally (e.g. for the log shown below) the first or second scroll operation locks the program (looks like some sort of threadlock?)
When I break the code when it's frozen, it's sitting on set => Native.LibVLCMediaPlayerSetTime(NativeReference, value); in Media.cs There is little in the logs to show any error.
I had noticed that both problems did seem more prevalent following these messages in the Log (again, not always)
Could not convert timestamp 0 for FFmpeg Timestamp conversion failed for 13066667: no reference clock
So - I tried another workaround to make my main process sleep when it saw this error in the Log (just a 100ms sleep) to give the player time to do whatever it was doing before allowing any other control input. Again, this seems to make things a bit more stable - but I still get the occasional lock up.
What is the expected correct behavior?
I'm looking for a workaround to avoid the lockup scenario. If there is some synchronisation / status I can get from the MediaPlayer to avoid whatever is causing the lock up (again, I'm assuming it's performance related, with the background threads still busy on a previous command when a new command is issues)
Does it work on other platforms? Does it work with the official VLC apps?
I've tried a few videos in the official VLC PLayer. It generally looks more resilient and, although I can recreate Problem 1 (video is frozen but player remains active) - but so far only with the "default.vlt" scrubber. As with my custom code, the video starts again when the "play" button is pressed.
Relevant LibVLC logs and/or screenshots
OK - so here's a long log from the program. (Note this is libVLC debug output as well as my own debug to show where my program is injecting commands - those come from Thread(1) ) & I add timestamp and source thread for each logged item. This is the latest example of when it froze. No real warning beforehand in the log, as far as I can see
Environment
- OS: Windows 10 Home 64Bit
- Version 21H2 Build 19044.1889
- Device: Windows PC
- LibVLC version and architecture 3.0.17.4 64Bit
- LibVLCSharp version 3.6.6
Possible fixes
The freeze is stemming from lines of code where I am repositioning the video: MediaPlayer.Time = newTime
I have done what I can to ensure that any previous activity has completed before issuing a subsequent repositioning command - but I'm assuming that there are circumstances where there is still a conflict.