Deadlock between Display and Window module when both are synced on UI thread
This is a tracking issue for the discussion on the mailing list here:
https://mailman.videolan.org/pipermail/vlc-devel/2019-December/130001.html
I stumble on this issue when refactoring the iOS vout, the chaining of events are the following:
-
the vout_thread starts.
-
it enables the UIView window. => the UIView register to its superview in the main thread.
-
it locks the display_lock and starts opening the GLES2 vout display. => the UIView gets notified of a resize from the main thread, calls vout_window_ReportSize, get stuck in the display_lock in the main_thread.
-
the display module creates an OpenGL provider, which needs to register the openGL-backed UIView surface from the main thread, which is stuck. It cannot be done asynchronously because we need to tell the core whether it was created correctly.
It can actually happen with any module.
The issue is that we cannot remove the constraint on "the Display should not wait on UI thread in the Open" like for any other method, because we actually need to check that we can open the module before returning in the Open, and we need to block the UI thread to allocate and sync UI resources.
This issue is preventing the refactor of iOS vout from being merged.