Regression: MKV with Vorbis/Opus plays silence for first seconds

• Last good version 2.2.8 (Vorbis, Opus in MKV plays fine)
• Bad versions 3.0.0 and all subsequent versions, tested with 4.0.0 nightly 20210328-0422
• Example debug log (VLC 3.0.0), when trying to play the provided vorbis sample:

main debug: end of audio preroll
mmdevice debug: state changed: 0
wasapi debug: reset
main warning: playback way too early (-257000): playing silence
main debug: inserting 12336 zeroes

Sample clips:

vlc-20927-test-vorbis.mkv (broken)
vlc-20927-test-aac.mkv (plays fine)

Can provide more detailed debug logs and test builds if necessary.