Regression: MKV with Vorbis/Opus plays silence for first seconds

Last good version 2.2.8 (Vorbis, Opus in MKV plays fine)
Bad versions 3.0.0 and all subsequent versions, tested with 4.0.0 nightly 20210328-0422
Example debug log (VLC 3.0.0), when trying to play the provided vorbis sample:

main debug: end of audio preroll
mmdevice debug: state changed: 0
wasapi debug: reset
main warning: playback way too early (-257000): playing silence
main debug: inserting 12336 zeroes

Sample clips:

vlc-20927-test-vorbis.mkv (broken)
vlc-20927-test-aac.mkv (plays fine)

Can provide more detailed debug logs and test builds if necessary.

Edited Mar 28, 2021 by rctgamer3

To upload designs, you'll need to enable LFS and have an admin enable hashed storage. More information