Freeze when subtitles delayed too much

When I delay a little bit, like 20 seconds, it freeze for 10 seconds then resume the video. When I delay a bit more, like two minutes, it just freeze eternally. MPV was able to delay the subtitle in real time without any freeze or lags, with same files. So it doesn't seem to be a computer performance issue

main warning: playback way too early (-10001780): playing silence
main debug: inserting 441078 zeroes
pulse debug: started
pulse debug: underflow