Stutters following always the same pattern

Using VLC Flatpak from Flathub.

I always have the same type of stutter. Whenever playing a video/audio, the first seconds there is

1. sound for 0,5s
2. silence for 0,5s
3. sound for 0,2s
4. silence for 1s
5. sound continuing normally until the end

This only happens for the sound, the video plays normally. Now, playing an mp4 Video of Pinocchio, the same pattern also happened in the mid of the video, following the same pattern. The sections work normally, I have checked using rewinding, there are no issues.

This issue occurs since about half a year. It is not occurring using MPV, but I prefer VLC.

org.videolan.VLC	3.0.18	stable	flathub

--- Software ---
OS: Fedora Linux 38.20230808.0 (Kinoite)
KDE Plasma: 5.27.6
KDE Frameworks: 5.108.0
Qt: 5.15.10
Kernel: 6.4.7-200.fc38.x86_64
Compositor: wayland

--- Hardware ---
CPU: AMD Ryzen 5 PRO 3500U w/ Radeon Vega Mobile Gfx
RAM: 13.5 GB
GPU: AMD Radeon Vega 8 Graphics
Video memory: 2048MB
Audio: Alsa, Pipewire, Jack running

I havent changed any deep settings like file cache value, but will check to change to 20.000ms instead of 5.000ms