Playback rate is oscillating randomly while playing audio files

This doesn't happen for video files - if a video stream is present, the audio stream keeps it's playback rate at a steady level.

However when playing back audio files, the pitch (playback rate) randomly can go up or down so much it sounds like transposing the whole track almost 1 semitone up or down.

The pitch changes are smooth, like someone is gently increasing/decreasing voltage on a tape machine's motor.

Usually it slowly goes up above 1.0x , then falls below normal speed, to finally get back to normal.

I always had this issue on different Android devices since forever.