PulseAudio via pipewire-pulse: first timing points are bad, leading to lot of frame drops

On debian SID, when playing any video:

[000061c000060880] avcodec decoder error: more than 25 frames of late video -> dropping frame (computer too slow ?)
[000061c000060880] avcodec decoder error: more than 25 frames of late video -> dropping frame (computer too slow ?)
[000061c000060880] avcodec decoder error: more than 25 frames of late video -> dropping frame (computer too slow ?)
[000061c000060880] avcodec decoder error: more than 25 frames of late video -> dropping frame (computer too slow ?)

$ pulseaudio --version
pulseaudio 15.0

Here are the timing points sent by the pulse module:

system_ts:  19932542445 audio_ts: 234
system_ts:  19932542507 audio_ts: 245
system_ts:  19932552958 audio_ts: 147
system_ts:  19932555784 audio_ts: 163
system_ts:  19932566537 audio_ts: 312
system_ts:  19932587337 audio_ts: 21460
system_ts:  19932628469 audio_ts: 64185
system_ts:  19932709232 audio_ts: 149516
system_ts:  19932869944 audio_ts: 298810 <----- Points start beging "valid" here (ie. coeff = 1.0)
system_ts:  19933191328 audio_ts: 618854
system_ts:  19933832168 audio_ts: 1258833
system_ts:  19935112509 audio_ts: 2538736
system_ts:  19936613304 audio_ts: 4053394
system_ts:  19938113816 audio_ts: 5546819
system_ts:  19939615159 audio_ts: 7040161
system_ts:  19941116232 audio_ts: 8554844
system_ts:  19942617212 audio_ts: 10048146
system_ts:  19944117576 audio_ts: 11541429
system_ts:  19945618035 audio_ts: 13056181
system_ts:  19947119228 audio_ts: 14549502

Here is a quick hack fixing the video lateness:

diff --git a/src/clock/clock.c b/src/clock/clock.c
index 23727513f0..33cb4efa59 100644
--- a/src/clock/clock.c
+++ b/src/clock/clock.c
@@ -434,7 +434,7 @@ vlc_clock_main_t *vlc_clock_main_New(struct vlc_logger *parent_logger, struct vl
     main_clock->input_dejitter = DEFAULT_PTS_DELAY;
     main_clock->output_dejitter = AOUT_MAX_PTS_ADVANCE * 2;

-    AvgInit(&main_clock->coeff_avg, 10);
+    AvgInit(&main_clock->coeff_avg, 1);
     AvgResetAndFill(&main_clock->coeff_avg, main_clock->coeff);

     return main_clock;

With an AVG of 10 and pulse updating points every 0.5-1.5 seconds, you need to wait 10 seconds to get a valid coeff/offset once PulseAudio start to send valid points. Meanwhile, the vout already dropped most of its frames.

I don't know if it is a recent regression from Pulse/Sid. The issue was reported by @rom1v. I will upgrade my debian (still on stable) soon to see if I reproduce it.

The AVG hack is not a proper solution, here are the solutions I could think of:

• Filter bad timing points from the Core
• Filter bad timing points from the PulseModule
• Reset the AVG if the coeff difference is too big.

Regression from !2018 (merged)