Weird seeking in MP4 files without base media decode time
I have a fragmented MP4 file with a sidx atom and without tfdt atoms. I attached a sample video (due to the upload limit it is quite short, to see the issue I recommend seeking to around the halfway mark repeatedly). Playing this file in VLC looks normal for the most part, but seeking behaves weirdly. Specifically, clicking on the seek bar to seek somewhere does seek successfully, but the seek bar quickly moves to about 2 seconds forward from where it was. The time indicator and seek bar say that it is playing from where it was, but the actual video is playing from where the seek went to.
I poked around in the code for a bit and found some confusing parts that seem to have something to do with it (near line 4627 of modules/demux/mp4/mp4.c):
if( !b_has_base_media_decode_time && i_moof_time != INT64_MAX )
i_traf_start_time = MP4_rescale( i_moof_time, p_sys->i_timescale, p_track->i_timescale );
/* That should not happen */
if( !b_has_base_media_decode_time )
i_traf_start_time = MP4_rescale( p_sys->i_nztime, CLOCK_FREQ, p_track->i_timescale );
I don't fully understand the workings here, but two things stand out to me:
- The former
if
is never effective. If it executes,b_has_base_media_decode_time
must be false, and therefore the secondif
also executes and overwrites thei_traf_start_time
. Previousif
s setb_has_base_media_decode_time
to true in order to avoid this, perhaps it was forgotten? - The first
if
rescalesi_moof_time
fromp_sys->i_timescale
top_track->i_timescale
, but I'm not sure i_moof_time is inp_sys->i_timescale
. Following it up(/along) the call chain, FragCreateTrunIndex<-FragPrepareChunk<-FragSeekLoadFragment<-FragGetMoofBySidxIndex, we can see the line (~4777)*pi_sampletime = MP4_rescale( i_time, p_data->i_timescale, CLOCK_FREQ );
, which seems to indicate that the time is given in a CLOCK_FREQ timescale.
Again, I don't have a detailed understanding of how the demuxer works, but I can say that "fixing" these two things allow the video to seek normally. It is possible that our muxer is mixing something up and confusing VLC but I haven't found that (and the above code is suspicious to me anyway). In the meantime I will be seeing if our muxer can add the base media decode time to the tfdt atoms as a workaround.issue_26341