[3.0] demux: mp4: add sanity check on duration

The duration is coded on 4 or 8 bytes as unsigned. But it is converted to signed in the MP4_rescale() which creates bogus computation. A scaled duration should likely never reach INT64_MAX.

Fixes #29256 (closed)

Backport of !7576 (merged)