[3.0] demux: mp4: discard edit list if entries cannot be converted to vlc_tick_t

The values are likely wrong.

Fixes #28997 (closed)

(cherry picked from commit d1f757d7)

This adds a new option to 3.0

Backport of !7547 (merged) and !772 (merged)