Do something with AVAmbientViewingEnvironment
I was on a call with some VideoToolbox folks from Apple earlier and discussing HDR tone mapping, and they pointed me at the AmbientViewingEnvironment
MP4 box (exposed from lavc as AVAmbientViewingEnvironment
when coded as an HEVC SEI) as being substantially relevant to correct mapping of user-generated HDR content. In particular, iPhones (currently) report an expected ambient viewing environment of ~314 lux, while most movies are mastered for something closer to ~16. Video shot on iPhones expects to be tone-mapped with that difference in mind.
It's not entirely clear what we're supposed to do with this information, and there apparently isn't an exact standard, but they suggested trying playing with the values and seeing how QuickTime responds. I'm filing this issue to keep track of that investigation, and any work to actually do anything with the information that we decide is useful.
I'd imagine mpv will need a corresponding issue, and ffmpeg will need one to report this from MP4 metadata in addition to the SEI, but I figured I'd file here first, since I'd expect this to be where people are most likely to have any idea of what to actually do with the information.