demux: ps: improve LPCM/MLP AOB detection
The current code fails to detect MLP packets with p_pkt[i_start + 6] == 0x80. p_pkt[i_start + 5] >= 0xc0 is almost never true.
This test is needed to differentiate between AOB and VOB sources which have different IDs in the PS data. LPCM and MLP seem to have a fixed substream ID (this is already assumed by the code). We can split the detection for LPCM and MLP.
LPCM in VOB always have a byte set to 0x80 and never in AOB, this can easily do the differentiation.
MLP doesn't exist in VOB, however the 0xa1 substream ID could correspond to LPCM in VOB. We need some extra check that could help determine which it is. From testing on multiple AOB MLP files it seems
p_pkt[i_start + 3] == 0x06 works. I do not have a VOB with LPCM ID set to 0xa1 to verify it's never true.I used the MLP syncword from the MLP packetizer to check the data actually contain MLP data. That should be (mostly) safe.
This will be needed by !7542 (merged) to read all DVD-Audios properly.
There doesn't seem to be any way to identify MLP streams realiably, so we just make an educated guess based on what the caller knows about the stream.