colorspace: fix math in pl_primaries_compatible()

This function is supposed to work by checking if each primary in colorspace a is closer to the corresponding primary in colorspace b than to that colorspace's other primaries. However, that's not what it actually does for the blue and green primaries: instead of computing GR, the distance from a's green to b's red, it reuses RG, the distance from a's red to b's green, which is not the same. The same issue applies to BR and BG, which should both be computed but aren't.

Here are the distances that ought to be computed for the green channel when checking compatibility between BT.2020 and BT.709 (GG compared to GR and GB):

And here's what's actually computed by the current code (GG compared to RG and GB):