err2.ivf is encoded after both commits, using the same soruce video and encoding settings. There are still some mismatches, but apparently not as many.
So, let's scratch all I wrote earlier. I'm displaying invisible frames (because the mismatches don't align well with block boundaries) and am seeing this weird thing where invisible frames (after --skip=158) for 9 decodes correctly, but for 4/2 they do not. However, this is really weird, the show_existing_frame for these do display correctly. [edit later: we do not use show-existing-frame in this sample at all.] It appears that some of the postprocessing we do is not included in the reference buffers? Or something like that. First block in this frame with a mismatch is the arm (by4=8-16,bx4=24-26). I'm focussing on the smaller block under it (8x8) at by4=22-24,bx4=20-22.