Incorrect calculation of reference frames when using segmentation

If the segmentation has reference frame -1 (coded as zero), we are falling through to reading stuff from the bitstream, but should actually keep the -1.