Would be nicer to rebuild SPS/PPS on the fly as most of the parameters are the same, but dealing with matrixes adds too much code :/