Commit d4e07786 authored by Fiona Glaser's avatar Fiona Glaser

Fix regression in r736

r736 added intra RD refinement to B-frames; however, it is possible for subme=7 to be used without b-rdo.
This means intra RD isn't run, and therefore it is possible for intra chroma analysis to not have been run, since update_cache was never called for an intra block, and chroma ME is not required even at subme=7.
r801, which removed a memset, made this worse because previously the chroma prediction mode was at least initialized to zero; now it was not initialized at all.
Therefore, --no-chroma-me, --subme 7, and no --b-rdo had the potential to crash.
This change restricts intra RD refinement to only be run when --b-rdo is enabled (sensible to begin with), thus preventing a crash in this case.
parent 3a095b2c
......@@ -2580,7 +2580,7 @@ void x264_macroblock_analyse( x264_t *h )
h->mb.i_type = i_type;
h->mb.i_partition = i_partition;
if( h->mb.i_subpel_refine >= 7 && IS_INTRA( i_type ) )
if( analysis.b_mbrd && h->mb.i_subpel_refine >= 7 && IS_INTRA( i_type ) )
x264_intra_rd_refine( h, &analysis );
else if( h->param.analyse.b_bidir_me )
refine_bidir( h, &analysis );
......
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment