GitLab

This method (--b-adapt 2) uses a Viterbi algorithm somewhat similar to that used in trellis quantization.
Note that it is not fully optimized and is very slow with large --bframes values.
It also takes into account weightb, which should improve fade detection.
Additionally, changes were made to cache lowres intra results for each frame to avoid recalculating them.  This should improve performance in both B-frame decision methods.
This can also be done for motion vectors, which will dramatically improve b-adapt 2 performance when it is complete.
This patch also reads b_adapt and scenecut settings from the first pass so that the x264 header information in the output file will have correct information (since frametype decision is only done on the first pass).

This change improves VBV accuracy and improves bit distribution in CRF and 2pass.
Instead of being applied after ratecontrol, AQ becomes part of the complexity measure that ratecontrol uses.
This allows for modularity for changes to AQ; a new AQ algorithm can be introduced simply by introducing a new aq_mode and a corresponding if in adaptive_quant_frame.
This also allows quantizer field smoothing, since quantizers are calculated beofrehand rather during encoding.
Since there is no more reason for it, aq_mode 1 is removed.  The new mode 1 is in a sense a merger of the old modes 1 and 2.
WARNING: This change redefines CRF when using AQ, so output bitrate for a given CRF may be significantly different from before this change!

In some rare (but significant) cases, the optimized nal_encode algorithm gave incorrect results.

Also remove 5-year-old unnecessary #define that reduced speed unnecessarily under MSVC-compiled builds

Small speedup at very high bitrates

Also move some small functions in macroblock.c to a .h file so they can be inlined.

Unrolling the loop a bit improves performance

Take advantage of the H prediction method invented for merged intra SAD and apply it to regular prediction, too.

Roughly 30% faster i16x16 analysis under subme=1

Distinguish between unaligned and aligned uses of mbcmp
SAD_aligned, for MMX SADs, uses non-cacheline SADs.

(changes 2pass statsfile format)

Resolves some issues with QP51 in I-frames with scenecut

Due to a bug in gcc 3.4.x, in certain cases of inlining, the array_non_zero_int_mmx inline asssembly is miscompiled and causes a crash with --subme 7 --8x8dct.
This minor hack fixes this issue.

move some more shared macros to x264util.asm

A bit more loop unrolling, and moving some constant code to the global init function

Convert the style of the deblocking code to the standard x264 style
Eliminate some trailing whitespace

Along with a bit of related code reorganization and macroification

Faster variance calculation

Early termination for bS=0, alpha=0, beta=0
Refactoring, various other optimizations
About 30% faster deblocking overall.

Split into a separate commit because this changes rounding, and thus changes output slightly.

if vbv is not enabled (and especially in crf/cqp), we have to guess max bitrate, so we might underestimate the required level.

(no immediate effect, since nothing writes such values yet)

In rare cases CABAC desync could occur, causing bitstream corruption

Bigendian systems didn't have endian_fix32 defined