Commit a4178cc2 authored by Ronald S. Bultje's avatar Ronald S. Bultje

av1: use chroma txtp inference over default DCT_DCT if qidx=0

Fixes #320. The problem here is that qidx=0 is (in libaom) a shortcut
for lossless, which normally becomes WHT_WHT, but under some obscure
conditions, it can also be non-lossless, in which case qidx=0 implies
DCT_DCT for luma. For chroma, apparently we should use the default
inference pattern, which becomes DCT_DCT for inter also, but requires
the standard lookup table for intra.
parent 1d36922f
Pipeline #12197 passed with stages
in 15 minutes and 16 seconds
......@@ -352,13 +352,17 @@ static int decode_coefs(Dav1dTileContext *const t,
if (lossless) {
assert(t_dim->max == TX_4X4);
*txtp = WHT_WHT;
} else if (!f->frame_hdr->segmentation.qidx[b->seg_id] ||
t_dim->max + intra >= TX_64X64)
} else if (t_dim->max + intra >= TX_64X64) {
*txtp = DCT_DCT;
} else if (chroma) {
// inferred from either the luma txtp (inter) or a LUT (intra)
*txtp = intra ? dav1d_txtp_from_uvmode[b->uv_mode] :
get_uv_inter_txtp(t_dim, *txtp);
} else if (!f->frame_hdr->segmentation.qidx[b->seg_id]) {
// In libaom, lossless is checked by a literal qidx == 0, but not all
// such blocks are actually lossless. The remainder gets an implicit
// transform type (for luma)
*txtp = DCT_DCT;
} else {
unsigned idx;
if (intra) {
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment