VLC doesn't display 12-bit AV1 content with dav1d decoder

VLC doesn't display (black screen) 12-bit AV1 content with dav1d decoder. Tested with both WebM and IVF files and 4:0:0, 4:2:0, 4:2:2 and 4:4:4 chroma subsampling. All 10-bit files work. With the aom decoder 12-bit content is displayed,

Build: vlc-4.0.0-20190107-0831-dev-win64

Samples: https://mega.nz/#F!mrIyBQRZ!CMtErHbhHdnzk6FnX3a3IA

added Severity::normal Type::bug Version::master git + 1 deleted label

changed milestone to %Bugs paradize

added Component::Decoders label and removed 1 deleted label

Added attachment:

• VLC-debug.html

Original author: EwoutH

Added a log, with these errors:

chain error: Too high level of recursion (3)
main error: Failed to create video converter
main error: Failed to adapt decoder format to display
direct3d11 error: SetThumbNailClip failed: 0x800706f4
main error: video output creation failed
main error: failed to create video output
dav1d error: Decoder error -12!
main error: buffer deadlock prevented

When playing a 10-bit clip the only error that I get is

direct3d11 error: SetThumbNailClip failed: 0x800706f4

Original author: EwoutH

The video can be paused and unpaused, and the blue progress bar is moving.

Edit: Also added a log with the aom encoder preference.

Original author: EwoutH

Added attachment:

• VLC-debug-aom.html

Original author: EwoutH

This is probably the problem in dav1d.c. Kudo's to James Almer for the find.

static const struct
{
    vlc_fourcc_t          i_chroma;
    enum Dav1dPixelLayout i_chroma_id;
    uint8_t               i_bitdepth;
} chroma_table[] =
{
    {VLC_CODEC_GREY, DAV1D_PIXEL_LAYOUT_I400, 8},
    {VLC_CODEC_I420, DAV1D_PIXEL_LAYOUT_I420, 8},
    {VLC_CODEC_I422, DAV1D_PIXEL_LAYOUT_I422, 8},
    {VLC_CODEC_I444, DAV1D_PIXEL_LAYOUT_I444, 8},

    {VLC_CODEC_I420_10L, DAV1D_PIXEL_LAYOUT_I420, 10},
    {VLC_CODEC_I422_10L, DAV1D_PIXEL_LAYOUT_I422, 10},
    {VLC_CODEC_I444_10L, DAV1D_PIXEL_LAYOUT_I444, 10},
};

So 12-bit and 10-bit 4:0:0 (monochrome) need to be added. Is this right?

static const struct
{
    vlc_fourcc_t          i_chroma;
    enum Dav1dPixelLayout i_chroma_id;
    uint8_t               i_bitdepth;
} chroma_table[] =
{
    {VLC_CODEC_GREY, DAV1D_PIXEL_LAYOUT_I400, 8},
    {VLC_CODEC_I420, DAV1D_PIXEL_LAYOUT_I420, 8},
    {VLC_CODEC_I422, DAV1D_PIXEL_LAYOUT_I422, 8},
    {VLC_CODEC_I444, DAV1D_PIXEL_LAYOUT_I444, 8},

    {VLC_CODEC_GREY_10L, DAV1D_PIXEL_LAYOUT_I400, 10},
    {VLC_CODEC_I420_10L, DAV1D_PIXEL_LAYOUT_I420, 10},
    {VLC_CODEC_I422_10L, DAV1D_PIXEL_LAYOUT_I422, 10},
    {VLC_CODEC_I444_10L, DAV1D_PIXEL_LAYOUT_I444, 10},

    {VLC_CODEC_GREY_12L, DAV1D_PIXEL_LAYOUT_I400, 12},
    {VLC_CODEC_I420_12L, DAV1D_PIXEL_LAYOUT_I420, 12},
    {VLC_CODEC_I422_12L, DAV1D_PIXEL_LAYOUT_I422, 12},
    {VLC_CODEC_I444_12L, DAV1D_PIXEL_LAYOUT_I444, 12},
};

Original author: EwoutH

assigned to @robUx4 and unassigned @MigrationBot

Fixed by 1243e37e544415158b3cd202a0f12016924967a6

added Status::fixed label

closed