RTMP-Stream Video only works sometimes. Audio is fine.

Summary

Video playback of rtmp streams only works sometimes. Audio works fine. Same behaviour in:

• Unity Editor
• Windows Standalone Build
• Android (Quest 2) Build
• Android (Focus 3) Build

Minimal project and steps to reproduce

1. Create a new Unity Project (3D URP) with Unity Version 2021.1.28f1
2. Import "VLC for Unity (Windows)" through the package manager (v0.1.7)
3. Import "VLC for Unity (Android)" through the package manager (v0.1.7)
4. Open the included Demo-Scene "VLC Canvas Example"
5. Edit the path to a rtmp stream (e.g. rtmp://[some-host]/live)
6. Hit play (sometimes video works, sometimes video doesn't)
7. (Optional) Clicking the stop button, waiting a few seconds, and clicking Play will sometimes make the video play.

Note: Only installing one package also results in the same behaviour.

What is the current bug behavior?

The audio of the stream is played everytime but most of the times there is no video. (only the VLC cone stays visible)

What is the expected correct behavior?

Video should always play.

Does it work on other plaforms? Does it work with the official VLC apps?

The stream is received properly when using the VLC Media Player for Windows.

Relevant logs and/or screenshots

Unity Console Log when enabling "Log To Console" (video failed)

VLCPlayerExample Play
avcodec Debug: AVFormat(ffmpeg Lavf58.76.100) supported stream
avcodec Debug:     - format = flv (FLV (Flash Video))
main Debug: selecting program id=0
avcodec Debug: adding es: audio codec = mp4a (86018)
main Debug: ES track added: 'audio/0' (fourcc: 'mp4a')
avcodec Debug:     - duration = 0
main Debug: using demux module "avcodec"
main Debug: looking for audio decoder module matching "any": 20 candidates
faad Debug: using version unknown - 2.10.0
main Debug: using audio decoder module "faad"
main Debug: ES track selected: 'audio/0' (fourcc: 'mp4a')
main Debug: `rtmp://<REDACTED>/live' successfully opened
main Debug: program(0): using clock source: 'audio'
main Debug: Buffering 0%
main Debug: Buffering 2%
[shortened]
main Debug: Buffering 55%
avcodec Debug:     - start time = 6472293000
main Debug: Buffering 57%
[shortened]
main Debug: Buffering 70%
faad Warning: decoded zero sample
SetupTrackButtons
VLCPlayerExample Tracks Video
VLCPlayerExample SelectedTrack Video
VLCPlayerExample Tracks Audio
VLCPlayerExample SelectedTrack Audio
VLCPlayerExample Tracks Text
VLCPlayerExample SelectedTrack Text
main Debug: Buffering 91%
waveout Debug: Drivername: Microsoft Sound Mapper
waveout Debug: Manufacturer identifier: 0x1
waveout Debug: Driver Version: 5.0
waveout Debug: Product identifier: 0x2
waveout Debug: OpenWaveDevice-ID: 4294967295
main Debug: reusing audio output
main Debug: Buffering 89%
waveout Debug: waveformat.Format.wFormatTag      = 3
main Debug: Buffering 96%
waveout Debug: Trying 2 channels
main Debug: Stream buffering done (1003 ms in 1 ms)
main Debug: Buffering 98%
main Debug: Buffering 93%
waveout Debug: waveformat.Format.cbSize          = 0
waveout Debug: waveformat.Format.nChannels       = 2
waveout Debug: waveformat.Format.nSamplesPerSec  = 48000
waveout Debug: waveformat.Format.nAvgBytesPerSec = 384000
waveout Debug: waveformat.Format.nBlockAlign     = 8
waveout Debug: waveformat.Format.wBitsPerSample  = 32
waveout Debug: waveformat.Samples.wValidBitsPerSample = 32
waveout Debug: waveformat.Samples.wSamplesPerBlock = 32
waveout Debug: waveformat.dwChannelMask          = 3
main Debug: output 'f32l' 48000 Hz Stereo frame=1 samples/8 bytes
main Debug: using audio volume module "float_mixer"
main Debug: looking for audio volume module matching "any": 2 candidates
main Debug: input 'f32l' 48000 Hz Stereo frame=1 samples/8 bytes
main Debug: looking for audio filter module matching "scaletempo": 1 candidates
scaletempo Debug: format: 48000 rate, 2 nch, 4 bps, fl32
scaletempo Debug: params: 30 stride, 0.200 overlap, 14 search
scaletempo Debug: 1.000 scale, 1440.000 stride_in, 1440 stride_out, 1152 standing, 288 overlap, 672 search, 2400 queue, fl32 mode
main Debug: using audio filter module "scaletempo"
main Debug: conversion: 'f32l'->'f32l' 48000 Hz->48000 Hz Stereo->Stereo
main Debug: conversion pipeline complete
main Debug: conversion: 'f32l'->'f32l' 48000 Hz->48000 Hz Stereo->Stereo
main Debug: conversion pipeline complete
main Debug: looking for audio resampler module matching "any": 3 candidates
soxr Debug: Using SoX Resampler with 'cr32s' engine and 'Medium 16-bit with medium roll-off' quality to convert f32l/48000Hz to f32l/48000Hz.
main Debug: using audio resampler module "soxr"
main Debug: Decoder wait done in 9 ms

Unity Console Log on a successful video playback

VLCPlayerExample Play
main Debug: Creating an input for 'rtmp://[REDACTED]/live'
main Debug: using timeshift granularity of 50 MiB
main Debug: using timeshift path: C:\Users\[REDACTED]\AppData\Local\Temp
main Debug: `rtmp://[REDACTED]/live' gives access `rtmp' demux `any' path `[REDACTED]/live'
main Debug: creating access: rtmp://[REDACTED]/live
main Debug:  (path: \\[REDACTED]\live)
main Debug: looking for access module matching "rtmp": 1 candidates
avcodec Debug: CPU flags: 0x000fd3db
main Debug: using access module "avcodec"
avcodec Debug: seekable, size=-40
main Debug: looking for stream_filter module matching "prefetch,cache": 3 candidates
cache_read Debug: Using stream method for AStream*
cache_read Debug: starting pre-buffering
cache_read Debug: received first data after 0 ms
cache_read Debug: pre-buffering done 1024 bytes in 0s - 7407 KiB/s
main Debug: using stream_filter module "cache_read"
main Debug: looking for stream_filter module matching "any": 4 candidates
main Debug: no stream_filter modules matched with name any
main Debug: looking for stream_directory module matching "any": 1 candidates
main Debug: no stream_directory modules matched with name any
main Debug: attachment of directory-extractor failed for rtmp://[REDACTED]/live
main Debug: looking for stream_filter module matching "record": 1 candidates
main Debug: using stream_filter module "record"
main Debug: creating demux "any", URL: rtmp://[REDACTED]/live, path: (null)
main Debug: looking for demux module matching "any": 64 candidates
main Debug: looking for xml reader module matching "any": 1 candidates
main Debug: using xml reader module "xml"
main Debug: removing "xml reader" module "xml"
webvtt Debug: subtitle demux discarded
ts Debug: TS module discarded (lost sync)
mod Debug: MOD validation failed (ext=)
avcodec Debug: trying url: rtmp://[REDACTED]/live
avcodec Debug: CPU flags: 0x000fd3db
avcodec Debug: detected format: flv
main Debug: selecting program id=0
main Debug: ES track added: 'audio/0' (fourcc: 'mp4a')
avcodec Debug: adding es: audio codec = mp4a (86018)
main Debug: ES track added: 'video/1' (fourcc: 'h264')
avcodec Debug: adding es: video codec = h264 (27)
avcodec Debug: AVFormat(ffmpeg Lavf58.76.100) supported stream
avcodec Debug:     - format = flv (FLV (Flash Video))
avcodec Debug:     - start time = 7704208000
avcodec Debug:     - duration = 0
main Debug: using demux module "avcodec"
main Debug: looking for audio decoder module matching "any": 20 candidates
faad Debug: using version unknown - 2.10.0
main Debug: using audio decoder module "faad"
main Debug: ES track selected: 'audio/0' (fourcc: 'mp4a')
main Debug: looking for video decoder module matching "any": 17 candidates
avcodec Debug: using ffmpeg Lavc58.134.100
avcodec Debug: CPU flags: 0x000fd3db
avcodec Debug: allowing 6 thread(s) for decoding
avcodec Debug: codec (h264) started
avcodec Debug: using frame thread mode with 6 threads
main Debug: using video decoder module "avcodec"
main Debug: ES track selected: 'video/1' (fourcc: 'h264')
main Debug: `rtmp://[REDACTED]/live' successfully opened
main Debug: Buffering 0%
main Debug: program(0): using clock source: 'audio'
main Debug: Buffering 2%
[shortened]
main Debug: Buffering 17%
faad Warning: decoded zero sample
main Debug: Buffering 19%
main Debug: Buffering 21%
main Debug: Buffering 23%
main Debug: reusing audio output
main Debug: Buffering 25%
[shortened]
main Debug: Buffering 36%
SetupTrackButtons
main Debug: Buffering 38%
VLCPlayerExample Tracks Video
main Debug: Buffering 40%
[shortened]
main Debug: Buffering 55%
VLCPlayerExample SelectedTrack Video
main Debug: Buffering 57%
[shortened]
main Debug: Buffering 91%
VLCPlayerExample Tracks Audio
main Debug: Buffering 93%
VLCPlayerExample SelectedTrack Audio
main Debug: Buffering 96%
main Debug: Buffering 98%
main Debug: Stream buffering done (1002 ms in 1 ms)
VLCPlayerExample Tracks Text
VLCPlayerExample SelectedTrack Text
waveout Debug: Drivername: Microsoft Sound Mapper
waveout Debug: Driver Version: 5.0
waveout Debug: Manufacturer identifier: 0x1
waveout Debug: Product identifier: 0x2
waveout Debug: Trying 2 channels
waveout Debug: OpenWaveDevice-ID: 4294967295
waveout Debug: waveformat.Format.cbSize          = 0
waveout Debug: waveformat.Format.wFormatTag      = 3
waveout Debug: waveformat.Format.nChannels       = 2
waveout Debug: waveformat.Format.nSamplesPerSec  = 48000
waveout Debug: waveformat.Format.nAvgBytesPerSec = 384000
waveout Debug: waveformat.Format.nBlockAlign     = 8
waveout Debug: waveformat.Format.wBitsPerSample  = 32
waveout Debug: waveformat.Samples.wValidBitsPerSample = 32
waveout Debug: waveformat.Samples.wSamplesPerBlock = 32
waveout Debug: waveformat.dwChannelMask          = 3
main Debug: output 'f32l' 48000 Hz Stereo frame=1 samples/8 bytes
main Debug: looking for audio volume module matching "any": 2 candidates
main Debug: using audio volume module "float_mixer"
main Debug: input 'f32l' 48000 Hz Stereo frame=1 samples/8 bytes
main Debug: looking for audio filter module matching "scaletempo": 1 candidates
scaletempo Debug: format: 48000 rate, 2 nch, 4 bps, fl32
scaletempo Debug: params: 30 stride, 0.200 overlap, 14 search
scaletempo Debug: 1.000 scale, 1440.000 stride_in, 1440 stride_out, 1152 standing, 288 overlap, 672 search, 2400 queue, fl32 mode
main Debug: using audio filter module "scaletempo"
main Debug: conversion: 'f32l'->'f32l' 48000 Hz->48000 Hz Stereo->Stereo
main Debug: conversion pipeline complete
main Debug: conversion: 'f32l'->'f32l' 48000 Hz->48000 Hz Stereo->Stereo
main Debug: conversion pipeline complete
main Debug: looking for audio resampler module matching "any": 3 candidates
soxr Debug: Using SoX Resampler with 'cr32s' engine and 'Medium 16-bit with medium roll-off' quality to convert f32l/48000Hz to f32l/48000Hz.
main Debug: using audio resampler module "soxr"
main Error: buffer deadlock prevented
main Debug: Decoder wait done in 8 ms
avcodec Debug: available hardware decoder output format 119 (cuda)
avcodec Debug: available hardware decoder output format 118 (d3d11va_vld)
avcodec Debug: available hardware decoder output format 53 (dxva2_vld)
avcodec Debug: available hardware decoder output format 174 (d3d11)
avcodec Debug: available software decoder output format 0 (yuv420p)
avcodec Debug: trying format d3d11va_vld
main Debug: vout: none found
main Debug: vout change: decoder size
main Debug: vout change: visible size
main Debug: looking for text renderer module matching "any": 2 candidates
freetype Debug: Using DWrite backend
main Debug: using text renderer module "freetype"
main Debug: looking for video converter module matching "any": 22 candidates
swscale Debug: 32x32 (32x32) chroma: YUVA colorspace: Undefined -> 16x16 (16x16) chroma: RGBA colorspace: Undefined with scaling using Bicubic (good quality)
main Debug: using video converter module "swscale"
main Debug: looking for video converter module matching "any": 22 candidates
yuvp Debug: YUVP to YUVA converter
main Debug: using video converter module "yuvp"
main Debug: Deinterlacing available
main Debug: deinterlace -1, mode auto, is_needed 0
main Debug: looking for vout window module matching "wextern": 4 candidates
main Debug: using vout window module "wextern"
main Debug: looking for inhibit module matching "any": 1 candidates
main Debug: using inhibit module "inhibit"
main Debug: looking for decoder device module matching "d3d11": 1 candidates
d3d11_filters Warning: external ID3D11DeviceContext mutex not provided, using internal one
d3d11_filters Debug: ATI WDDM driver 27.20.20913.2000
main Debug: using decoder device module "d3d11_filters"
d3d11va Debug: CreateDevice succeed
d3d11va Debug: - 'MPEG-2 variable-length decoder' is supported
d3d11va Debug: - 'MPEG-2 inverse discrete cosine transform' is supported
d3d11va Debug: - 'H.264 variable-length decoder, no film grain technology' is supported
d3d11va Debug: - 'H.264 variable-length decoder, no film grain technology, Flash' is supported
d3d11va Debug: - 'Unknown decoder 0x6719b6fb-0x5cad-0x4acb-0xb00af3bfdec38727' is supported
d3d11va Debug: - 'Unknown decoder 0x9901ccd3-0xca12-0x4b7e-0x867ae2223d9255c3' is supported
d3d11va Debug: - 'H.264 stereo high profile, mbs flag set' is supported
d3d11va Debug: - 'H.264 stereo high profile' is supported
d3d11va Debug: - 'VC-1 variable-length decoder' is supported
d3d11va Debug: - 'Unknown decoder 0xca15d19a-0x2b48-0x43d6-0x979e7a6e9c802ff8' is supported
d3d11va Debug: - 'MPEG-4 Part 2 variable-length decoder, Simple&Advanced Profile, Avivo' is supported
d3d11va Debug: - 'MPEG-4 Part 2 variable-length decoder, Simple&Advanced Profile, no GMC' is supported
d3d11va Debug: - 'VC-1 inverse discrete cosine transform' is supported
d3d11va Debug: - 'Windows Media Video 9 IDCT' is supported
d3d11va Debug: - 'Unknown decoder 0x103473e4-0x10ea-0x11df-0x9a922ba055d89593' is supported
d3d11va Debug: - 'Unknown decoder 0x84ad67f6-0x4c21-0x419a-0x9f0b24f0578906c1' is supported
d3d11va Debug: - 'Unknown decoder 0x725ad240-0x786c-0x471e-0xad3c38f739936517' is supported
d3d11va Debug: - 'Unknown decoder 0x95664ff5-0x9e03-0x4c74-0xbb4f9178d6035e58' is supported
d3d11va Debug: - 'Unknown decoder 0x603a4756-0xa864-0x4f91-0xbb622c935b7a1391' is supported
d3d11va Debug: - 'Unknown decoder 0xc58b9a06-0x7e89-0x11e1-0xbb0070b34824019b' is supported
d3d11va Debug: Trying to use 'H.264 variable-length decoder, no film grain technology' as input
d3d11va Debug: favor decoder format VA_NV12
d3d11va Debug: NV12 output is supported for decoder H.264 variable-length decoder, no film grain technology.
d3d11va Debug: Using output format NV12 for decoder H.264 variable-length decoder, no film grain technology
d3d11va Debug: va_pool_SetupDecoder id 27 1920x1088 count: 26
d3d11va Debug: ID3D11VideoDecoderOutputView succeed with 26 surfaces (1920x1088)
d3d11va Debug: we got 3 decoder configurations
d3d11va Debug: configuration[0] ConfigBitstreamRaw 2
d3d11va Debug: configuration[1] ConfigBitstreamRaw 2
d3d11va Debug: configuration[2] ConfigBitstreamRaw 2
d3d11va Debug: DxCreateDecoderSurfaces succeed
d3d11va Notice: Using D3D11VA (AMD Radeon (TM) R9 390 Series, vendor 1002(ATI), device 67b1, revision 80)
main Debug: vout change: chroma
main Debug: requested window size: 1920x1080
main Debug: resized to 1920x1080
main Debug: deinterlace -1, mode auto, is_needed 0
main Debug: Opening vout display wrapper
main Debug: looking for vout display module matching "direct3d11": 1 candidates
direct3d11 Debug: Using pixel format VA_NV12 for chroma DX11
direct3d11 Warning: don't know how to transfer from 1 to 1
direct3d11 Debug: using GPU render fence
direct3d11 Warning: don't know how to transfer from 1 to 1
direct3d11 Debug: Direct3D11 resources created
direct3d11 Debug: Direct3D11 display adapter successfully initialized
direct3d11 Debug: Direct3D11 Open Succeeded
main Debug: using vout display module "direct3d11"
main Debug: original format sz 1920x1088, of (0,0), vsz 1920x1080, 4cc DX11, sar 1:1, orient: normal, msk r0x0 g0x0 b0x0
main Debug: removing "text renderer" module "freetype"
main Debug: looking for text renderer module matching "any": 2 candidates
freetype Debug: Using DWrite backend
main Debug: using text renderer module "freetype"
main Debug: requested window size: 1920x1080
main Debug: looking for sub source module matching "logo": 1 candidates
logo Debug: logo file name logo.png, delay -1, alpha -1
main Debug: creating access: file:///C:/_REPOS/VLC_RTMP/logo.png
main Debug:  (path: C:\_REPOS\VLC_RTMP\logo.png)
main Debug: looking for access module matching "file": 5 candidates
main Debug: using access module "filesystem"
main Debug: looking for stream_filter module matching "prefetch,cache": 3 candidates
cache_read Debug: Using stream method for AStream*
cache_read Debug: starting pre-buffering
cache_read Debug: received first data after 0 ms
cache_read Debug: pre-buffering done 1024 bytes in 0s - 11111 KiB/s
main Debug: using stream_filter module "cache_read"
main Debug: looking for stream_filter module matching "any": 4 candidates
main Debug: no stream_filter modules matched with name any
main Debug: removing "stream_filter" module "cache_read"
main Debug: removing "access" module "filesystem"
main Debug: looking for video decoder module matching "any": 17 candidates
main Debug: using video decoder module "png"
main Debug: looking for video converter module matching "any": 22 candidates
swscale Debug: 600x180 (600x180) chroma: RGBA colorspace: BT709 -> 600x180 (600x180) chroma: YUVA colorspace: BT709 with scaling using Bicubic (good quality)
main Debug: using video converter module "swscale"
main Debug: removing "video decoder" module "png"
main Debug: removing "video converter" module "swscale"
main Debug: using sub source module "logo"
main Debug: Filter 'logo' (000002790B0DC020) appended to chain
main Warning: original picture size is undefined

Environment

• OS: Windows 10
• Version 21H2
• Device: PC
• Unity Version: 2021.1.28f1
• LibVLC version and architecture: 4.0.0-dev-22482-g4cca6109e2
• LibVLCSharp version: 4.0.0.0
• VLC Unity plugin version: Windows 0.1.7 & Android 0.1.7
• Scripting backend used: Mono