Skip to content
Snippets Groups Projects
Commit 4b2e6df3 authored by Pierre Lamot's avatar Pierre Lamot
Browse files

rawvid: fix rounding division when calculating pitch

formats like NV12 where num and den are 2 had the wrong pitch

size is divided then multiplied, consistently with `rawvideo` decoder

(cherry-picked from commit 128f7ff6)
parent 96df5fd9
No related branches found
No related tags found
2 merge requests!3533Backport MR 3236 to 3.0.x,!3311[3.0] rawvid: fix rounding division when calculating pitch
Pipeline #315620 passed with stages
in 21 minutes and 21 seconds
......@@ -356,10 +356,10 @@ valid:
p_sys->frame_size = 0;
for (unsigned i=0; i<dsc->plane_count; i++)
{
unsigned pitch = (i_width + (dsc->p[i].w.den - 1))
* dsc->p[i].w.num / dsc->p[i].w.den * dsc->pixel_size;
unsigned lines = (i_height + (dsc->p[i].h.den - 1))
* dsc->p[i].h.num / dsc->p[i].h.den;
unsigned pitch = ((i_width + (dsc->p[i].w.den - 1)) / dsc->p[i].w.den)
* dsc->p[i].w.num * dsc->pixel_size;
unsigned lines = ((i_height + (dsc->p[i].h.den - 1)) / dsc->p[i].h.den)
* dsc->p[i].h.num;
p_sys->frame_size += pitch * lines;
}
p_sys->p_es_video = es_out_Add( p_demux->out, &p_sys->fmt_video );
......
0% Loading or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment