qsv: fix an error handling crash

By delaying encoder->p_sys initialization, a03fdfd9 introduced a crash due to Close requiring encoder->p_sys to be valid.

This patches refactors the encoder closure to workaround that.