avsb: fix invalid state when stopping/flush

In some corner cases where the module is flushed more than one times in a raw, it was possible that the renderer and observer were first started during the stop process