This completes 24ae7ed5. Even if the
timer is removed from the schedule, there is no warranty that it's not
pending execution, or waiting on our mutex, at the very moment.
So there is a small window of opportunity for the assertion to fail.

This removes the assertion in favor of a safer if(likely()).