demux: fix invalid free

upon `asprintf()` failure, the pointer variable used to capture
the result is left in an undefined state.

here, should such failure occur, the later `free(modbuf)` call
would be invalid. we must reset this variable to null upon
failure to fix this.

note that `modbuf` is not the same variable as `module`, which
seems to be causing some confusion in review. this patch has
absolutely no impact upon the operation of this function other
than fixing the bug just described only. it **does not** cause
the module search to operate upon a null search string. it does
not change the search string at all.

note that `module` initially holds a search string owned by the
caller of the function. the function may though need to replace
this with a new string that it must allocate on the heap. for
the purposes of ensuring that the function later only releases
the new string, a copy of the allocated string pointer is kept
in the `modbuf` variable. in fact the string pointer in `module`
is only replaced with the newly allocated one upon success of
`asprintf()`. thus, should the allocation fail, `module` will
remain as `"any"` (the allocation only occurs under an `"any"`
based search). the patch thus does not alter the search string,
and thus does not impact module loading in any way.