lua: fix likely double-frees and use of undefined pointer state
- Dec 29, 2021
-
-
Lyndon Brown authoredthe pointer `*pp_data`, provided by the callers of these callback functions, is left in an undefined state upon failed `asprintf()`. the returned success state is `VLC_SUCCESS` in such a circumstance, and along with questionable reliability of whether or not `*pi_data` equals `0` only under such a condition (consider the use of `vlclua_todata()`), means that it may in fact be impossible for calling code to determine whether or not it is safe to use their provided data pointer variable after execution of the callback. it is expected that calling code (difficult to track down due to the obfuscation of a callback mechanism) is likely misusing this unknown pointer state in such a failure condition, for instance via unconditional `free()`. resetting the pointer to null upon `asprintf()` failure eliminates the leak of unknown state, and assuming that calling code likely is modelled such that it initialises the variable to null, then upon return of `VLC_SUCCESS` from callback execution, unconditionally uses and frees the pointer, this should now work correctly in the scenario of `asprintf()` having failed. furthermore, the callback functions initially allocate memory to `*pp_data` via `vlclua_todata()`, then conditionally `free()` it before making an `asprintf()` call with `pp_data`. without the null reset, upon `asprintf()` failure, if the `asprintf()` implementation does not write to the pointer variable, and if this leads to the callback caller as above to `free()` the pointer variable, then `free()` will have been called twice upon the same allocation (a "double-free"). resetting the pointer to null upon failed `asprintf()` additionally fixes this issue. note that this had to be done in an additional third location in order to fully address the double-free bug.
7dfacd7e
-