Skip to content

lua: fix likely double-frees and use of undefined pointer state

Lyndon Brown requested to merge jnqnfe/vlc:if4 into master

Split from !1085 (closed).

the pointer *pp_data, provided by the callers of these callback functions, is left in an undefined state upon failed asprintf().

the returned success state is VLC_SUCCESS in such a circumstance, and along with questionable reliability of whether or not *pi_data equals 0 only under such a condition (consider the use of vlclua_todata()), means that it may in fact be impossible for calling code to determine whether or not it is safe to use their provided data pointer variable after execution of the callback. it is expected that calling code (difficult to track down due to the obfuscation of a callback mechanism) is likely misusing this unknown pointer state in such a failure condition, for instance via unconditional free().

resetting the pointer to null upon asprintf() failure eliminates the leak of unknown state, and assuming that calling code likely is modelled such that it initialises the variable to null, then upon return of VLC_SUCCESS from callback execution, unconditionally uses and frees the pointer, this should now work correctly in the scenario of asprintf() having failed.

furthermore, the callback functions initially allocate memory to *pp_data via vlclua_todata(), then conditionally free() it before making an asprintf() call with pp_data. without the null reset, upon asprintf() failure, if the asprintf() implementation does not write to the pointer variable, and if this leads to the callback caller as above to free() the pointer variable, then free() will have been called twice upon the same allocation (a "double-free"). resetting the pointer to null upon failed asprintf() additionally fixes this issue. note that this had to be done in an additional third location in order to fully address the double-free bug.

Edited by Lyndon Brown

Merge request reports