Neither --buildtype=plain nor --buildtype=debug set -ffast-math, so
llround() is kept as a function call and isn’t optimised out into
cvttsd2siq (on amd64), thus requiring the math lib to be linked.

Note that even with -ffast-math, it isn’t guaranteed that a call to
llround() will always be omitted (I have reproduced this on PowerPC), so
this fix is correct even if we ever decide to enable -ffast-math in
other build types.