Skip to content
Snippets Groups Projects
Commit 313d3f8e authored by Martin Storsjö's avatar Martin Storsjö
Browse files

d3d_dynamic_shader: Fix uses of out of bounds values in MultMat43

Since 6ab96386, the matrices in the
function are only 4x3, but the function kept using the fourth row of
the right hand matrix.

The out of bounds values were in practice multiplied with 0, which is
why this wasn't initially noticed. However, 0 * <uninit> can also
evaluate to NaN, if the uninitialized data happens to make the bit
pattern of a NaN.

In the upcoming Clang 14, the default options for float handling
were changed slightly (in Clang commit
https://github.com/llvm/llvm-project/commit/f04e387055e495e3e14570087d68e93593cf2918).

As Clang can see that these multiplications use undefined values out of
bounds, it assumes that those values are NaNs, which let Clang essentially
optimize out most of D3D_SetupQuad, just setting all of
quad->shaderConstants->Colorspace to NaNs.

This fixes the colorspace matrix when built with Clang 14, by assuming
the fourth, missing, row of the right hand matrix in MultMat43 to be
[0, 0, 0, 1].
parent c8797d1a
No related branches found
No related tags found
1 merge request!995d3d_dynamic_shader: Fix uses of out of bounds values in MultMat43
Pipeline #167148 passed with stage
in 14 minutes and 56 seconds
Loading
0% Loading or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment