This rather unfortunately requires segregating large parts of shaders/icc.c into 3-channel and 1-channel code paths.
shaders/icc.c
Example image: