This shouldn't be enforced. We already use -O2 on release builds. -O4 was removed in 8d098023 -O3 was added with the contrib in 9c0b836a
mentioned in commit 220091ac