Skip to content
Snippets Groups Projects

Draft: add an option to use Link Time Optimization

Open Steve Lhomme requested to merge robUx4/vlc:lto into master
3 unresolved threads

In contribs and in VLC. For Windows builds it saves about 10% of size. But it takes a lot of time & memory to link. So we should only do that for nightlies and releases.

It's only enabled for Windows builds for now.

Draft to check all Windows targets builds in the CI with the option on.

Merge request reports

Members who can merge are allowed to add commits.

Merge request pipeline #475369 canceled

Merge request pipeline canceled for c1ff2598

Approval is optional
Merge blocked: 5 checks failed
Merge request must not be draft.
Unresolved discussions must be resolved.
Pipeline must succeed.
Merge conflicts must be resolved.
Merge request must be rebased, because a fast-forward merge is not possible.

Merge details

  • The source branch is 3519 commits behind the target branch.
  • 13 commits will be added to master.
  • Source branch will be deleted.

Activity

Filter activity
  • Approvals
  • Assignees & reviewers
  • Comments (from bots)
  • Comments (from users)
  • Commits & branches
  • Edits
  • Labels
  • Lock status
  • Mentions
  • Merge request status
  • Tracking
  • Steve Lhomme mentioned in merge request !5439 (merged)

    mentioned in merge request !5439 (merged)

  • Steve Lhomme added 22 commits

    added 22 commits

    • 626609cc...9bf69cf5 - 3 commits from branch videolan:master
    • 9bf69cf5...98d66b2a - 9 earlier commits
    • d63a05a5 - contrib: gettext: update to 0.22.5
    • 0d0b564e - contrib: gettext: don't build command-line tools when cross-compiling
    • 8e488730 - contrib: ffmpeg: fix Windows builds with LLVM LTO
    • 22ba8ae6 - contrib: add an option to enable Link Time Optimization
    • 66276e88 - contrib: ffmpeg: enable Link Time Optimization using the official flag
    • e355391d - contrib: qt: enable Link Time Optimization with the official flag
    • e69f9b1c - configure: add an option to enable Link Time Optimization
    • 7eb4447c - package/win32: add -t option to enable Link Time Optimization
    • 734c6270 - CI: enable LTO in windows nightlies
    • 0b4f1b30 - TEST CI: enable LTO on regular Windows builds

    Compare with previous version

  • Steve Lhomme added 44 commits

    added 44 commits

    • 0b4f1b30...20ec82c3 - 34 earlier commits
    • 21cc7261 - contrib: gcrypt: don't build command-line tools when cross-compiling
    • 7aa8def6 - contrib: orc: only build/install the library
    • f43ab851 - contrib: ffmpeg: fix Windows builds with LLVM LTO
    • c82db112 - contrib: add an option to enable Link Time Optimization
    • 9bee766e - contrib: ffmpeg: enable Link Time Optimization using the official flag
    • a6c7034e - contrib: qt: enable Link Time Optimization with the official flag
    • 508221d3 - configure: add an option to enable Link Time Optimization
    • c70c3424 - package/win32: add -t option to enable Link Time Optimization
    • 405c5581 - CI: enable LTO in windows nightlies
    • 6b01eb8a - TEST CI: enable LTO on regular Windows builds

    Compare with previous version

  • Steve Lhomme mentioned in merge request !5450 (merged)

    mentioned in merge request !5450 (merged)

  • Steve Lhomme added 46 commits

    added 46 commits

    • 6b01eb8a...72a52fad - 36 earlier commits
    • 8b33533f - contrib: orc: only build/install the library
    • 5b937ba0 - contrib: ffmpeg: fix Windows builds with LLVM LTO
    • aea74f8b - contrib: add an option to enable Link Time Optimization
    • 704771d8 - contrib: ffmpeg: enable Link Time Optimization using the official flag
    • e60ebc9a - contrib: qt: enable Link Time Optimization with the official flag
    • faff3d8c - configure: add an option to enable Link Time Optimization
    • e3b5635a - package/win32: add -t option to enable Link Time Optimization
    • 01ee89da - CI: enable LTO in windows nightlies
    • b5daa646 - TEST CI: enable LTO on regular Windows builds
    • 6c1059f7 - contrib: goom: bypass MMX checks and assume it's there

    Compare with previous version

  • Steve Lhomme added 56 commits

    added 56 commits

    • 6c1059f7...502824e3 - 30 commits from branch videolan:master
    • 502824e3...eed737a7 - 16 earlier commits
    • 5f219a99 - configure: fix libssp.a detection when LTO is enabled
    • 8e5b6867 - contrib: ffmpeg: fix Windows builds with LLVM LTO
    • 10cd31a8 - contrib: add an option to enable Link Time Optimization
    • 5a8d60a9 - contrib: ffmpeg: enable Link Time Optimization using the official flag
    • 0190fb6e - contrib: qt: enable Link Time Optimization with the official flag
    • 1be897e4 - configure: add an option to enable Link Time Optimization
    • 239bb209 - package/win32: add -t option to enable Link Time Optimization
    • 2b987bfe - package/win32: warn when Link Time Optimization is not used in Release mode
    • 62fed8e0 - CI: enable LTO in windows LLVM nightlies
    • 23913ff9 - TEST CI: enable LTO on regular Windows builds

    Compare with previous version

    • Very cool. What's the gain on installed size? not installer

      Edited by Jean-Baptiste Kempf
    • Author Developer

      For most DLLs that's about 20% smaller. For Qt I got mine from 55 MB to 40 MB, but I build with -O0. The ffmpeg DLL actually got bigger. I'd have to check with builds in release mode.

      We might turn this on in the CI for nightly builds. But if we want the full effect, we need to build the contribs with -flto as well. Either we always builds contribs that way, or we rebuild contribs in nightly all the time (for those targets using LTO).

    • Author Developer

      Or we separate nightly contribs (or simply LTO contribs) from regular contribs. So the contribs could be cached for many days if they don't change.

    • Author Developer

      BTW, once I get this to fully work with gcc, we could try this on 3.0 too, maybe before the next release.

    • LTO will likely expose latent bugs somewhere. I really don't think it's a good idea to enable in 3.0 at this point, as we simply don't have the QA capability.

    • "For most DLLs that's about 20% smaller." interesting notably for large ones.

      But we need numbers with -O2 release mode, though...

    • Either we always builds contribs that way, or we rebuild contribs in nightly all the time (for those targets using LTO).

      Prebuilt contribs with LTO on should probably use the -fno-fat-lto-objects option to reduce the prebuilt archive size and the build time.

    • Author Developer

      Prebuilt contribs with LTO on should probably use the -fno-fat-lto-objects option to reduce the prebuilt archive size and the build time.

      This might be a GCC thing. In llvm-mingw the LTO format is thin by default (ie it does not contain the non-lto variant to link with).

    • Please register or sign in to reply
  • Steve Lhomme added 54 commits

    added 54 commits

    • 23913ff9...c85ace4b - 42 commits from branch videolan:master
    • c85ace4b...5f2e0518 - 2 earlier commits
    • 517a626d - configure: fix libssp.a detection when LTO is enabled
    • 502349e7 - contrib: ffmpeg: fix Windows builds with LLVM LTO
    • 59b57e88 - contrib: add an option to enable Link Time Optimization
    • d96feb63 - contrib: ffmpeg: enable Link Time Optimization using the official flag
    • f0d3e4fb - contrib: qt: enable Link Time Optimization with the official flag
    • 98516073 - configure: add an option to enable Link Time Optimization
    • 76b9be13 - package/win32: add -t option to enable Link Time Optimization
    • b3ac1a46 - package/win32: warn when Link Time Optimization is not used in Release mode
    • e81a445a - CI: enable LTO in windows LLVM nightlies
    • 51dbe085 - TEST CI: enable LTO on regular Windows builds

    Compare with previous version

  • Steve Lhomme mentioned in merge request !5463 (closed)

    mentioned in merge request !5463 (closed)

  • Steve Lhomme added 13 commits

    added 13 commits

    • 51dbe085...7da6d4cf - 3 earlier commits
    • 166c70b7 - configure: fix libssp.a detection when LTO is enabled
    • c2468784 - contrib: ffmpeg: fix Windows builds with LLVM LTO
    • 1c5ab1c0 - contrib: add an option to enable Link Time Optimization
    • dc791dba - contrib: ffmpeg: enable Link Time Optimization using the official flag
    • cc43dad3 - contrib: qt: enable Link Time Optimization with the official flag
    • 502a8244 - configure: add an option to enable Link Time Optimization
    • b75d45c9 - package/win32: add -t option to enable Link Time Optimization
    • 0bae5fc1 - package/win32: warn when Link Time Optimization is not used in Release mode
    • 953a7505 - CI: enable LTO in windows LLVM nightlies
    • 018c1efd - TEST CI: enable LTO on regular Windows builds

    Compare with previous version

    • Author Developer

      Sad news, I did a release build for win64 with and without LTO to compare the sizes. My original finding was screwed because libplacebo_vk_plugin.dll goes from 35 MB to 3.3 MB. If you exclude that DLL the DLL size (excluding Qt) goes from 153 MB to 145 MB, with some DLLs actually growing.

      The LTO Qt DLL failed to build with LLVM16 (crash in LLDB). But the DLL size is 36 MB big. This is not too far from the 30 MB that @jbk wanted.

      I attach the list of each DLL with its size with and without LTO.

      without LTO / with LTO

      If we enable LTO only for select modules (ie the ones not growing) we may gain a little more. There's also still hope to make the Qt DLL a bit smaller, but we have to have the right toolchain for that.

    • Why sad news?

    • Author Developer

      Because I was expecting more than that. And also because a lot of DLLs are actually bigger. IMO if we want to use this we may want to select which plugins use it or not. That's more work (than a generic option) and requires testing this per platform...

      Also the fact I can't even link the Qt plugin on my machine is a problem. I should try with the latest LLVM.

    • Author Developer

      I tried with LLVM18, I managed to get a libqt_plugin.dll which was pretty much the same size as without LTO. But I messed up my build as it was using -O0. Without the -O0 (so default VLC optimization in release mode) the linker still crashes. So we can't tell for sure the code would be smaller, but probably not.

      However for the same set of DLLs we get from 153 MB to 158 MB! It's actually bigger. That's a comparison between LLVM16 without LTO and LLVM18 with LTO. Maybe the LLVM18 without LTO is even bigger. I may try that because that means for 4.0 release we might want to stick to LLVM16.

      LLVM18 with LTO

    • LTO is meant to improve selected optimisations. If we use O2 or O3 (as we should, IMO), the point is to make the code faster, not smaller.

      I'm all for enabling LTO if it works, but we need to do it gor the right reasons.

      If the problem is the Qt DLL size, then we have to split the DLL into multiple DLLs. We can't risk a situation where VLC is unbuildable due to insane linker memory or processor usage, or because the next minor version of Qt or the VLC GUI just went beyond some limit.

    • Author Developer

      If we want smaller builds we may just use -Os. There's still some dead code that seems to be trimmed with LTO that is not without. But maybe there's some other linking option that we should look for.

    • Author Developer

      The LLVM18 build without LTO is 157 MB, compared to 153 MB with LLVM16 and 162 MB with LTO. So we don't save that much with space, but we do. We could save more by picking which DLLs use LTO and which doesn't.

    • Please register or sign in to reply
  • Steve Lhomme added 8 commits

    added 8 commits

    • f0fb6b57 - contrib: add an option to enable Link Time Optimization
    • c94439da - contrib: ffmpeg: enable Link Time Optimization using the official flag
    • b70eb836 - contrib: qt: enable Link Time Optimization with the official flag
    • 951e15eb - configure: add an option to enable Link Time Optimization
    • 15ecd451 - package/win32: add -t option to enable Link Time Optimization
    • 74534cda - package/win32: warn when Link Time Optimization is not used in Release mode
    • 4b049eca - CI: enable LTO in windows LLVM nightlies
    • c1ff2598 - TEST CI: enable LTO on regular Windows builds

    Compare with previous version

  • Please register or sign in to reply
    Loading