Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Implement proper caching in VM builder v2 #55

Open
wants to merge 5 commits into
base: dev
Choose a base branch
from

Conversation

aleasims
Copy link
Collaborator

@aleasims aleasims commented Feb 18, 2025

Changes

  • Added --cache-dir option to build subcommand. If this option is not set, the default path will be used. On Linux this most likely will be $HOME/.cache/gvltctl.
  • Moved Linux kernel cache from $HOME/.linux-builds to cache directory. Your $HOME/.linux-builds is no longer needed and can be safely removed. Note that this will mean that your kernel will be re-built next time you run the tool.
  • Implemented cache directory structure for Linux kernel. The key is repository URL in base64 + version. Kernel sources are stored in CACHE/linux-build/<url_base64_encoded>/<version> directory. If this directory exists, the builder assumes, that there is a proper compiled kernel in it. So it will not re-compile it. It is safe to remove the whole directory, but modifying its content will have no effect.
  • Implemented NVIDIA drivers cache. The key is driver version + kernel release. Dumped driver is stored in CACHE/nvidia/<driver_version>/<kernel_release> (driver version is hardcoded right now, because we only use one single version).
  • Fixed caching of base image file. Now we checksum base image and compare it to the one stored in cache. Base image file is stored in CACHE/<crc32_of_content>.base.img.
  • Implemented caching of MIA. MIA will be downloaded into cache only the first time. The key is platform + version (platform is always x86_64-unknown-linux-gnu). Downloaded binary is stored in CACHE/mia/<platform>/mia-<version>. If --mia-version latest is used, it will be resolved to numeric before downloading and stored under numeric key. This way cache entry won't be outdated, if new MIA release will come.

Notes

  • With NVIDIA drivers caching my build time reduced by ~40 seconds (on 24 threads). If everything else is cached it was around 60% of the time. The most time-consuming step now is encoding SquashFS.
  • Kernel cache works poorly if used latest version. It will cache the revision which was "latest" at the moment of cloning and won't update it. Probably it's no a huge deal.
  • When using cached MIA with --mia-version latest (which is used by default), there still will be a network request to check, which version is actually latest at the moment. It means that VM building still requires internet connection.

@aleasims aleasims self-assigned this Feb 18, 2025
Use checksum as filename to avoid cache invalidation
@aleasims aleasims marked this pull request as ready for review February 20, 2025 21:36
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant