Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

CI: Decrease compilation times with Ccache #13111

Open
wants to merge 1 commit into
base: master
Choose a base branch
from

Conversation

HybridDog
Copy link
Contributor

@HybridDog HybridDog commented Jan 3, 2023

I've excluded the docker build since it would probably require a modified Dockerfile,
and Ccache did not work with macos and VS 2019 builds for some reason.

  • Goal of the PR: Speed up the CI
  • How does the PR work? It uses the github action cache together with Ccache to re-use previous compilations.
  • Does it resolve any reported issue? Yes, it resolves Github CI: Reduce compilation times with ccache #11667.
  • Does this relate to a goal in the roadmap? I don't think so.
  • If not a bug fix, why is this PR needed? What usecases does it solve? The usecases are already mentioned in the Issue.

How to test

  • Fork this branch
  • Enable github actions on the fork
  • Wait until the action for the forked branch completes
  • Do a change in, for example, the build.yml workflow file, create a commit and push to the forked branch
  • Wait until the action for the forked branch completes
  • Compare the times between the first and second github action runs

@TurkeyMcMac TurkeyMcMac added @ Build CMake, build scripts, official builds, compiler and linker errors Performance labels Jan 3, 2023
@TurkeyMcMac
Copy link
Contributor

Is it possible that the ccache action could be changed after this is added to introduce malware? If so, maybe we should fork the action under the minetest namespace.

@HybridDog
Copy link
Contributor Author

Is it possible that the ccache action could be changed after this is added to introduce malware?

Since the repository belongs to hendrikmuhs, I assume he/she is able to change the action with malicious intent.

@rubenwardy
Copy link
Contributor

rubenwardy commented Jan 6, 2023

Is it possible that the ccache action could be changed after this is added to introduce malware? If so, maybe we should fork the action under the minetest namespace.

HybridDog previously attempted to introduce malware into a mod, should definitely be checking this stuff

@AFCMS
Copy link
Contributor

AFCMS commented Feb 9, 2023

Is it possible that the ccache action could be changed after this is added to introduce malware? If so, maybe we should fork the action under the minetest namespace.

https://docs.github.com/en/actions/security-guides/security-hardening-for-github-actions#using-third-party-actions

You can pin action to a specific commit hash instead of a release to make sure it cannot be modified.

@rubenwardy rubenwardy added the Maintenance Tasks to keep the codebase and related parts in order, including architectural improvements label May 27, 2023
@numberZero
Copy link
Contributor

Looks good. What’s holding it?

@nerzhul
Copy link
Contributor

nerzhul commented Jun 23, 2023

im' not in favor of use random user github action. Cannot we fork it on our side ?

@erlehmann
Copy link
Contributor

@HybridDog given that ccache sometimes has false positves, how is the cache evicted here?

@HybridDog
Copy link
Contributor Author

given that ccache sometimes has false positves, how is the cache evicted here?

I haven't encountered false positives yet with ccache and I don't know in which situations they appear. I've configured the CI to use a separate cache per Job, as suggested there.
As far as I understand that page, with Pull Requests the CI can download the cache associated with the master branch but not override this cache.
It also mentions that the cache will be deleted if it hasn't been accessed in the last week.

@erlehmann
Copy link
Contributor

erlehmann commented Oct 22, 2023

given that ccache sometimes has false positves, how is the cache evicted here?

I haven't encountered false positives yet with ccache and I don't know in which situations they appear.

The elephant in the room is that a ccache false positive is hard to diagnose unless you have really good tests for that code path and the tests are not affected in the same way as the code is.

Here are some reasons for ccache false positves:

The Gentoo (a “compile everything” GNU/Linux distribution) wiki suggests disabling ccache before reporting a bug: https://wiki.gentoo.org/wiki/Handbook:Parts/Working/Features#Caching_compilation_objects

Reminder: I am not against use of ccache – I just wanted developers to have an easy way to manually evict the cache.

@sfan5 sfan5 added the Rebase needed The PR needs to be rebased by its author label Mar 20, 2024
@Zughy
Copy link
Contributor

Zughy commented Mar 21, 2024

@HybridDog rebase needed

I've excluded the docker build since it would probably require a modified Dockerfile.
@sfan5 sfan5 removed the Rebase needed The PR needs to be rebased by its author label Mar 24, 2024
@sfan5
Copy link
Collaborator

sfan5 commented Mar 24, 2024

Question: do we actually want to do this? My opinion on it is neutral.

@HybridDog
Copy link
Contributor Author

I think longer build times slightly increase the load on Microsoft's servers because GitHub is owned by Microsoft. I don't like Microsoft, so my opinion is neutral, too.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
@ Build CMake, build scripts, official builds, compiler and linker errors Maintenance Tasks to keep the codebase and related parts in order, including architectural improvements Performance
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Github CI: Reduce compilation times with ccache
10 participants