Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

cargo-chef seems to build the API twice #155

Closed
RemiBardon opened this issue Jan 22, 2025 · 2 comments · Fixed by #169
Closed

cargo-chef seems to build the API twice #155

RemiBardon opened this issue Jan 22, 2025 · 2 comments · Fixed by #169
Assignees
Labels
bug Something isn't working ci Changes / improvements to the CI

Comments

@RemiBardon
Copy link
Member

RemiBardon commented Jan 22, 2025

In 6ad0bed, I introduced cargo-chef to speed Docker builds up. It works by computing if dependencies have changed and leverages Docker's cache to avoid rebuilding them if they haven't changed (i.e. if the Cargo.lock and Cargo.toml files haven't changed, to simplify).

Because of how smoke tests are defined, we have to expose all of the API's code through a library, and have smoke tests use it. Because we have such a library, I decided to have main.rs depend on it, exactly as smoke tests do. However, looking at GitHub Actions logs, I spotted

#23 176.9    Compiling prose-pod-api v0.0.1 (/usr/src/prose-pod-api/crates/rest-api)
...
#23 DONE 179.3s

in the RUN cargo chef cook --profile="dev" --recipe-path recipe.json Docker layer

and

#29 153.2    Compiling prose-pod-api v0.7.0 (/usr/src/prose-pod-api/crates/rest-api)
...
#29 DONE 178.8s

in RUN cargo install --path crates/rest-api --bin prose-pod-api --profile="dev".

The binary is version 0.7.0, so 0.0.1 must definitely be the library being compiled. This means cargo-chef detects the whole API code (except main.rs) as a dependency! That's why every GitHub release build, even in debug mode, would take exactly 40 minutes, no matter if we changed the dependencies or not.


Even worse, it looks like some dependencies are computed twice 😰 See how much time it takes to build the library and the binary:

#23 [builder 3/6] RUN cargo chef cook --profile="dev" --recipe-path recipe.json
...
#23 170.8    Compiling axum-extra v0.10.0
#23 176.9    Compiling prose-pod-api v0.0.1 (/usr/src/prose-pod-api/crates/rest-api)
#23 178.6     Finished `dev` profile [unoptimized] target(s) in 2m 58s
#23 DONE 179.3s
#29 [builder 6/6] RUN cargo install --path crates/rest-api --bin prose-pod-api --profile="dev"
...
#29 105.2    Compiling axum-extra v0.10.0
#29 112.5    Compiling rustls-webpki v0.102.8
...
#29 137.2    Compiling service v0.7.0 (/usr/src/prose-pod-api/crates/service)
#29 153.2    Compiling prose-pod-api v0.7.0 (/usr/src/prose-pod-api/crates/rest-api)
#29 178.1     Finished `dev` profile [unoptimized] target(s) in 2m 57s
#29 178.3   Installing /usr/local/cargo/bin/prose-pod-api
#29 178.3    Installed package `prose-pod-api v0.7.0 (/usr/src/prose-pod-api/crates/rest-api)` (executable `prose-pod-api`)
#29 DONE 178.8s
...

(Also notice `axum-extra` taking ~6 seconds both times, for example)

---

Hopefully, if we stop depending on the `prose_pod_api` library in `main.rs`, it should fix the issue.

In addition, I will prevent `task release` from updating the dependencies in `patch` versions. It will lead to more cache reuses.
@RemiBardon RemiBardon added the bug Something isn't working label Jan 22, 2025
@RemiBardon RemiBardon self-assigned this Jan 22, 2025
@github-project-automation github-project-automation bot moved this to Backlog & Ideas 💡 in Prose Pod API to-do list Jan 22, 2025
@RemiBardon RemiBardon moved this from Backlog & Ideas 💡 to In Progress 🏗️ in Prose Pod API to-do list Jan 22, 2025
@RemiBardon
Copy link
Member Author

The issue won't disappear as easily because of our binary still depending on service, but I'll try to make cargo-chef ignore it (as long as we only have a REST API).

@RemiBardon RemiBardon moved this from In Progress 🏗️ to Backlog & Ideas 💡 in Prose Pod API to-do list Jan 23, 2025
@RemiBardon
Copy link
Member Author

RemiBardon commented Jan 24, 2025

Seems like LukeMathWalker/cargo-chef#273.

@RemiBardon RemiBardon changed the title Do not use the prose_pod_api lib in main.rs cargo-chef seems to build the API twice Jan 24, 2025
@RemiBardon RemiBardon moved this from Backlog & Ideas 💡 to Not Started 🕑 in Prose Pod API to-do list Jan 24, 2025
@RemiBardon RemiBardon moved this from Not Started 🕑 to In Progress 🏗️ in Prose Pod API to-do list Jan 24, 2025
@RemiBardon RemiBardon added the ci Changes / improvements to the CI label Jan 24, 2025
RemiBardon added a commit that referenced this issue Jan 24, 2025
RemiBardon added a commit that referenced this issue Jan 24, 2025
RemiBardon added a commit that referenced this issue Jan 24, 2025
RemiBardon added a commit that referenced this issue Jan 24, 2025
RemiBardon added a commit that referenced this issue Jan 25, 2025
RemiBardon added a commit that referenced this issue Jan 26, 2025
@RemiBardon RemiBardon linked a pull request Jan 26, 2025 that will close this issue
@github-project-automation github-project-automation bot moved this from In Progress 🏗️ to Done ✅ in Prose Pod API to-do list Jan 26, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working ci Changes / improvements to the CI
Projects
Status: Done ✅
Development

Successfully merging a pull request may close this issue.

1 participant