Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feature Idea: Git diff/changed based dependency resolution #1114

Open
DanCardin opened this issue Feb 23, 2022 · 4 comments
Open

Feature Idea: Git diff/changed based dependency resolution #1114

DanCardin opened this issue Feb 23, 2022 · 4 comments

Comments

@DanCardin
Copy link

The point of this idea is to define git/source-control change based dependencies on tasks such that a task which depends on them will no-op if nothing has changed. At it's most basic, if i say task "foo" depends on src/.*, and git diff --name-only | grep 'src/.*' doesn't yield anything then "foo" should be considered done without executing.

Imagine a folder structure

justfile
subproject1/
subproject2/
subproject1and2/  # depends on 1 and 2
Dockerfile

and justfile (syntax just off the top of my head)

build-subproject1: git:src/* git:Cargo.toml
    cd subproject1  # I haven't yet checked whether `cd` works like this in just.
    cargo build

build-subproject2: build-subproject1 git:src/* git:Cargo.toml
    cd subproject2
    cargo build

build-docker: Dockerfile git:subproject1/src/*
    ...

build: build-subproject1 build-subproject2 build-docker

The basic idea is that (even locally, but especially in CI), if I make/just build, you really only want to have to build any individual subproject if src/* for that project has changed. With build-subproject1: git:src/* git:Cargo.toml you're saying "its only possible for the result of build-subproject1 to change and need to be re-run if src/* changed or if Cargo.toml changed.

Perhaps rust subprojects aren't the best examples, I personally conceived of the idea in a mono-ish repo set of python sub-projects which have much more obvious dependence on on-disk file changes.

The Dockerfile example is another typical one. If you specify all the things you depend upon in the Dockerfile, then you could reliably avoid multi-minute docker build/push jobs in CI where you may not have proper image caches ready.

Or even in this (just) project, you update the README, presumably your CI will then waste a bunch of credits spending time building, testing, etc on jobs that cannot possibly produce different results (at least as far as your PR changeset is concerned).

Long story short, I recently got far enough through prototyping my own make-esque-rust-tool to prove to myself that the idea would work; before i realized how much effort it would actually take for me to realistically replace my current-day uses of make.
Then just just popped up on my reddit feed and i figured I'd at least write the idea down somewhere it might possibly get turned into a feature and save me a bunch of effort :P

@endigma
Copy link

endigma commented Apr 21, 2023

go-task has this, for a real example case

@casey
Copy link
Owner

casey commented Jun 15, 2024

I think this will be covered by #1906.

@casey casey closed this as completed Jun 15, 2024
@casey
Copy link
Owner

casey commented Jun 15, 2024

Actually, reopening until it's actually implemented.

@casey casey reopened this Jun 15, 2024
@DanCardin
Copy link
Author

It's not entirely obvious to me how that feature would solve this issue, in and of itself. Although it does seem like the way this would get implemented. One would have to manually insert @# {{ git diff --name-only | grep 'src/.*' }} (per my example) into each task?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants