Skip to content

Commit 9090042

Browse files
borsbors
committed
Auto merge of #10383 - dtolnay-contrib:keepgoing, r=ehuss
Unstable --keep-going flag ## Summary This PR adds an unstable `--keep-going` flag documented as follows: > `cargo build --keep-going` (and similarly for `check`, `test` etc) will build as many crates in the dependency graph as possible, rather than aborting the build at the first one that fails to build. > > For example if the current package depends on dependencies `fails` and `works`, one of which fails to build, `cargo check -j1` may or may not build the one that succeeds (depending on which one of the two builds Cargo picked to run first), whereas `cargo check -j1 --keep-going` would definitely run both builds, even if the one run first fails. > > The `-Z unstable-options` command-line option must be used in order to use `--keep-going` while it is not yet stable: > > ```console > cargo check --keep-going -Z unstable-options > ``` ## Prior art [Buck](https://buck.build/) and [Bazel](https://bazel.build/) and Make all have this flag (though Bazel calls it `--keep_going` 🤮) with exactly this behavior. ## Motivation I need this in order to make https://github.com/dtolnay/trybuild not super slow. Trybuild wants to run Cargo on a bunch of test cases, each of which is a bin crate. The bad options currently available are: - Give each test case its own target dir and run build on them in parallel. This is bad because all the test cases have the same dependencies in common (whatever `dev-dependencies` are declared by the project). If there are 100 test cases, all the dependencies would end up getting built 100 times, which is 100x slower than necessary despite the parallelism. - Reuse a single target dir for all the test cases. Two Cargos can't operate in parallel on the same target directory, so this forces the test cases to be built serially. This is much slower than necessary on a many-core system, and compounds all of the overheads in Cargo because the project structure must be reloaded by each invocation. The good option I'd like to switch to is: - Run `cargo build --bins --keep-going --message-format=json` to build *all* the test cases in parallel. Use the filepaths in the JSON messages to ascribe diagnostics to which bin they're from.
2 parents b1636fc + 4e45f58 commit 9090042

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

62 files changed

+337
-21
lines changed

‎src/bin/cargo/commands/package.rs

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -53,6 +53,7 @@ pub fn exec(config: &mut Config, args: &ArgMatches) -> CliResult {
5353
to_package: specs,
5454
targets: args.targets(),
5555
jobs: args.jobs()?,
56+
keep_going: args.keep_going(),
5657
cli_features: args.cli_features()?,
5758
},
5859
)?;

‎src/bin/cargo/commands/publish.rs

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -45,6 +45,7 @@ pub fn exec(config: &mut Config, args: &ArgMatches) -> CliResult {
4545
to_publish: args.packages_from_flags()?,
4646
targets: args.targets(),
4747
jobs: args.jobs()?,
48+
keep_going: args.keep_going(),
4849
dry_run: args.is_present("dry-run"),
4950
registry,
5051
cli_features: args.cli_features()?,

‎src/cargo/core/compiler/build_config.rs

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -15,6 +15,8 @@ pub struct BuildConfig {
1515
pub requested_kinds: Vec<CompileKind>,
1616
/// Number of rustc jobs to run in parallel.
1717
pub jobs: u32,
18+
/// Do not abort the build as soon as there is an error.
19+
pub keep_going: bool,
1820
/// Build profile
1921
pub requested_profile: InternedString,
2022
/// The mode we are compiling in.
@@ -56,6 +58,7 @@ impl BuildConfig {
5658
pub fn new(
5759
config: &Config,
5860
jobs: Option<u32>,
61+
keep_going: bool,
5962
requested_targets: &[String],
6063
mode: CompileMode,
6164
) -> CargoResult<BuildConfig> {
@@ -84,6 +87,7 @@ impl BuildConfig {
8487
Ok(BuildConfig {
8588
requested_kinds,
8689
jobs,
90+
keep_going,
8791
requested_profile: InternedString::new("dev"),
8892
mode,
8993
message_format: MessageFormat::Human,

‎src/cargo/core/compiler/job_queue.rs

Lines changed: 3 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -825,13 +825,14 @@ impl<'cfg> DrainState<'cfg> {
825825
//
826826
// After a job has finished we update our internal state if it was
827827
// successful and otherwise wait for pending work to finish if it failed
828-
// and then immediately return.
828+
// and then immediately return (or keep going, if requested by the build
829+
// config).
829830
let mut errors = ErrorsDuringDrain { count: 0 };
830831
// CAUTION! Do not use `?` or break out of the loop early. Every error
831832
// must be handled in such a way that the loop is still allowed to
832833
// drain event messages.
833834
loop {
834-
if errors.count == 0 {
835+
if errors.count == 0 || cx.bcx.build_config.keep_going {
835836
if let Err(e) = self.spawn_work_if_possible(cx, jobserver_helper, scope) {
836837
self.handle_error(&mut cx.bcx.config.shell(), &mut errors, e);
837838
}

‎src/cargo/ops/cargo_compile.rs

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -85,8 +85,10 @@ pub struct CompileOptions {
8585

8686
impl CompileOptions {
8787
pub fn new(config: &Config, mode: CompileMode) -> CargoResult<CompileOptions> {
88+
let jobs = None;
89+
let keep_going = false;
8890
Ok(CompileOptions {
89-
build_config: BuildConfig::new(config, None, &[], mode)?,
91+
build_config: BuildConfig::new(config, jobs, keep_going, &[], mode)?,
9092
cli_features: CliFeatures::new_all(false),
9193
spec: ops::Packages::Packages(Vec::new()),
9294
filter: CompileFilter::Default {

‎src/cargo/ops/cargo_fetch.rs

Lines changed: 8 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -20,8 +20,15 @@ pub fn fetch<'a>(
2020
let (packages, resolve) = ops::resolve_ws(ws)?;
2121

2222
let jobs = Some(1);
23+
let keep_going = false;
2324
let config = ws.config();
24-
let build_config = BuildConfig::new(config, jobs, &options.targets, CompileMode::Build)?;
25+
let build_config = BuildConfig::new(
26+
config,
27+
jobs,
28+
keep_going,
29+
&options.targets,
30+
CompileMode::Build,
31+
)?;
2532
let data = RustcTargetData::new(ws, &build_config.requested_kinds)?;
2633
let mut fetched_packages = HashSet::new();
2734
let mut deps_to_fetch = ws.members().map(|p| p.package_id()).collect::<Vec<_>>();

‎src/cargo/ops/cargo_package.rs

Lines changed: 9 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -30,6 +30,7 @@ pub struct PackageOpts<'cfg> {
3030
pub allow_dirty: bool,
3131
pub verify: bool,
3232
pub jobs: Option<u32>,
33+
pub keep_going: bool,
3334
pub to_package: ops::Packages,
3435
pub targets: Vec<String>,
3536
pub cli_features: CliFeatures,
@@ -177,6 +178,7 @@ pub fn package(ws: &Workspace<'_>, opts: &PackageOpts<'_>) -> CargoResult<Option
177178
allow_dirty: opts.allow_dirty,
178179
verify: opts.verify,
179180
jobs: opts.jobs,
181+
keep_going: opts.keep_going,
180182
to_package: ops::Packages::Default,
181183
targets: opts.targets.clone(),
182184
cli_features: cli_features,
@@ -755,7 +757,13 @@ fn run_verify(
755757
ops::compile_with_exec(
756758
&ws,
757759
&ops::CompileOptions {
758-
build_config: BuildConfig::new(config, opts.jobs, &opts.targets, CompileMode::Build)?,
760+
build_config: BuildConfig::new(
761+
config,
762+
opts.jobs,
763+
opts.keep_going,
764+
&opts.targets,
765+
CompileMode::Build,
766+
)?,
759767
cli_features: opts.cli_features.clone(),
760768
spec: ops::Packages::Packages(Vec::new()),
761769
filter: ops::CompileFilter::Default {

‎src/cargo/ops/registry.rs

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -80,6 +80,7 @@ pub struct PublishOpts<'cfg> {
8080
pub verify: bool,
8181
pub allow_dirty: bool,
8282
pub jobs: Option<u32>,
83+
pub keep_going: bool,
8384
pub to_publish: ops::Packages,
8485
pub targets: Vec<String>,
8586
pub dry_run: bool,
@@ -147,6 +148,7 @@ pub fn publish(ws: &Workspace<'_>, opts: &PublishOpts<'_>) -> CargoResult<()> {
147148
to_package: ops::Packages::Default,
148149
targets: opts.targets.clone(),
149150
jobs: opts.jobs,
151+
keep_going: opts.keep_going,
150152
cli_features: cli_features,
151153
},
152154
)?

‎src/cargo/util/command_prelude.rs

Lines changed: 20 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -71,6 +71,10 @@ pub trait AppExt: Sized {
7171
.short('j')
7272
.value_name("N"),
7373
)
74+
._arg(opt(
75+
"keep-going",
76+
"Do not abort the build as soon as there is an error (unstable)",
77+
))
7478
}
7579

7680
fn arg_targets_all(
@@ -353,6 +357,10 @@ pub trait ArgMatchesExt {
353357
self.value_of_u32("jobs")
354358
}
355359

360+
fn keep_going(&self) -> bool {
361+
self._is_present("keep-going")
362+
}
363+
356364
fn targets(&self) -> Vec<String> {
357365
self._values_of("target")
358366
}
@@ -506,7 +514,13 @@ pub trait ArgMatchesExt {
506514
}
507515
}
508516

509-
let mut build_config = BuildConfig::new(config, self.jobs()?, &self.targets(), mode)?;
517+
let mut build_config = BuildConfig::new(
518+
config,
519+
self.jobs()?,
520+
self.keep_going(),
521+
&self.targets(),
522+
mode,
523+
)?;
510524
build_config.message_format = message_format.unwrap_or(MessageFormat::Human);
511525
build_config.requested_profile = self.get_profile_name(config, "dev", profile_checking)?;
512526
build_config.build_plan = self.is_valid_and_present("build-plan");
@@ -540,6 +554,11 @@ pub trait ArgMatchesExt {
540554
}
541555
}
542556

557+
if build_config.keep_going {
558+
config
559+
.cli_unstable()
560+
.fail_if_stable_opt("--keep-going", 10496)?;
561+
}
543562
if build_config.build_plan {
544563
config
545564
.cli_unstable()

‎src/doc/man/cargo-bench.md

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -139,6 +139,7 @@ Rust test harness runs benchmarks serially in a single thread.
139139

140140
{{#options}}
141141
{{> options-jobs }}
142+
{{> options-keep-going }}
142143
{{/options}}
143144

144145
{{> section-environment }}

‎src/doc/man/cargo-build.md

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -91,6 +91,7 @@ See <https://github.com/rust-lang/cargo/issues/5579> for more information.
9191

9292
{{#options}}
9393
{{> options-jobs }}
94+
{{> options-keep-going }}
9495
{{> options-future-incompat }}
9596
{{/options}}
9697

‎src/doc/man/cargo-check.md

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -76,6 +76,7 @@ they have `required-features` that are missing.
7676

7777
{{#options}}
7878
{{> options-jobs }}
79+
{{> options-keep-going }}
7980
{{> options-future-incompat }}
8081
{{/options}}
8182

‎src/doc/man/cargo-doc.md

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -110,6 +110,7 @@ and supports common Unix glob patterns.
110110

111111
{{#options}}
112112
{{> options-jobs }}
113+
{{> options-keep-going }}
113114
{{/options}}
114115

115116
{{> section-environment }}

‎src/doc/man/cargo-fix.md

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -156,6 +156,7 @@ When no target selection options are given, `cargo fix` will fix all targets
156156

157157
{{#options}}
158158
{{> options-jobs }}
159+
{{> options-keep-going }}
159160
{{/options}}
160161

161162
{{> section-environment }}

‎src/doc/man/cargo-install.md

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -181,6 +181,7 @@ See also the `--profile` option for choosing a specific profile by name.
181181

182182
{{#options}}
183183
{{> options-jobs }}
184+
{{> options-keep-going }}
184185
{{/options}}
185186

186187
### Display Options

‎src/doc/man/cargo-package.md

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -112,6 +112,7 @@ Allow working directories with uncommitted VCS changes to be packaged.
112112

113113
{{#options}}
114114
{{> options-jobs }}
115+
{{> options-keep-going }}
115116
{{/options}}
116117

117118
### Display Options

‎src/doc/man/cargo-publish.md

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -91,6 +91,7 @@ which defaults to `crates-io`.
9191

9292
{{#options}}
9393
{{> options-jobs }}
94+
{{> options-keep-going }}
9495
{{/options}}
9596

9697
### Display Options

‎src/doc/man/cargo-run.md

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -90,6 +90,7 @@ Run the specified example.
9090

9191
{{#options}}
9292
{{> options-jobs }}
93+
{{> options-keep-going }}
9394
{{/options}}
9495

9596
{{> section-environment }}

‎src/doc/man/cargo-rustc.md

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -102,6 +102,7 @@ See the [the reference](../reference/profiles.html) for more details on profiles
102102

103103
{{#options}}
104104
{{> options-jobs }}
105+
{{> options-keep-going }}
105106
{{> options-future-incompat }}
106107
{{/options}}
107108

‎src/doc/man/cargo-rustdoc.md

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -98,6 +98,7 @@ if its name is the same as the lib target. Binaries are skipped if they have
9898

9999
{{#options}}
100100
{{> options-jobs }}
101+
{{> options-keep-going }}
101102
{{/options}}
102103

103104
{{> section-environment }}

‎src/doc/man/cargo-test.md

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -155,6 +155,7 @@ includes an option to control the number of threads used:
155155
{{#options}}
156156

157157
{{> options-jobs }}
158+
{{> options-keep-going }}
158159
{{> options-future-incompat }}
159160

160161
{{/options}}

‎src/doc/man/generated_txt/cargo-bench.txt

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -361,6 +361,11 @@ OPTIONS
361361
<https://doc.rust-lang.org/cargo/reference/config.html>. Defaults to
362362
the number of CPUs.
363363

364+
--keep-going
365+
Build as many crates in the dependency graph as possible, rather
366+
than aborting the build on the first one that fails to build.
367+
Unstable, requires -Zunstable-options.
368+
364369
ENVIRONMENT
365370
See the reference
366371
<https://doc.rust-lang.org/cargo/reference/environment-variables.html>

‎src/doc/man/generated_txt/cargo-build.txt

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -307,6 +307,11 @@ OPTIONS
307307
<https://doc.rust-lang.org/cargo/reference/config.html>. Defaults to
308308
the number of CPUs.
309309

310+
--keep-going
311+
Build as many crates in the dependency graph as possible, rather
312+
than aborting the build on the first one that fails to build.
313+
Unstable, requires -Zunstable-options.
314+
310315
--future-incompat-report
311316
Displays a future-incompat report for any future-incompatible
312317
warnings produced during execution of this command

‎src/doc/man/generated_txt/cargo-check.txt

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -301,6 +301,11 @@ OPTIONS
301301
<https://doc.rust-lang.org/cargo/reference/config.html>. Defaults to
302302
the number of CPUs.
303303

304+
--keep-going
305+
Build as many crates in the dependency graph as possible, rather
306+
than aborting the build on the first one that fails to build.
307+
Unstable, requires -Zunstable-options.
308+
304309
--future-incompat-report
305310
Displays a future-incompat report for any future-incompatible
306311
warnings produced during execution of this command

‎src/doc/man/generated_txt/cargo-doc.txt

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -272,6 +272,11 @@ OPTIONS
272272
<https://doc.rust-lang.org/cargo/reference/config.html>. Defaults to
273273
the number of CPUs.
274274

275+
--keep-going
276+
Build as many crates in the dependency graph as possible, rather
277+
than aborting the build on the first one that fails to build.
278+
Unstable, requires -Zunstable-options.
279+
275280
ENVIRONMENT
276281
See the reference
277282
<https://doc.rust-lang.org/cargo/reference/environment-variables.html>

‎src/doc/man/generated_txt/cargo-fix.txt

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -374,6 +374,11 @@ OPTIONS
374374
<https://doc.rust-lang.org/cargo/reference/config.html>. Defaults to
375375
the number of CPUs.
376376

377+
--keep-going
378+
Build as many crates in the dependency graph as possible, rather
379+
than aborting the build on the first one that fails to build.
380+
Unstable, requires -Zunstable-options.
381+
377382
ENVIRONMENT
378383
See the reference
379384
<https://doc.rust-lang.org/cargo/reference/environment-variables.html>

‎src/doc/man/generated_txt/cargo-install.txt

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -257,6 +257,11 @@ OPTIONS
257257
<https://doc.rust-lang.org/cargo/reference/config.html>. Defaults to
258258
the number of CPUs.
259259

260+
--keep-going
261+
Build as many crates in the dependency graph as possible, rather
262+
than aborting the build on the first one that fails to build.
263+
Unstable, requires -Zunstable-options.
264+
260265
Display Options
261266
-v, --verbose
262267
Use verbose output. May be specified twice for "very verbose" output

‎src/doc/man/generated_txt/cargo-package.txt

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -189,6 +189,11 @@ OPTIONS
189189
<https://doc.rust-lang.org/cargo/reference/config.html>. Defaults to
190190
the number of CPUs.
191191

192+
--keep-going
193+
Build as many crates in the dependency graph as possible, rather
194+
than aborting the build on the first one that fails to build.
195+
Unstable, requires -Zunstable-options.
196+
192197
Display Options
193198
-v, --verbose
194199
Use verbose output. May be specified twice for "very verbose" output

‎src/doc/man/generated_txt/cargo-publish.txt

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -156,6 +156,11 @@ OPTIONS
156156
<https://doc.rust-lang.org/cargo/reference/config.html>. Defaults to
157157
the number of CPUs.
158158

159+
--keep-going
160+
Build as many crates in the dependency graph as possible, rather
161+
than aborting the build on the first one that fails to build.
162+
Unstable, requires -Zunstable-options.
163+
159164
Display Options
160165
-v, --verbose
161166
Use verbose output. May be specified twice for "very verbose" output

‎src/doc/man/generated_txt/cargo-run.txt

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -217,6 +217,11 @@ OPTIONS
217217
<https://doc.rust-lang.org/cargo/reference/config.html>. Defaults to
218218
the number of CPUs.
219219

220+
--keep-going
221+
Build as many crates in the dependency graph as possible, rather
222+
than aborting the build on the first one that fails to build.
223+
Unstable, requires -Zunstable-options.
224+
220225
ENVIRONMENT
221226
See the reference
222227
<https://doc.rust-lang.org/cargo/reference/environment-variables.html>

‎src/doc/man/generated_txt/cargo-rustc.txt

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -296,6 +296,11 @@ OPTIONS
296296
<https://doc.rust-lang.org/cargo/reference/config.html>. Defaults to
297297
the number of CPUs.
298298

299+
--keep-going
300+
Build as many crates in the dependency graph as possible, rather
301+
than aborting the build on the first one that fails to build.
302+
Unstable, requires -Zunstable-options.
303+
299304
--future-incompat-report
300305
Displays a future-incompat report for any future-incompatible
301306
warnings produced during execution of this command

0 commit comments

Comments
 (0)