download-rustc downloads rustc-dev even when it is not needed

[RalfJung]

The typical reason to download nightly rustc in ./x is to build the standard library. That only needs rustc + std. However, bootstrap currently also downloads rustc-dev. To minimize download time, mobile data usage, and disk space consumption, it'd be nice to avoid the download when it is not needed.

Cc @onur-ozkan

[onur-ozkan]

This will add a lot of complexity on bootstrap such as having different download, extract and copy logic depending on whether ci-rustc is enabled and whether it's running on a library or non-library step.

Also, you only download it once per HEAD so unless you frequently run git pull, it shouldn't significantly affect your network usage. I also wonder if there is any space for increasing the compression level (if it's not the highest one already) before uploading them.

[RalfJung]

Also, you only download it once per HEAD so unless you frequently run git pull, it shouldn't significantly affect your network usage

I have to pull at least when there is a conflict for one of my PRs... Which is pretty common. So there's hardly a train ride where I do not pull.

[jyn514]

@onur-ozkan i think this should be simpler to implement than you imagine. rustc-dev can have its own stamp file and directory, and we only add it to the sysroot if someone calls builder.ensure(RustcDev). (in fact I thought we did that already, but maybe I am misremembering).

@rustbot label A-download-rustc

[onur-ozkan]

The part I was concerned more was to update/replace current if download_rustc() on for non-tools and tools spots. I feel like it will be quite easy to use the wrong one instead of the other accidently.

[RalfJung]

So the build system does not currently know whether a build needs the rustc libs?

I think ideally, a build step needs to say "I need rustc libs". If it doesn't do that, the libs are not in the search path and hence the build fails. If it does do that, that informs the download_rustc check.