Tracking Issue for cargo-script RFC 3424 #12207

ehuss · 2023-05-31T03:55:52Z

feat: Initial support for single-file packages ### What does this PR try to resolve? This is the first step towards #12207. In particular, this focuses on pulling in the [demo](https://github.com/epage/cargo-script-mvs) roughly as-is to serve as a baseline for further PRs. I have a couple months of runtime (multiple times a week) using the version of the demo included here. ### How should we test and review this PR? Commit-by-commit. Most likely, the last (docs) commit should be done first to provide context for the others. Naming is hard. I came up with these terms just so we can have ways to refer to them. Feedback is welcome. - `-Zscript` for this general feature (not great but didn't want to spend too long coming up with a throwaway name) - "single-file package": Rust code and a cargo manifest in a single file - "embedded manifest": the explicit manifest inside of a single-file package - "manifest command": when we interpret `cargo <name>` as referring to a single-file package, and similar to "built-in commands" and "external commands". Keep in mind that this is a very hacky solution with many deficiencies and is mostly starting as a baseline for implementing and reviewing those improvements, including - Switching from `regex` to `syn` for extracting manifests for greater resilience - Matching `cargo new`s logic when sanitizing the inferred package name - Allowing `cargo <name>` to also be a `Cargo.toml` file (for consistency in where manifests are accepted) - Allowing single-file packages to be used wherever a manifest is accepted To minimize conflict pain, I would ask that we consider what feedback can be handled in a follow up (though still mention it!). It'll be much easier creating multiple, independent PRs once this baseline is merged to address concerns. ### Additional information The only affect for people on stable is that they may get a warning if they have an external subcommand that will be shadowed when this feature is implemented. This will allow us to collect feedback, without blocking people, so we can have an idea of how "safe" our precedence scheme is for interpreting `cargo <name>`. As of right now, aliases with a `.` in them will be ignored (well, technically their suffix will be exposed as an alias). We directly inject the name into a lookup string into the config that uses `.` as the separator, so we drill down until we get to the leaf element. Ideally, we would be generating / parsing the lookup key using TOML key syntax so we can better report that this won't be supported after this change :)

fix(embedded): Align package name sanitization with cargo-new ### What does this PR try to resolve? This is a follow up to #12245 which is working to resolve the tracking issue #12207 This first aligns sanitization of package names with the central package name validation logic, putting the code next to each other so they can more easily stay in sync. Oddly enough, cargo-new is stricter than our normal package-name validation. I went ahead and sanitized along with that as well. In working on this, I was bothered by - the mix of `-` and `_` in file names because of sanitization, so I made it more consistent by detecting which the user is using - -using `_` in bins, so I switched the default to `-` ### How should we test and review this PR? One existing test covers a variety of sanitization needs Another existing test hit one of the other cases (`test` being reserved) ### Additional information For implementation convenience, I changed the directory we write the manifest to. The impact of this should be minimal since - We reuse the full file name, so if it worked for the user it should work for us - We should be moving away from the temp manifest in future commits

refactor(embedded): Switch to `syn` for parsing doc comments This is a follow up to #12245 which is working to resolve #12207 The hope is this will result in more resilient comment handling, being more consistent with rustdoc. I also hoped for less code but `syn` is doing less than I had expected, requiring us to copy code over from other parts of rust. It seems every proc macro has to do this but there is no guide to it, so they all do it differently, only covering the cases they thought to test for. Note that this still won't support `include_str!()`.

fix(embeded): Don't pollute the scripts dir with `target/` ### What does this PR try to resolve? This PR is part of #12207. This specific behavior was broken in #12268 when we stopped using an intermediate `Cargo.toml` file. Unlike pre-#12268, - We are hashing the path, rather than the content, with the assumption that people change content more frequently than the path - We are using a simpler hash than `blake3` in the hopes that we can get away with it Unlike the Pre-RFC demo - We are not forcing a single target dir for all scripts in the hopes that we get #5931 ### How should we test and review this PR? A new test was added specifically to show the target dir behavior, rather than overloading an existing test or making all tests sensitive to changes in this behavior. ### Additional information In the future, we might want to resolve symlinks before we get to this point

Rustin170506 · 2024-10-31T15:01:13Z

This will be messy. We'll likely need to change SourceId to include the file name if it has an embedded manifest.

I will give this one a try.

This avoids syntax errors until Rust supports the syntax in its tooling: rust-lang/cargo#12207 (comment)

Rustin170506 · 2024-11-11T14:20:58Z

This will be messy. We'll likely need to change SourceId to include the file name if it has an embedded manifest.

Not sure if this is the right place to discuss implementation details, but I wanted to ask if the output makes sense.

$pwd
/Users/rustin/code/rs-scripts

$ls
sql.rs  sql1.rs sql2.rs sql3.rs sql4.rs sql5.rs sql6.rs sql7.rs sql8.rs sql9.rs

$cargo pkgid sql2.rs
path+file:///Users/rustin/code/rs-scripts/sql2.rs#embedded

epage · 2024-11-11T14:40:55Z

@Rustin170506 I feel like there might be enough discussion on that that maybe we should create a dedicated issue?

### What does this PR try to resolve? We removed rejected syntax in #13861 haven't update frontmatter parsing to what [RFC 3503](https://rust-lang.github.io/rfcs/3503-frontmatter.html#reference-level-explanation) says. This adds tests and fixes various cases to ensure we correctly detect the frontmatter and the infostring. This is part of #12207 ### How should we test and review this PR? ### Additional information

### What does this PR try to resolve? This adds support for cargo script for more cargo commands and is a part of #12207 Unfortunately, there is a double-warning for unspecified editions for `cargo remove`. Rather than addressing that here, I'll be noting it in the tracking issue. ### How should we test and review this PR? ### Additional information

At this point it is unlikely for a script to be written for pre-2024 edition and migrated to 2024+ but this code is independent of Editions so this means we have one less bug and are better prepared for the next edition. When we add `cargo fix` support for regular manifest warnings, we'll need to take into account cargo scripts. This is a part of rust-lang#12207.

### What does this PR try to resolve? At this point it is unlikely for a script to be written for pre-2024 edition and migrated to 2024+ but this code is independent of Editions so this means we have one less bug and are better prepared for the next edition. When we add `cargo fix` support for regular manifest warnings, we'll need to take into account cargo scripts. This is a part of #12207. ### How should we test and review this PR? ### Additional information While looking for where the tests go, I found a couple places with a hard coded latest-edition in test output and fixed it.

This has been around since rust-lang#12224 and isn't in the RFC, so its being removed. This is part of rust-lang#12207.

This has been around since #12224 and isn't in the RFC, so its being removed. This is part of #12207.

epage · 2025-02-04T19:56:56Z

#14961 added a test, pointing out how cargo install --path foo.rs behaviors (errors). I've added support for this as an open question. I lean towards supporting it as it seems relatively innocuous and our overall goal is "behaves like a regular package" with cases like workspace membership, packaging support, etc being explicitly deferred out.

@Rustin170506 since you were just dealing with that stuff, would be interested in your thoughts and if you want to pick that work up.

epage · 2025-02-10T21:59:00Z

Starting the conversation on the rustc side of this: https://rust-lang.zulipchat.com/#narrow/channel/182449-t-compiler.2Fhelp/topic/Pointers.20for.20implementing.20cargo.20scrpt.20support/near/498872327

…5168) ### What does this PR try to resolve? This is a part of #12207 Now that the experiment is over and we can be intrusive, this merges all of the permanent cargo-script manifest logic into the regular manifest loading code path with the goal of reducing the number of places people need to be aware to check to know how things work so we're less likely to overlook them (e.g. `package.name`s being optional). This should also make error reporting better as we will eventually only use the real manifest users specify, instead of one with a lot of edits by us. This comes at the cost of some uglier code needed to check some of these cases. ### How should we test and review this PR? ### Additional information

### What does this PR try to resolve? This is part of #12207. I found these while implementing frontmatter support within rustc. I'll likely do another pass when I finish rustc support to - Unify tests between cargo and rustc - Improve error messages ### How should we test and review this PR? ### Additional information

ehuss added S-accepted Status: Issue or feature is accepted, and has a team member available to help mentor or review C-tracking-issue Category: A tracking issue for something unstable. Z-script Nightly: cargo script labels May 31, 2023

ehuss mentioned this issue May 31, 2023

eRFC: single-file packages ("cargo script") integration rust-lang/rfcs#3424

Merged