"Failed to parse manifest" on bootloader-0.10.0-alpha-03

[jyn514]

Not sure what's going wrong here, cargo doc works locally. The crate doesn't show up in the web UI at all.

Feb 21 12:36:47 docsrs cratesfyi[28601]: 2021/02/21 12:36:47 [ERROR] docs_rs::build_queue: Failed to build package bootloader-0.10.0-alpha-03 from queue: failed to parse maniest
Feb 21 12:36:47 docsrs cratesfyi[28601]: Backtrace:    0: failure::backtrace::internal::InternalBacktrace::new
Feb 21 12:36:47 docsrs cratesfyi[28601]:    1: <failure::backtrace::Backtrace as core::default::Default>::default
Feb 21 12:36:47 docsrs cratesfyi[28601]:    2: rustwide::build::BuildBuilder::run
Feb 21 12:36:47 docsrs cratesfyi[28601]:    3: docs_rs::docbuilder::rustwide_builder::RustwideBuilder::build_package
Feb 21 12:36:47 docsrs cratesfyi[28601]:    4: docs_rs::build_queue::BuildQueue::process_next_crate
Feb 21 12:36:47 docsrs cratesfyi[28601]:    5: <std::panic::AssertUnwindSafe<F> as core::ops::function::FnOnce<()>>::call_once
Feb 21 12:36:47 docsrs cratesfyi[28601]:    6: docs_rs::utils::queue_builder::queue_builder
Feb 21 12:36:47 docsrs cratesfyi[28601]:    7: std::sys_common::backtrace::__rust_begin_short_backtrace
Feb 21 12:36:47 docsrs cratesfyi[28601]:    8: core::ops::function::FnOnce::call_once{{vtable.shim}}
Feb 21 12:36:47 docsrs cratesfyi[28601]:    9: <alloc::boxed::Box<F> as core::ops::function::FnOnce<A>>::call_once
Feb 21 12:36:47 docsrs cratesfyi[28601]:              at /rustc/d3fb005a39e62501b8b0b356166e515ae24e2e54/src/liballoc/boxed.rs:1076
Feb 21 12:36:47 docsrs cratesfyi[28601]:       <alloc::boxed::Box<F> as core::ops::function::FnOnce<A>>::call_once
Feb 21 12:36:47 docsrs cratesfyi[28601]:              at /rustc/d3fb005a39e62501b8b0b356166e515ae24e2e54/src/liballoc/boxed.rs:1076
Feb 21 12:36:47 docsrs cratesfyi[28601]:       std::sys::unix::thread::Thread::new::thread_start
Feb 21 12:36:47 docsrs cratesfyi[28601]:              at /rustc/d3fb005a39e62501b8b0b356166e515ae24e2e54/src/libstd/sys/unix/thread.rs:87
Feb 21 12:36:47 docsrs cratesfyi[28601]:   10: start_thread
Feb 21 12:36:47 docsrs cratesfyi[28601]:   11: __clone

edit: this happens because docsrs_metadata is parsing Cargo.toml.orig when it should be parsing Cargo.toml.

[jyn514]

This is coming from the docsrs_metadata crate:

Error: Building documentation failed

Caused by:
    failed to parse manifest

Stack backtrace:
   0: failure::backtrace::internal::InternalBacktrace::new
             at /home/joshua/.local/lib/cargo/registry/src/github.com-1ecc6299db9ec823/failure-0.1.8/src/backtrace/internal.rs:46
   1: failure::backtrace::Backtrace::new
             at /home/joshua/.local/lib/cargo/registry/src/github.com-1ecc6299db9ec823/failure-0.1.8/src/backtrace/mod.rs:121
   2: <failure::error::error_impl::ErrorImpl as core::convert::From<F>>::from
             at /home/joshua/.local/lib/cargo/registry/src/github.com-1ecc6299db9ec823/failure-0.1.8/src/error/error_impl.rs:19
   3: <failure::error::Error as core::convert::From<F>>::from
             at /home/joshua/.local/lib/cargo/registry/src/github.com-1ecc6299db9ec823/failure-0.1.8/src/error/mod.rs:36
   4: docs_rs::docbuilder::rustwide_builder::RustwideBuilder::build_package::{{closure}}
             at src/docbuilder/rustwide_builder.rs:310

docs.rs/src/docbuilder/rustwide_builder.rs

Line 310 in ba9364b

let metadata = Metadata::from_crate_root(&build.host_source_dir())?;

There are two bugs here:

1. The error message should be showing the parse error as a cause. Not sure what's wrong there, the thiserror docs say that #[from] implies #[source] but that doesn't seem to be the case.
2. We fail to parse valid toml files:

failed to parse manifest: unexpected character found: `<` at line 3 column 1

Which is very strange because it doesn't have a < character except for an email address:

[[bin]]
name = "builder"
required-features = ["builder"]

[[bin]]
name = "runner"
required-features = ["runner"]

[[bin]]
name = "bios"
required-features = ["bios_bin"]

[[bin]]
name = "uefi"
required-features = ["uefi_bin"]
[build-dependencies.llvm-tools-build]
optional = true
package = "llvm-tools"
version = "0.1"

[build-dependencies.serde]
features = ["derive"]
optional = true
version = "1.0"

[build-dependencies.toml]
optional = true
version = "0.5.1"
[dependencies.anyhow]
optional = true
version = "1.0.32"

[dependencies.argh]
optional = true
version = "0.1.3"

[dependencies.bit_field]
optional = true
version = "0.10.0"

[dependencies.conquer-once]
default-features = false
optional = true
version = "0.2.1"

[dependencies.displaydoc]
optional = true
version = "0.1.7"

[dependencies.fatfs]
optional = true
version = "0.3.4"

[dependencies.font8x8]
default-features = false
features = ["unicode"]
optional = true
version = "0.2.5"

[dependencies.gpt]
optional = true
version = "2.0.0"

[dependencies.json]
optional = true
version = "0.12.4"

[dependencies.llvm-tools]
optional = true
version = "0.1.1"

[dependencies.log]
optional = true
version = "0.4.8"

[dependencies.rsdp]
optional = true
version = "1.0.0"

[dependencies.spinning_top]
optional = true
version = "0.2.1"

[dependencies.thiserror]
optional = true
version = "1.0.20"

[dependencies.uefi]
optional = true
version = "0.7.0"

[dependencies.usize_conversions]
optional = true
version = "0.2.0"

[dependencies.x86_64]
optional = true
version = "0.13.2"

[dependencies.xmas-elf]
optional = true
version = "0.6.2"

[features]
binary = ["llvm-tools-build", "x86_64", "toml", "xmas-elf", "usize_conversions", "log", "conquer-once", "spinning_top", "serde"]
bios_bin = ["binary", "vga_320x200", "rsdp"]
builder = ["argh", "thiserror", "displaydoc", "anyhow", "llvm-tools", "json", "fatfs", "gpt"]
default = []
map_physical_memory = []
recursive_page_table = []
runner = ["anyhow"]
uefi_bin = ["binary", "uefi", "font8x8"]
vga_320x200 = ["font8x8"]

[package]
authors = ["Philipp Oppermann <[email protected]>"]
build = "build.rs"
description = "An experimental pure-Rust x86 bootloader."
edition = "2018"
license = "MIT/Apache-2.0"
name = "bootloader"
repository = "https://github.com/rust-osdev/bootloader"
version = "0.10.0-alpha-03"
[package.metadata.bootloader]
build-std = "core"
target = "x86_64-bootloader.json"
[package.metadata.docs.rs]
default-target = "x86_64-unknown-linux-gnu"
features = ["recursive_page_table", "map_physical_memory"]

[package.metadata.release]
disable-publish = true
no-dev-version = true
pre-release-commit-message = "Release version {{version}}"

[[package.metadata.release.pre-release-replacements]]
exactly = 1
file = "Changelog.md"
replace = "# Unreleased\n\n# {{version}} – {{date}}"
search = "# Unreleased"
[profile.dev]
panic = "abort"

[profile.release]
debug = true
lto = false
overflow-checks = true
panic = "abort"

[jyn514]

Uhhh apparently merge conflict markers got in there? Unsure if this is actually a docs.rs bug, it's strange that what it's trying to parse isn't the same as the file copied though:

2021/06/12 13:26:12 [INFO] rustwide::prepare: tweaked toml for crates.io crate bootloader 0.10.0-alpha-03 written to .workspace/builds/bootloader-0.10.0-alpha-03/source/Cargo.toml
2021/06/12 13:26:12 [INFO] rustwide::prepare: crate crates.io crate bootloader 0.10.0-alpha-03 already has a lockfile, it will not be regenerated
2021/06/12 13:26:12 [INFO] rustwide::cmd: running `Command { std: "/home/joshua/src/rust/docs.rs/.workspace/cargo-home/bin/cargo" "+nightly" "fetch" "--locked" "--manifest-path" "Cargo.toml", kill_on_drop: false }`
[crates/metadata/lib.rs:297] manifest = "[package]
name = "bootloader"
<<<<<<< HEAD
version = "0.10.0-alpha-01"
=======
version = "0.9.12"
>>>>>>> main

[jyn514]

Apparently .orig has the conflict markers but .toml does not; seems like this might be a rustwide bug? cc @pietroalbini

Anyway, I think this can be closed on the docs.rs side as a duplicate of #797.

[pietroalbini]

Huh why are we caring about the .orig? Only .toml should be parsed by Rustwide and docs.rs...

[Nemo157]

docs.rs/crates/metadata/lib.rs

Lines 169 to 170 in 803cebf

	/// If both `Cargo.toml` and `Cargo.toml.orig` exist in the directory,
	/// `Cargo.toml.orig` will take precedence.

So not rustwide related, I'm not sure why we prefer the .orig here, the only difference normally is stripped dev-dependencies which don't matter for a doc build.

[Nemo157]

8cd6725

I don't see any rationale for why we do, I think it would make sense to change this to ignore the .orig file.

[jyn514]

That seems fine, but the bug I mean is that rustwide strips conflict markers, that doesn't seem right.

[jyn514]

Or I guess maybe it's cargo doing it? Something is fishy.

[Nemo157]

Yeah, it must have been whatever was used to publish the crate (which was likely cargo), it's in the tarball nothing to do with what we're doing on our end:

> cargo download -x bootloader==0.10.0-alpha-03 && rg '<<<<<'
INFO: cargo-download v0.1.2
INFO: Crate `bootloader==0.10.0-alpha-03` downloaded successfully
INFO: Crate content extracted to ./bootloader-0.10.0-alpha-03/
bootloader-0.10.0-alpha-03/Cargo.lock.orig
64:<<<<<<< HEAD

bootloader-0.10.0-alpha-03/Cargo.toml.orig
3:<<<<<<< HEAD
39:<<<<<<< HEAD

[Nemo157]

Disappointingly it doesn't appear that the version of cargo used to build the tarball is recorded anywhere, if I move Cargo.toml.orig over the Cargo.toml and try cargo package with current and a year-and-a-half old nightly they both fail to parse the manifest, so I have no idea how it got published.

[jyn514]

Ok cool, I'll reopen this to track parsing Cargo.toml instead of Cargo.toml.orig.