Add Prost and Tonic rules. (bazelbuild#2033)

* Setup Prost and Tonic rules.

* Regenerate documentation

* Add more tests.

* Add more tests and address feedback.

* Regenerate documentation

* Add to proto docs page.

* Bump min supported bazel version.

* buildifier

* Always enable backtracing.

* Add more info to failing rename.

* Set min rust version to 1.62.0

* Handle rust keywords as package names.

* exclude windows from prost toolchain support.

* buildifier

* redundant

* Use prost-types to parse the file descriptor set.

* Cleanup and more tests.

* Move prost-types to toolchain definition.

* fix rustfmt

* Add example of building protos with complex imports

* impl Display

* Fix all tests

* Add rust checks for the complex import protos.

* Address feedback

* Fix buildifier

* Depend on remote-apis repo.

* Remove bazel remote apis due to file length and transitive dependency issues.

* Update patch and docs.

* Regenerate documentation

* Regenerate documentation

* Update docs.

---------

Co-authored-by: Daniel Wagner-Hall <[email protected]>

Author: 2 people

.bazelci/presubmit.yml

@@ -3,7 +3,7 @@ aspects_flags: &aspects_flags
   - "--config=rustfmt"
   - "--config=clippy"
 min_rust_version_shell_commands: &min_rust_version_shell_commands
-  - sed -i 's|^rust_register_toolchains(|rust_register_toolchains(versions = ["1.59.0"],\n|' WORKSPACE.bazel
+  - sed -i 's|^rust_register_toolchains(|rust_register_toolchains(versions = ["1.62.0"],\n|' WORKSPACE.bazel
 nightly_flags: &nightly_flags
   - "--//rust/toolchain/channel=nightly"
 nightly_aspects_flags: &nightly_aspects_flags
@@ -205,15 +205,15 @@ tasks:
   ubuntu1804:
     name: "Min Bazel Version"
     # If updating the minimum bazel version, please also update /docs/index.md
-    bazel: "5.2.0"
+    bazel: "6.0.0"
     platform: ubuntu1804
     build_targets: *default_linux_targets
     test_targets: *default_linux_targets
     coverage_targets: *default_linux_targets
     post_shell_commands: *coverage_validation_post_shell_commands
   ubuntu1804_with_aspects:
     name: "Min Bazel Version With Aspects"
-    bazel: "5.2.0"
+    bazel: "6.0.0"
     platform: ubuntu1804
     build_targets: *default_linux_targets
     test_targets: *default_linux_targets

WORKSPACE.bazel

@@ -10,14 +10,18 @@ load("@rules_rust//crate_universe:repositories.bzl", "crate_universe_dependencie
 
 crate_universe_dependencies(bootstrap = True)
 
-load("@rules_rust//proto:repositories.bzl", "rust_proto_repositories")
+load("@rules_rust//proto:repositories.bzl", "rust_proto_dependencies", "rust_proto_register_toolchains")
 
-rust_proto_repositories()
+rust_proto_dependencies()
+
+rust_proto_register_toolchains()
 
 load("@rules_rust//proto:transitive_repositories.bzl", "rust_proto_transitive_repositories")
 
 rust_proto_transitive_repositories()
 
+register_toolchains("@rules_rust//proto/prost/private/...")
+
 load("@rules_rust//bindgen:repositories.bzl", "rust_bindgen_dependencies", "rust_bindgen_register_toolchains")
 
 rust_bindgen_dependencies()

crate_universe/private/crates_vendor.bzl

@@ -346,7 +346,7 @@ bazel run //3rdparty:crates_vendor -- --repin
 ```
 
 Under the hood, `--repin` will trigger a [cargo update](https://doc.rust-lang.org/cargo/commands/cargo-update.html)
-call against the generated workspace. The following table describes how to controll particular values passed to the
+call against the generated workspace. The following table describes how to control particular values passed to the
 `cargo update` command.
 
 | Value | Cargo command |

docs/BUILD.bazel

@@ -121,6 +121,8 @@ PAGES = dict([
             "rust_proto_repositories",
             "rust_proto_transitive_repositories",
             "rust_proto_toolchain",
+            "rust_prost_library",
+            "rust_tonic_library",
         ],
     ),
     page(

docs/crate_universe.md

@@ -462,7 +462,7 @@ bazel run //3rdparty:crates_vendor -- --repin
 ```
 
 Under the hood, `--repin` will trigger a [cargo update](https://doc.rust-lang.org/cargo/commands/cargo-update.html)
-call against the generated workspace. The following table describes how to controll particular values passed to the
+call against the generated workspace. The following table describes how to control particular values passed to the
 `cargo update` command.
 
 | Value | Cargo command |

docs/flatten.md

@@ -31,6 +31,7 @@
 * [rust_library](#rust_library)
 * [rust_library_group](#rust_library_group)
 * [rust_proc_macro](#rust_proc_macro)
+* [rust_prost_library](#rust_prost_library)
 * [rust_proto_library](#rust_proto_library)
 * [rust_proto_repositories](#rust_proto_repositories)
 * [rust_proto_toolchain](#rust_proto_toolchain)
@@ -43,6 +44,7 @@
 * [rust_stdlib_filegroup](#rust_stdlib_filegroup)
 * [rust_test](#rust_test)
 * [rust_test_suite](#rust_test_suite)
+* [rust_tonic_library](#rust_tonic_library)
 * [rust_toolchain](#rust_toolchain)
 * [rust_toolchain_repository](#rust_toolchain_repository)
 * [rust_toolchain_repository_proxy](#rust_toolchain_repository_proxy)
@@ -1794,6 +1796,25 @@ Registers the default toolchains for the `rules_rust` [bindgen][bg] rules.
 | <a id="rust_bindgen_register_toolchains-register_toolchains"></a>register_toolchains |  Whether or not to register toolchains.   |  `True` |
 
 
+<a id="rust_prost_library"></a>
+
+## rust_prost_library
+
+<pre>
+rust_prost_library(<a href="#rust_prost_library-name">name</a>, <a href="#rust_prost_library-kwargs">kwargs</a>)
+</pre>
+
+A rule for generating a Rust library using Prost.
+
+**PARAMETERS**
+
+
+| Name  | Description | Default Value |
+| :------------- | :------------- | :------------- |
+| <a id="rust_prost_library-name"></a>name |  The name of the target.   |  none |
+| <a id="rust_prost_library-kwargs"></a>kwargs |  Additional keyword arguments for the underlying <code>rust_prost_library</code> rule.   |  none |
+
+
 <a id="rust_proto_repositories"></a>
 
 ## rust_proto_repositories
@@ -1998,6 +2019,25 @@ rust_test_suite(
 | <a id="rust_test_suite-kwargs"></a>kwargs |  Additional keyword arguments for the underyling [rust_test](#rust_test) targets. The <code>tags</code> argument is also passed to the generated <code>test_suite</code> target.   |  none |
 
 
+<a id="rust_tonic_library"></a>
+
+## rust_tonic_library
+
+<pre>
+rust_tonic_library(<a href="#rust_tonic_library-name">name</a>, <a href="#rust_tonic_library-kwargs">kwargs</a>)
+</pre>
+
+A rule for generating a Rust library using Prost and Tonic.
+
+**PARAMETERS**
+
+
+| Name  | Description | Default Value |
+| :------------- | :------------- | :------------- |
+| <a id="rust_tonic_library-name"></a>name |  The name of the target.   |  none |
+| <a id="rust_tonic_library-kwargs"></a>kwargs |  Additional keyword arguments for the underlying <code>rust_tonic_library</code> rule.   |  none |
+
+
 <a id="rust_toolchain_repository"></a>
 
 ## rust_toolchain_repository

docs/index.md

@@ -93,7 +93,7 @@ Failure to do so will result in rules attempting to match a `stable` toolchain w
 
 ## Supported bazel versions
 
-The oldest version of Bazel the `main` branch is tested against is `5.2.0`. Previous versions may still be functional in certain environments, but this is the minimum version we strive to fully support.
+The oldest version of Bazel the `main` branch is tested against is `6.0.0`. Previous versions may still be functional in certain environments, but this is the minimum version we strive to fully support.
 
 We test these rules against the latest rolling releases of Bazel, and aim for compatibility with them, but prioritise stable releases over rolling releases where necessary.
 

docs/rust_proto.md

@@ -6,15 +6,22 @@
 * [rust_proto_repositories](#rust_proto_repositories)
 * [rust_proto_transitive_repositories](#rust_proto_transitive_repositories)
 * [rust_proto_toolchain](#rust_proto_toolchain)
+* [rust_prost_library](#rust_prost_library)
+* [rust_tonic_library](#rust_tonic_library)
 
 
 ## Overview
-
 These build rules are used for building [protobufs][protobuf]/[gRPC][grpc] in [Rust][rust] with Bazel.
 
+There are two rule sets. The first ruleset defines the `rust_proto_library` and `rust_grpc_library`
+rules which generate Rust code using the [`rust-protobuf`] dependencies. The second ruleset defines
+the `rust_prost_library` and `rust_tonic_library` rules which generate Rust code using the [`prost`]
+and [`tonic`] dependencies respectively.
+
 [rust]: http://www.rust-lang.org/
 [protobuf]: https://developers.google.com/protocol-buffers/
 [grpc]: https://grpc.io
+[`rust-protobuf`]: https://github.com/stepancheg/rust-protobuf/
 
 See the [protobuf example](../examples/proto) for a more complete example of use.
 
@@ -33,17 +40,21 @@ load("@rules_rust//proto:transitive_repositories.bzl", "rust_proto_transitive_re
 rust_proto_transitive_repositories()
 ```
 
-This will load crate dependencies of protobuf that are generated using
-[crate_universe](./crate_universe.md) inside the rules_rust
-repository. However, using those dependencies might conflict with other uses
-of [crate_universe](./crate_universe.md). If you need to change
-those dependencies, please see the [dedicated section below](#custom-deps).
+This will load the required dependencies for the Proto, Prost, and Tonic rules. It will also
+register a default toolchain for the `rust_proto_library` and `rust_grpc_library` rules. The
+`prost` and `tonic` rules do not specify a default toolchain in order to avoid mismatched
+dependency issues.
+
+To customize the `rust_proto_library` and `rust_grpc_library` toolchain, please see the section
+[Customizing `rust-protobuf` Dependencies](#custom-proto-deps).
+
+To setup the `prost` and `tonic` toolchain, please see the section [Customizing `prost` and `tonic` Dependencies](#custom-prost-deps).
 
 For additional information about Bazel toolchains, see [here](https://docs.bazel.build/versions/master/toolchains.html).
 
-## <a name="custom-deps">Customizing dependencies
+## <a name="custom-proto-deps">Customizing `rust-protobuf` Dependencies
 
-These rules depends on the [`protobuf`](https://crates.io/crates/protobuf) and
+These rules depend on the [`protobuf`](https://crates.io/crates/protobuf) and
 the [`grpc`](https://crates.io/crates/grpc) crates in addition to the [protobuf
 compiler](https://github.com/google/protobuf). To obtain these crates,
 `rust_proto_repositories` imports the given crates using BUILD files generated with
@@ -114,6 +125,111 @@ but due to [bazelbuild/bazel#6889](https://github.com/bazelbuild/bazel/issues/68
 all dependencies added via the toolchain ends-up being in the wrong
 configuration.
 
+## <a name="custom-prost-deps">Customizing `prost` and `tonic` Dependencies
+
+These rules depend on the [`prost`] and [`tonic`] dependencies. To setup the necessary toolchain
+for these rules, you must define a toolchain with the [`prost`], [`prost-types`], [`tonic`],[`protoc-gen-prost`], and [`protoc-gen-tonic`] crates as well as the [`protoc`] binary.
+
+[`prost`]: https://crates.io/crates/prost
+[`prost-types`]: https://crates.io/crates/prost-types
+[`protoc-gen-prost`]: https://crates.io/crates/protoc-gen-prost
+[`protoc-gen-tonic`]: https://crates.io/crates/protoc-gen-tonic
+[`tonic`]: https://crates.io/crates/tonic
+[`protoc`]: https://github.com/protocolbuffers/protobuf
+
+To get access to these crates, you can use the [crate_universe](./crate_universe.md) repository
+rules. For example:
+
+```python
+load("//crate_universe:defs.bzl", "crate", "crates_repository")
+
+crates_repository(
+    name = "crates_io",
+    annotations = {
+        "protoc-gen-prost": [crate.annotation(
+            gen_binaries = ["protoc-gen-prost"],
+            patch_args = [
+                "-p1",
+            ],
+            patches = [
+                # Note: You will need to use this patch until a version greater than `0.2.2` of
+                # `protoc-gen-prost` is released.
+                "@rules_rust//proto/prost/private/3rdparty/patches:protoc-gen-prost.patch",
+            ],
+        )],
+        "protoc-gen-tonic": [crate.annotation(
+            gen_binaries = ["protoc-gen-tonic"],
+        )],
+    },
+    cargo_lockfile = "Cargo.Bazel.lock",
+    mode = "remote",
+    packages = {
+        "prost": crate.spec(
+            version = "0",
+        ),
+        "prost-types": crate.spec(
+            version = "0",
+        ),
+        "protoc-gen-prost": crate.spec(
+            version = "0",
+        ),
+        "protoc-gen-tonic": crate.spec(
+            version = "0",
+        ),
+        "tonic": crate.spec(
+            version = "0",
+        ),
+    },
+    repository_name = "rules_rust_prost",
+    tags = ["manual"],
+)
+```
+
+You can then define a toolchain with the `rust_prost_toolchain` rule which uses the crates
+defined above. For example:
+
+```python
+load("@rules_rust//proto/prost:defs.bzl", "rust_prost_toolchain")
+load("@rules_rust//rust:defs.bzl", "rust_library_group")
+
+rust_library_group(
+    name = "prost_runtime",
+    deps = [
+        "@crates_io//:prost",
+    ],
+)
+
+rust_library_group(
+    name = "tonic_runtime",
+    deps = [
+        ":prost_runtime",
+        "@crates_io//:tonic",
+    ],
+)
+
+rust_prost_toolchain(
+    name = "prost_toolchain_impl",
+    prost_plugin = "@crates_io//:protoc-gen-prost__protoc-gen-prost",
+    prost_runtime = ":prost_runtime",
+    prost_types = "@crates_io//:prost-types",
+    proto_compiler = "@com_google_protobuf//:protoc",
+    tonic_plugin = "@crates_io//:protoc-gen-tonic__protoc-gen-tonic",
+    tonic_runtime = ":tonic_runtime",
+)
+
+toolchain(
+    name = "prost_toolchain",
+    toolchain = "default_prost_toolchain_impl",
+    toolchain_type = "//proto/prost:toolchain_type",
+)
+```
+
+Lastly, you must register the toolchain in your `WORKSPACE` file. For example:
+
+```python
+register_toolchains("//toolchains:prost_toolchain")
+```
+
 
 <a id="rust_grpc_library"></a>
 
@@ -260,6 +376,25 @@ See @rules_rust//proto:BUILD for examples of defining the toolchain.
 | <a id="rust_proto_toolchain-protoc"></a>protoc |  The location of the <code>protoc</code> binary. It should be an executable target.   | <a href="https://bazel.build/concepts/labels">Label</a> | optional | <code>@com_google_protobuf//:protoc</code> |
 
 
+<a id="rust_prost_library"></a>
+
+## rust_prost_library
+
+<pre>
+rust_prost_library(<a href="#rust_prost_library-name">name</a>, <a href="#rust_prost_library-kwargs">kwargs</a>)
+</pre>
+
+A rule for generating a Rust library using Prost.
+
+**PARAMETERS**
+
+
+| Name  | Description | Default Value |
+| :------------- | :------------- | :------------- |
+| <a id="rust_prost_library-name"></a>name |  The name of the target.   |  none |
+| <a id="rust_prost_library-kwargs"></a>kwargs |  Additional keyword arguments for the underlying <code>rust_prost_library</code> rule.   |  none |
+
+
 <a id="rust_proto_repositories"></a>
 
 ## rust_proto_repositories
@@ -292,3 +427,22 @@ This macro should be called immediately after the `rust_proto_repositories` macr
 
 
 
+<a id="rust_tonic_library"></a>
+
+## rust_tonic_library
+
+<pre>
+rust_tonic_library(<a href="#rust_tonic_library-name">name</a>, <a href="#rust_tonic_library-kwargs">kwargs</a>)
+</pre>
+
+A rule for generating a Rust library using Prost and Tonic.
+
+**PARAMETERS**
+
+
+| Name  | Description | Default Value |
+| :------------- | :------------- | :------------- |
+| <a id="rust_tonic_library-name"></a>name |  The name of the target.   |  none |
+| <a id="rust_tonic_library-kwargs"></a>kwargs |  Additional keyword arguments for the underlying <code>rust_tonic_library</code> rule.   |  none |
+
+