Rust on Zephyr: cmake and code organization

[d3zd3z]

Introduction

RFC #65837 describes the work necessary to support applications written in Rust into Zephyr. This RFC attempts to discuss how that implementation will be done, practically, within the Zephyr source tree. This will cover where files will be placed into the source tree, and how they will integrate with the various toolchains.

As a bit of a background, this RFC assumes the reader is familiar with the build tools used within Zephyr, including CMake, Kconfig and devicetree. The primary build and dependency tool used for Zephyr is Cargo. Although Cargo is primarily concerned with building standalone applications, there are a few reasons that it is still a useful tool for writing Rust applications within Zephyr:

• Cargo already supports the idea of building a static library.
• Cargo already supports cross compiling.
• Cargo manages external dependencies. Although the core Zephyr system will not want to depend on external dependencies that are downloaded during the build, those making applications in Rust that use zephyr will likely want to have access to the "crates.io" ecosystem. There are a large number of external crates that would be quite useful for someone developing an application, with a growing number of them supporting what is known as "nostd", or building without using the rich-OS-based standard library.

Before diving further in, it is important to defined a few terms:

• Crate: A crate is the primary unit of larger-scale modularity in Rust programs. It can be thought of as similar to what might be included within a shared library in a C or C++ program. The source code for a crate lives under a single directory tree, and is built in its entirety by the rust compiler. The source code within a crate lives in its own namespace, although crates it depends on can be brought into that namespace (using unique names at the top level), and the published names from a given crate can be exported in a similar manner to other crates that depend upon it.
• Module: A module is a unit of namespace within a crate. Each source file within the crate is a module, although modules can also be written directly in the source code (and these are equivalent). The language provides a set of mechanisms, for importing symbols into modules. Modules are hierarchical, which is modeled with subdirectories in the filesystem. However, unlike languages like Java, since the crate has its own namespace, there is little need to have unnecessary top-level modules, and the desired functionality is generally placed directly where it most makes sense.
• Source file: A single file on the filesystem, ending in .rs that corresponds to a module.

The rust toolchain has the following components:

• rustup: This is the most common tool developers use to install rust toolchains. It is especially helpful for embedded development as it supports not only installing the basic toolchain, but builds of the core and alloc libraries for many of the platforms supported by Rust.
• cargo: The build and dependency management system of Rust. The build is described by a Cargo.toml file at the top of a crate. The Cargo.toml file describes the version, name information, license, and attributes of the crate. There is a feature mechanism that can be used for conditional compilation. The Cargo.toml file also lists the direct dependencies of the crate (these can be conditionalized by features). For example, it is common, in embedded Rust development, to have a crate that implements some functionality that will have a std (or nostd) feature. When std is available, additional debugging and testing will be available. Often these crates can be developed as ordinary userspace code on the development machine, and then made to work in the embedded system with relatively little effort.
• rustc: This is the main compiler. Although it can be invoked, like gcc, on individual components of source code, it is usually left to cargo to do this.
• rust-fmt: This is a tool for automatic source code reformatting. It is common for developers to configured the IDE to run this tool before saving or before committing.
• rust-analyzer: This is a language server for Rust that allows editors and IDEs to have source code analysis features available. Allowing this to work on Zephyr applications is an important constraint to the implementation proposed below.

The rust compile provides the following libraries:

• core: The core library provides functionality necessary for any Rust program. It is fairly small. It can somewhat thought of as similar to libgcc, but it does provide a public API, which provides intrinsics and support that doesn't depend on any functionality outside of Rust.
• alloc: The alloc library, which must be provided an allocator, provides standard library support for types that require dynamic memory. Its use is optional for a Rust application within Zephyr, although a lot of upcoming functionality will depend upon it (for example, providing synchroniazation libraries that have a similar API to standard Rust ones will depend on allocation). Some of the things it provides are Vec, dynamically allocated arrays, String, and various dictionary and set types.
• std: The std library provides the rest of the functionality provided by the Rust standard library. This is support that depends upon the underlying operating system. The intent is to not provide std at this time, but make similar functionality, when it makes sense, available in other crates that we provide.

Problem description

We would like developers to be able to write Rust applications, on top of Zephyr, that feel comfortable for development, both from the perspective of a Rust developer, and from the perspective of a Zephyr developer. This RFC does not address the details of how Rust code will interface with Zephyr, but with how the rust code will be organized, and how it will be built.

A Rust application will be similar to how a C application is produced for Zephyr. At the top-level, there will be a CMakeLists.txt file that brings in the Zephyr cmake package, and describes that this is an application. The mechanisms for config and dts overrides will be identical to that of a C program. The main difference is that instead of needing to specify a list of C source files, the cmake file will have a single line:

rust_cargo_application()

after the Zephyr package. The name is sufficiently detailed to allow for other uses of Rust within Zephyr, but captures the initial implementation of writing a main application using Cargo.

Beyond this, the application will have a fairly ordinary Cargo.toml file. This file should specify that this crate is to be built as a static library, but otherwise, only needs to list information relevant to this particular application (features and dependencies). This application can be built with the same west build tools for other programs. In addition, the build will make available a template file that can be copied or symlinked to .cargo/config.toml in the source tree that will provide enough information to cargo so that tools such as rust-analyzer will work unchanged with the source code. With this configuration file, the analyzer will be seeing the program as it is when cross compiled to work with Zephyr, which will make all functionality available, including config options (conditional Rust code by kconfig option will be greyed out by an IDE for example).

Proposed change

There are two locations within the tree that will need to be added to in order to support this type of Rust application:

• lib/rust: This directory will have support that is needed to implement Rust support. Details below.
• cmake/modules/rust.cmake: Implements the rust_cargo_application() function, and additional functionality for the build to work.

One outstanding question is whether Rust code necessary to build Zephyr applications in Rust (support libraries) should be included directly in the Zephyr tree (perhaps under lib/rust), or whether there should be a module to support this code. It is also possible to have a hybrid approach, such as if we need to mirror a crate to implement functionality, it could be placed in a module, and the core support would be within the Zephyr tree.

Detailed RFC

The bulk of the cmake work for this will be contained within cmake/modules/rust.cmake. This provides a function rust_cargo_application() that can be used by an application to build a Zephyr application where the main program is in Rust. This does a few things to make this work:

• It provides a dependency target built around an application static library. This library will be produced by cargo, and cmake rules will be used to ensure that cargo is run at the proper time within the build.
• It determines configuration needed for cargo so that the build output is put in the Zephyr build directory (proposing to put this in build/rust when the build is in build).
• It maps between the architecture configuration settings that select the compiler version and flags to appropriate flag to pass to cargo to produce a compatible build. Primarily, this will produce a "target triple", that aligns with the target triple used by LLVM.
• It adds all Kconfig bool flags that are set to the build so that they can be used for conditional compilation. Cargo requires conditional flags to be defined at build invocation time, and not by any source files present within the build.
• It collects the crates that are part of the build that are provided by source code within the Zephyr tree, or modules, and creates configuration settings so that cargo will use these in place rather than attempting to download them as dependencies. Only dependencies specified by the user beyond these in the application's Cargo.toml will be resolved via conventional mechanisms.
• It writes the configuration to a file build/rust/sample-cargo-config.toml which can be copied or symlinked to .cargo/config.toml to allow IDE and other tools to work without modification.
• It invokes cargo with the necessary command line flags.

The above change is sufficient to build a simple "hello world" Rust application that does a handful of manual tasks, and uses hand-crafted bindings to C functions within Zephyr.

Beyond this, there will be one more crates (either under lib/rust/..., or in one or more modules) that implement modules to make development of programs in Rust on Zephyr more usable:

• zephyr: This Zephyr create will be the main interface into the Zephyr API. As additional work provides bindings to the Zephyr interfaces, device tree, etc, this will provide these interfaces. In addition, it will provide a "panic" implementation to Rust so that panic calls within Rust code will call into Zephyr's panic.
• Other crates to implement this. For example zephyr-sys will likely be where bindings take place.
• One or more compile-time crates (that provide compile time support to an application) to provide functionality to a Zephyr application:
  • Kconfig accessibility. The kconfig values that are not Bool will be available in a crate. Although generated at compile time, these will be re-exported through the zephyr crate, and can be directly accessed.
  • Devicetree accessibility. Provides, at compile time, a module namespace that maps to the generated device tree. This will provide direct properties, and with future work, provide convenient functions to get Rust-usable bindings directly to various drivers.

Proposed change (Detailed)

• Create cmake/modules/rust.cmake, and add a line to cmake/modules/zephyr_default.cmake to have it included.
• Add zephyr crate under lib/rust/zephyr to implement basic functionality.

This is a framework for future work.

Dependencies

As this describes the initial work for Rust on Zephyr, the primary dependencies will be on the user having a functioning install of the Rust toolchain. This can generally be managed by rustup. It is probably desirable to extend Twister so that it can decide whether or not to run rust tests based on whether the toolchain is present.

Concerns and Unresolved Questions

• There is an outstanding issue with the Rust toolchain, where it provides an equivalent of libgcc directly into the generated library file. However, the implementations are not sufficiently fine grained to prevent conflicts with libgcc. There is a workaround to simply tell the linker to ignore these duplicate symbols, but there should be an effort to help fix these within upstream.
• We will need to decide how much of this gets tested by CI. CI support will require adding to Rust toolchain to the CI docker image, as well as necessary support to make twister work. In addition, future work could allow Rust code that uses Rust's unit tests to be buildable to run on target.

Alternatives

There has been previous work for Rust on Zephyr that added std support. The result was complex to build, as it required the user to rebuild the rust toolchain, and worked with a very specific version of Zephyr. This proposal tries to loosen the coupling between Rust versions and Zephyr versions, as well as realizing that std isn't a very good initial goal.