diff --git a/Documentation/process/changes.rst b/Documentation/process/changes.rst index bb96ca0f774b9a..52284fdbaf23ee 100644 --- a/Documentation/process/changes.rst +++ b/Documentation/process/changes.rst @@ -31,7 +31,7 @@ you probably needn't concern yourself with pcmciautils. ====================== =============== ======================================== GNU C 5.1 gcc --version Clang/LLVM (optional) 11.0.0 clang --version -Rust (optional) 1.73.0 rustc --version +Rust (optional) 1.75.0 rustc --version bindgen (optional) 0.65.1 bindgen --version GNU make 3.82 make --version bash 4.2 bash --version diff --git a/Documentation/rust/arch-support.rst b/Documentation/rust/arch-support.rst index b91e9ef4d0c21e..6bcb3b97c5b610 100644 --- a/Documentation/rust/arch-support.rst +++ b/Documentation/rust/arch-support.rst @@ -15,6 +15,7 @@ support corresponds to ``S`` values in the ``MAINTAINERS`` file. ============ ================ ============================================== Architecture Level of support Constraints ============ ================ ============================================== +``arm64`` Maintained Little Endian only. ``um`` Maintained ``x86_64`` only. ``x86`` Maintained ``x86_64`` only. ============ ================ ============================================== diff --git a/Documentation/rust/coding-guidelines.rst b/Documentation/rust/coding-guidelines.rst index aa8ed082613e12..05542840b16cca 100644 --- a/Documentation/rust/coding-guidelines.rst +++ b/Documentation/rust/coding-guidelines.rst @@ -177,6 +177,19 @@ please take a look at the ``rustdoc`` book at: https://doc.rust-lang.org/rustdoc/how-to-write-documentation.html +In addition, the kernel supports creating links relative to the source tree by +prefixing the link destination with ``srctree/``. For instance: + +.. code-block:: rust + + //! C header: [`include/linux/printk.h`](srctree/include/linux/printk.h) + +or: + +.. code-block:: rust + + /// [`struct mutex`]: srctree/include/linux/mutex.h + Naming ------ diff --git a/Documentation/rust/general-information.rst b/Documentation/rust/general-information.rst index 081397827a7eab..236c6dd3c647f8 100644 --- a/Documentation/rust/general-information.rst +++ b/Documentation/rust/general-information.rst @@ -77,3 +77,27 @@ configuration: #[cfg(CONFIG_X="y")] // Enabled as a built-in (`y`) #[cfg(CONFIG_X="m")] // Enabled as a module (`m`) #[cfg(not(CONFIG_X))] // Disabled + + +Testing +------- + +There are the tests that come from the examples in the Rust documentation +and get transformed into KUnit tests. These can be run via KUnit. For example +via ``kunit_tool`` (``kunit.py``) on the command line:: + + ./tools/testing/kunit/kunit.py run --make_options LLVM=1 --arch x86_64 --kconfig_add CONFIG_RUST=y + +Alternatively, KUnit can run them as kernel built-in at boot. Refer to +Documentation/dev-tools/kunit/index.rst for the general KUnit documentation +and Documentation/dev-tools/kunit/architecture.rst for the details of kernel +built-in vs. command line testing. + +Additionally, there are the ``#[test]`` tests. These can be run using +the ``rusttest`` Make target:: + + make LLVM=1 rusttest + +This requires the kernel ``.config`` and downloads external repositories. +It runs the ``#[test]`` tests on the host (currently) and thus is fairly +limited in what these tests can test. diff --git a/Documentation/rust/quick-start.rst b/Documentation/rust/quick-start.rst index f382914f41917a..cc3f11e0d441f8 100644 --- a/Documentation/rust/quick-start.rst +++ b/Documentation/rust/quick-start.rst @@ -33,14 +33,18 @@ A particular version of the Rust compiler is required. Newer versions may or may not work because, for the moment, the kernel depends on some unstable Rust features. -If ``rustup`` is being used, enter the checked out source code directory -and run:: +If ``rustup`` is being used, enter the kernel build directory (or use +``--path=`` argument to the ``set`` sub-command) and run:: rustup override set $(scripts/min-tool-version.sh rustc) This will configure your working directory to use the correct version of -``rustc`` without affecting your default toolchain. If you are not using -``rustup``, fetch a standalone installer from: +``rustc`` without affecting your default toolchain. + +Note that the override applies to the current working directory (and its +sub-directories). + +If you are not using ``rustup``, fetch a standalone installer from: https://forge.rust-lang.org/infra/other-installation-methods.html#standalone @@ -76,7 +80,7 @@ libclang ``libclang`` (part of LLVM) is used by ``bindgen`` to understand the C code in the kernel, which means LLVM needs to be installed; like when the kernel -is compiled with ``CC=clang`` or ``LLVM=1``. +is compiled with ``LLVM=1``. Linux distributions are likely to have a suitable one available, so it is best to check that first. @@ -229,10 +233,6 @@ at the moment. That is:: make LLVM=1 -For architectures that do not support a full LLVM toolchain, use:: - - make CC=clang - Using GCC also works for some configurations, but it is very experimental at the moment. diff --git a/Makefile b/Makefile index e78ee7db0729b6..45cf497c6715c8 100644 --- a/Makefile +++ b/Makefile @@ -555,7 +555,6 @@ KBUILD_CFLAGS += -fno-strict-aliasing KBUILD_CPPFLAGS := -D__KERNEL__ KBUILD_RUSTFLAGS := $(rust_common_flags) \ - --target=$(objtree)/scripts/target.json \ -Cpanic=abort -Cembed-bitcode=n -Clto=n \ -Cforce-unwind-tables=n -Ccodegen-units=1 \ -Csymbol-mangling-version=v0 \ diff --git a/arch/arm64/Kconfig b/arch/arm64/Kconfig index 7b071a00425d2b..1aaa1d456d9d4b 100644 --- a/arch/arm64/Kconfig +++ b/arch/arm64/Kconfig @@ -227,6 +227,7 @@ config ARM64 select HAVE_FUNCTION_ARG_ACCESS_API select MMU_GATHER_RCU_TABLE_FREE select HAVE_RSEQ + select HAVE_RUST if CPU_LITTLE_ENDIAN select HAVE_STACKPROTECTOR select HAVE_SYSCALL_TRACEPOINTS select HAVE_KPROBES diff --git a/arch/arm64/Makefile b/arch/arm64/Makefile index 9a2d3723cd0fa9..7af127552f5302 100644 --- a/arch/arm64/Makefile +++ b/arch/arm64/Makefile @@ -41,6 +41,8 @@ KBUILD_CFLAGS += -mgeneral-regs-only \ KBUILD_CFLAGS += $(call cc-disable-warning, psabi) KBUILD_AFLAGS += $(compat_vdso) +KBUILD_RUSTFLAGS += --target=aarch64-unknown-none -Ctarget-feature="-neon" + KBUILD_CFLAGS += $(call cc-option,-mabi=lp64) KBUILD_AFLAGS += $(call cc-option,-mabi=lp64) @@ -65,7 +67,9 @@ endif ifeq ($(CONFIG_ARM64_BTI_KERNEL),y) KBUILD_CFLAGS += -mbranch-protection=pac-ret+bti + KBUILD_RUSTFLAGS += -Zbranch-protection=bti,pac-ret else ifeq ($(CONFIG_ARM64_PTR_AUTH_KERNEL),y) + KBUILD_RUSTFLAGS += -Zbranch-protection=pac-ret ifeq ($(CONFIG_CC_HAS_BRANCH_PROT_PAC_RET),y) KBUILD_CFLAGS += -mbranch-protection=pac-ret else diff --git a/arch/x86/Makefile b/arch/x86/Makefile index 1a068de12a564f..86d347ff6b3362 100644 --- a/arch/x86/Makefile +++ b/arch/x86/Makefile @@ -68,6 +68,7 @@ export BITS # https://gcc.gnu.org/bugzilla/show_bug.cgi?id=53383 # KBUILD_CFLAGS += -mno-sse -mno-mmx -mno-sse2 -mno-3dnow -mno-avx +KBUILD_RUSTFLAGS += --target=$(objtree)/scripts/target.json KBUILD_RUSTFLAGS += -Ctarget-feature=-sse,-sse2,-sse3,-ssse3,-sse4.1,-sse4.2,-avx,-avx2 ifeq ($(CONFIG_X86_KERNEL_IBT),y) diff --git a/rust/Makefile b/rust/Makefile index 543b37f6c77f12..cb40a1ce133158 100644 --- a/rust/Makefile +++ b/rust/Makefile @@ -78,6 +78,7 @@ quiet_cmd_rustdoc = RUSTDOC $(if $(rustdoc_host),H, ) $< $(rustc_target_flags) -L$(objtree)/$(obj) \ --output $(rustdoc_output) \ --crate-name $(subst rustdoc-,,$@) \ + $(if $(rustdoc_host),,--sysroot=/dev/null) \ @$(objtree)/include/generated/rustc_cfg $< # The `html_logo_url` and `html_favicon_url` forms of the `doc` attribute @@ -98,7 +99,8 @@ rustdoc: rustdoc-core rustdoc-macros rustdoc-compiler_builtins \ $(Q)find $(rustdoc_output) -name '*.html' -type f -print0 | xargs -0 sed -Ei \ -e 's:rust-logo-[0-9a-f]+\.svg:logo.svg:g' \ -e 's:favicon-[0-9a-f]+\.svg:logo.svg:g' \ - -e 's:::g' + -e 's:::g' \ + -e 's:::g' $(Q)for f in $(rustdoc_output)/static.files/rustdoc-*.css; do \ echo ".logo-container > img { object-fit: contain; }" >> $$f; done @@ -178,6 +180,7 @@ quiet_cmd_rustdoc_test_kernel = RUSTDOC TK $< --extern build_error --extern macros \ --extern bindings --extern uapi \ --no-run --crate-name kernel -Zunstable-options \ + --sysroot=/dev/null \ --test-builder $(objtree)/scripts/rustdoc_test_builder \ $< $(rustdoc_test_kernel_quiet); \ $(objtree)/scripts/rustdoc_test_gen @@ -294,6 +297,7 @@ bindgen_skip_c_flags := -mno-fp-ret-in-387 -mpreferred-stack-boundary=% \ # Derived from `scripts/Makefile.clang`. BINDGEN_TARGET_x86 := x86_64-linux-gnu +BINDGEN_TARGET_arm64 := aarch64-linux-gnu BINDGEN_TARGET := $(BINDGEN_TARGET_$(SRCARCH)) # All warnings are inhibited since GCC builds are very experimental, @@ -337,6 +341,8 @@ quiet_cmd_bindgen = BINDGEN $@ $(obj)/bindings/bindings_generated.rs: private bindgen_target_flags = \ $(shell grep -Ev '^#|^$$' $(srctree)/$(src)/bindgen_parameters) +$(obj)/bindings/bindings_generated.rs: private bindgen_target_extra = ; \ + sed -Ei 's/pub const RUST_CONST_HELPER_([a-zA-Z0-9_]*)/pub const \1/g' $@ $(obj)/bindings/bindings_generated.rs: $(src)/bindings/bindings_helper.h \ $(src)/bindgen_parameters FORCE $(call if_changed_dep,bindgen) @@ -402,6 +408,7 @@ quiet_cmd_rustc_library = $(if $(skip_clippy),RUSTC,$(RUSTC_OR_CLIPPY_QUIET)) L --emit=metadata=$(dir $@)$(patsubst %.o,lib%.rmeta,$(notdir $@)) \ --crate-type rlib -L$(objtree)/$(obj) \ --crate-name $(patsubst %.o,%,$(notdir $@)) $< \ + --sysroot=/dev/null \ $(if $(rustc_objcopy),;$(OBJCOPY) $(rustc_objcopy) $@) rust-analyzer: @@ -428,8 +435,11 @@ $(obj)/core.o: private skip_clippy = 1 $(obj)/core.o: private skip_flags = -Dunreachable_pub $(obj)/core.o: private rustc_objcopy = $(foreach sym,$(redirect-intrinsics),--redefine-sym $(sym)=__rust$(sym)) $(obj)/core.o: private rustc_target_flags = $(core-cfgs) -$(obj)/core.o: $(RUST_LIB_SRC)/core/src/lib.rs scripts/target.json FORCE +$(obj)/core.o: $(RUST_LIB_SRC)/core/src/lib.rs FORCE $(call if_changed_dep,rustc_library) +ifdef CONFIG_X86_64 +$(obj)/core.o: scripts/target.json +endif $(obj)/compiler_builtins.o: private rustc_objcopy = -w -W '__*' $(obj)/compiler_builtins.o: $(src)/compiler_builtins.rs $(obj)/core.o FORCE diff --git a/rust/alloc/alloc.rs b/rust/alloc/alloc.rs index 8cb4a31cf6e54d..8a6be8c981731a 100644 --- a/rust/alloc/alloc.rs +++ b/rust/alloc/alloc.rs @@ -345,18 +345,31 @@ extern "Rust" { fn __rust_alloc_error_handler(size: usize, align: usize) -> !; } -/// Abort on memory allocation error or failure. +/// Signal a memory allocation error. /// -/// Callers of memory allocation APIs wishing to abort computation +/// Callers of memory allocation APIs wishing to cease execution /// in response to an allocation error are encouraged to call this function, -/// rather than directly invoking `panic!` or similar. +/// rather than directly invoking [`panic!`] or similar. /// -/// The default behavior of this function is to print a message to standard error -/// and abort the process. -/// It can be replaced with [`set_alloc_error_hook`] and [`take_alloc_error_hook`]. +/// This function is guaranteed to diverge (not return normally with a value), but depending on +/// global configuration, it may either panic (resulting in unwinding or aborting as per +/// configuration for all panics), or abort the process (with no unwinding). +/// +/// The default behavior is: +/// +/// * If the binary links against `std` (typically the case), then +/// print a message to standard error and abort the process. +/// This behavior can be replaced with [`set_alloc_error_hook`] and [`take_alloc_error_hook`]. +/// Future versions of Rust may panic by default instead. +/// +/// * If the binary does not link against `std` (all of its crates are marked +/// [`#![no_std]`][no_std]), then call [`panic!`] with a message. +/// [The panic handler] applies as to any panic. /// /// [`set_alloc_error_hook`]: ../../std/alloc/fn.set_alloc_error_hook.html /// [`take_alloc_error_hook`]: ../../std/alloc/fn.take_alloc_error_hook.html +/// [The panic handler]: https://doc.rust-lang.org/reference/runtime.html#the-panic_handler-attribute +/// [no_std]: https://doc.rust-lang.org/reference/names/preludes.html#the-no_std-attribute #[stable(feature = "global_alloc", since = "1.28.0")] #[rustc_const_unstable(feature = "const_alloc_error", issue = "92523")] #[cfg(all(not(no_global_oom_handling), not(test)))] @@ -366,13 +379,20 @@ pub const fn handle_alloc_error(layout: Layout) -> ! { panic!("allocation failed"); } + #[inline] fn rt_error(layout: Layout) -> ! { unsafe { __rust_alloc_error_handler(layout.size(), layout.align()); } } - unsafe { core::intrinsics::const_eval_select((layout,), ct_error, rt_error) } + #[cfg(not(feature = "panic_immediate_abort"))] + unsafe { + core::intrinsics::const_eval_select((layout,), ct_error, rt_error) + } + + #[cfg(feature = "panic_immediate_abort")] + ct_error(layout) } // For alloc test `std::alloc::handle_alloc_error` can be used directly. @@ -397,9 +417,10 @@ pub mod __alloc_error_handler { if unsafe { __rust_alloc_error_handler_should_panic != 0 } { panic!("memory allocation of {size} bytes failed") } else { - core::panicking::panic_nounwind_fmt(format_args!( - "memory allocation of {size} bytes failed" - )) + core::panicking::panic_nounwind_fmt( + format_args!("memory allocation of {size} bytes failed"), + /* force_no_backtrace */ false, + ) } } } diff --git a/rust/alloc/boxed.rs b/rust/alloc/boxed.rs index 9620eba1726872..f5f40778a19373 100644 --- a/rust/alloc/boxed.rs +++ b/rust/alloc/boxed.rs @@ -161,7 +161,7 @@ use core::marker::Tuple; use core::marker::Unsize; use core::mem::{self, SizedTypeProperties}; use core::ops::{ - CoerceUnsized, Deref, DerefMut, DispatchFromDyn, Generator, GeneratorState, Receiver, + CoerceUnsized, Coroutine, CoroutineState, Deref, DerefMut, DispatchFromDyn, Receiver, }; use core::pin::Pin; use core::ptr::{self, NonNull, Unique}; @@ -211,7 +211,7 @@ impl Box { /// ``` /// let five = Box::new(5); /// ``` - #[cfg(all(not(no_global_oom_handling)))] + #[cfg(not(no_global_oom_handling))] #[inline(always)] #[stable(feature = "rust1", since = "1.0.0")] #[must_use] @@ -2110,28 +2110,28 @@ impl AsMut for Box { #[stable(feature = "pin", since = "1.33.0")] impl Unpin for Box where A: 'static {} -#[unstable(feature = "generator_trait", issue = "43122")] -impl + Unpin, R, A: Allocator> Generator for Box +#[unstable(feature = "coroutine_trait", issue = "43122")] +impl + Unpin, R, A: Allocator> Coroutine for Box where A: 'static, { type Yield = G::Yield; type Return = G::Return; - fn resume(mut self: Pin<&mut Self>, arg: R) -> GeneratorState { + fn resume(mut self: Pin<&mut Self>, arg: R) -> CoroutineState { G::resume(Pin::new(&mut *self), arg) } } -#[unstable(feature = "generator_trait", issue = "43122")] -impl, R, A: Allocator> Generator for Pin> +#[unstable(feature = "coroutine_trait", issue = "43122")] +impl, R, A: Allocator> Coroutine for Pin> where A: 'static, { type Yield = G::Yield; type Return = G::Return; - fn resume(mut self: Pin<&mut Self>, arg: R) -> GeneratorState { + fn resume(mut self: Pin<&mut Self>, arg: R) -> CoroutineState { G::resume((*self).as_mut(), arg) } } @@ -2448,4 +2448,8 @@ impl core::error::Error for Box { fn source(&self) -> Option<&(dyn core::error::Error + 'static)> { core::error::Error::source(&**self) } + + fn provide<'b>(&'b self, request: &mut core::error::Request<'b>) { + core::error::Error::provide(&**self, request); + } } diff --git a/rust/alloc/lib.rs b/rust/alloc/lib.rs index 73b9ffd845d952..345cf5c9cf92d7 100644 --- a/rust/alloc/lib.rs +++ b/rust/alloc/lib.rs @@ -80,6 +80,8 @@ not(no_sync), target_has_atomic = "ptr" ))] +#![cfg_attr(not(bootstrap), doc(rust_logo))] +#![cfg_attr(not(bootstrap), feature(rustdoc_internals))] #![no_std] #![needs_allocator] // Lints: @@ -90,8 +92,8 @@ #![warn(missing_docs)] #![allow(explicit_outlives_requirements)] #![warn(multiple_supertrait_upcastable)] -#![cfg_attr(not(bootstrap), allow(internal_features))] -#![cfg_attr(not(bootstrap), allow(rustdoc::redundant_explicit_links))] +#![allow(internal_features)] +#![allow(rustdoc::redundant_explicit_links)] // // Library features: // tidy-alphabetical-start @@ -115,13 +117,13 @@ #![feature(const_eval_select)] #![feature(const_maybe_uninit_as_mut_ptr)] #![feature(const_maybe_uninit_write)] -#![feature(const_maybe_uninit_zeroed)] #![feature(const_pin)] #![feature(const_refs_to_cell)] #![feature(const_size_of_val)] #![feature(const_waker)] #![feature(core_intrinsics)] #![feature(core_panic)] +#![feature(deprecated_suggestion)] #![feature(dispatch_from_dyn)] #![feature(error_generic_member_access)] #![feature(error_in_core)] @@ -140,12 +142,11 @@ #![feature(maybe_uninit_uninit_array)] #![feature(maybe_uninit_uninit_array_transpose)] #![feature(pattern)] -#![feature(pointer_byte_offsets)] +#![feature(ptr_addr_eq)] #![feature(ptr_internals)] #![feature(ptr_metadata)] #![feature(ptr_sub_ptr)] #![feature(receiver_trait)] -#![feature(saturating_int_impl)] #![feature(set_ptr_value)] #![feature(sized_type_properties)] #![feature(slice_from_ptr_range)] @@ -168,7 +169,7 @@ // // Language features: // tidy-alphabetical-start -#![cfg_attr(not(test), feature(generator_trait))] +#![cfg_attr(not(test), feature(coroutine_trait))] #![cfg_attr(test, feature(panic_update_hook))] #![cfg_attr(test, feature(test))] #![feature(allocator_internals)] diff --git a/rust/alloc/raw_vec.rs b/rust/alloc/raw_vec.rs index a7425582a323f1..f1b8cec8cc6268 100644 --- a/rust/alloc/raw_vec.rs +++ b/rust/alloc/raw_vec.rs @@ -338,10 +338,13 @@ impl RawVec { /// The same as `reserve`, but returns on errors instead of panicking or aborting. pub fn try_reserve(&mut self, len: usize, additional: usize) -> Result<(), TryReserveError> { if self.needs_to_grow(len, additional) { - self.grow_amortized(len, additional) - } else { - Ok(()) + self.grow_amortized(len, additional)?; + } + unsafe { + // Inform the optimizer that the reservation has succeeded or wasn't needed + core::intrinsics::assume(!self.needs_to_grow(len, additional)); } + Ok(()) } /// The same as `reserve_for_push`, but returns on errors instead of panicking or aborting. @@ -378,7 +381,14 @@ impl RawVec { len: usize, additional: usize, ) -> Result<(), TryReserveError> { - if self.needs_to_grow(len, additional) { self.grow_exact(len, additional) } else { Ok(()) } + if self.needs_to_grow(len, additional) { + self.grow_exact(len, additional)?; + } + unsafe { + // Inform the optimizer that the reservation has succeeded or wasn't needed + core::intrinsics::assume(!self.needs_to_grow(len, additional)); + } + Ok(()) } /// Shrinks the buffer down to the specified capacity. If the given amount @@ -569,6 +579,7 @@ fn alloc_guard(alloc_size: usize) -> Result<(), TryReserveError> { // ensure that the code generation related to these panics is minimal as there's // only one location which panics rather than a bunch throughout the module. #[cfg(not(no_global_oom_handling))] +#[cfg_attr(not(feature = "panic_immediate_abort"), inline(never))] fn capacity_overflow() -> ! { panic!("capacity overflow"); } diff --git a/rust/alloc/slice.rs b/rust/alloc/slice.rs index 6ac463bd3edc53..1181836da5f462 100644 --- a/rust/alloc/slice.rs +++ b/rust/alloc/slice.rs @@ -594,7 +594,7 @@ impl [T] { /// ``` #[rustc_allow_incoherent_impl] #[stable(feature = "rust1", since = "1.0.0")] - #[deprecated(since = "1.3.0", note = "renamed to join")] + #[deprecated(since = "1.3.0", note = "renamed to join", suggestion = "join")] pub fn connect(&self, sep: Separator) -> >::Output where Self: Join, diff --git a/rust/alloc/vec/mod.rs b/rust/alloc/vec/mod.rs index 209a88cfe598f1..0d95fd7ef3375f 100644 --- a/rust/alloc/vec/mod.rs +++ b/rust/alloc/vec/mod.rs @@ -1228,8 +1228,8 @@ impl Vec { /// Shortens the vector, keeping the first `len` elements and dropping /// the rest. /// - /// If `len` is greater than the vector's current length, this has no - /// effect. + /// If `len` is greater or equal to the vector's current length, this has + /// no effect. /// /// The [`drain`] method can emulate `truncate`, but causes the excess /// elements to be returned instead of dropped. @@ -1336,6 +1336,15 @@ impl Vec { /// is never written to (except inside an `UnsafeCell`) using this pointer or any pointer /// derived from it. If you need to mutate the contents of the slice, use [`as_mut_ptr`]. /// + /// This method guarantees that for the purpose of the aliasing model, this method + /// does not materialize a reference to the underlying slice, and thus the returned pointer + /// will remain valid when mixed with other calls to [`as_ptr`] and [`as_mut_ptr`]. + /// Note that calling other methods that materialize mutable references to the slice, + /// or mutable references to specific elements you are planning on accessing through this pointer, + /// as well as writing to those elements, may still invalidate this pointer. + /// See the second example below for how this guarantee can be used. + /// + /// /// # Examples /// /// ``` @@ -1349,8 +1358,25 @@ impl Vec { /// } /// ``` /// + /// Due to the aliasing guarantee, the following code is legal: + /// + /// ```rust + /// unsafe { + /// let mut v = vec![0, 1, 2]; + /// let ptr1 = v.as_ptr(); + /// let _ = ptr1.read(); + /// let ptr2 = v.as_mut_ptr().offset(2); + /// ptr2.write(2); + /// // Notably, the write to `ptr2` did *not* invalidate `ptr1` + /// // because it mutated a different element: + /// let _ = ptr1.read(); + /// } + /// ``` + /// /// [`as_mut_ptr`]: Vec::as_mut_ptr + /// [`as_ptr`]: Vec::as_ptr #[stable(feature = "vec_as_ptr", since = "1.37.0")] + #[rustc_never_returns_null_ptr] #[inline] pub fn as_ptr(&self) -> *const T { // We shadow the slice method of the same name to avoid going through @@ -1366,6 +1392,15 @@ impl Vec { /// Modifying the vector may cause its buffer to be reallocated, /// which would also make any pointers to it invalid. /// + /// This method guarantees that for the purpose of the aliasing model, this method + /// does not materialize a reference to the underlying slice, and thus the returned pointer + /// will remain valid when mixed with other calls to [`as_ptr`] and [`as_mut_ptr`]. + /// Note that calling other methods that materialize references to the slice, + /// or references to specific elements you are planning on accessing through this pointer, + /// may still invalidate this pointer. + /// See the second example below for how this guarantee can be used. + /// + /// /// # Examples /// /// ``` @@ -1383,7 +1418,25 @@ impl Vec { /// } /// assert_eq!(&*x, &[0, 1, 2, 3]); /// ``` + /// + /// Due to the aliasing guarantee, the following code is legal: + /// + /// ```rust + /// unsafe { + /// let mut v = vec![0]; + /// let ptr1 = v.as_mut_ptr(); + /// ptr1.write(1); + /// let ptr2 = v.as_mut_ptr(); + /// ptr2.write(2); + /// // Notably, the write to `ptr2` did *not* invalidate `ptr1`: + /// ptr1.write(3); + /// } + /// ``` + /// + /// [`as_mut_ptr`]: Vec::as_mut_ptr + /// [`as_ptr`]: Vec::as_ptr #[stable(feature = "vec_as_ptr", since = "1.37.0")] + #[rustc_never_returns_null_ptr] #[inline] pub fn as_mut_ptr(&mut self) -> *mut T { // We shadow the slice method of the same name to avoid going through @@ -1512,7 +1565,8 @@ impl Vec { #[stable(feature = "rust1", since = "1.0.0")] pub fn swap_remove(&mut self, index: usize) -> T { #[cold] - #[inline(never)] + #[cfg_attr(not(feature = "panic_immediate_abort"), inline(never))] + #[track_caller] fn assert_failed(index: usize, len: usize) -> ! { panic!("swap_remove index (is {index}) should be < len (is {len})"); } @@ -1553,7 +1607,8 @@ impl Vec { #[stable(feature = "rust1", since = "1.0.0")] pub fn insert(&mut self, index: usize, element: T) { #[cold] - #[inline(never)] + #[cfg_attr(not(feature = "panic_immediate_abort"), inline(never))] + #[track_caller] fn assert_failed(index: usize, len: usize) -> ! { panic!("insertion index (is {index}) should be <= len (is {len})"); } @@ -1614,7 +1669,7 @@ impl Vec { #[track_caller] pub fn remove(&mut self, index: usize) -> T { #[cold] - #[inline(never)] + #[cfg_attr(not(feature = "panic_immediate_abort"), inline(never))] #[track_caller] fn assert_failed(index: usize, len: usize) -> ! { panic!("removal index (is {index}) should be < len (is {len})"); @@ -2044,6 +2099,7 @@ impl Vec { } else { unsafe { self.len -= 1; + core::intrinsics::assume(self.len < self.capacity()); Some(ptr::read(self.as_ptr().add(self.len()))) } } @@ -2246,7 +2302,8 @@ impl Vec { A: Clone, { #[cold] - #[inline(never)] + #[cfg_attr(not(feature = "panic_immediate_abort"), inline(never))] + #[track_caller] fn assert_failed(at: usize, len: usize) -> ! { panic!("`at` split index (is {at}) should be <= len (is {len})"); } @@ -3403,6 +3460,36 @@ impl From<&mut [T]> for Vec { } } +#[cfg(not(no_global_oom_handling))] +#[stable(feature = "vec_from_array_ref", since = "1.74.0")] +impl From<&[T; N]> for Vec { + /// Allocate a `Vec` and fill it by cloning `s`'s items. + /// + /// # Examples + /// + /// ``` + /// assert_eq!(Vec::from(&[1, 2, 3]), vec![1, 2, 3]); + /// ``` + fn from(s: &[T; N]) -> Vec { + Self::from(s.as_slice()) + } +} + +#[cfg(not(no_global_oom_handling))] +#[stable(feature = "vec_from_array_ref", since = "1.74.0")] +impl From<&mut [T; N]> for Vec { + /// Allocate a `Vec` and fill it by cloning `s`'s items. + /// + /// # Examples + /// + /// ``` + /// assert_eq!(Vec::from(&mut [1, 2, 3]), vec![1, 2, 3]); + /// ``` + fn from(s: &mut [T; N]) -> Vec { + Self::from(s.as_mut_slice()) + } +} + #[cfg(not(no_global_oom_handling))] #[stable(feature = "vec_from_array", since = "1.44.0")] impl From<[T; N]> for Vec { diff --git a/rust/bindgen_parameters b/rust/bindgen_parameters index 552d9a85925b99..a721d466bee4b2 100644 --- a/rust/bindgen_parameters +++ b/rust/bindgen_parameters @@ -20,3 +20,7 @@ # `seccomp`'s comment gets understood as a doctest --no-doc-comments + +# These functions use the `__preserve_most` calling convention, which neither bindgen +# nor Rust currently understand, and which Clang currently declares to be unstable. +--blocklist-function __list_.*_report diff --git a/rust/bindings/bindings_helper.h b/rust/bindings/bindings_helper.h index eaf01df7d97a71..1bd8f39898adbf 100644 --- a/rust/bindings/bindings_helper.h +++ b/rust/bindings/bindings_helper.h @@ -7,17 +7,26 @@ */ #include +#include #include #include +#include +#include +#include +#include #include +#include #include +#include +#include #include #include #include #include +#include #include /* `bindgen` gets confused at certain things. */ -const size_t BINDINGS_ARCH_SLAB_MINALIGN = ARCH_SLAB_MINALIGN; -const gfp_t BINDINGS_GFP_KERNEL = GFP_KERNEL; -const gfp_t BINDINGS___GFP_ZERO = __GFP_ZERO; +const size_t RUST_CONST_HELPER_ARCH_SLAB_MINALIGN = ARCH_SLAB_MINALIGN; +const gfp_t RUST_CONST_HELPER_GFP_KERNEL = GFP_KERNEL; +const gfp_t RUST_CONST_HELPER___GFP_ZERO = __GFP_ZERO; diff --git a/rust/bindings/lib.rs b/rust/bindings/lib.rs index 9bcbea04dac305..40ddaee50d8bd8 100644 --- a/rust/bindings/lib.rs +++ b/rust/bindings/lib.rs @@ -48,6 +48,3 @@ mod bindings_helper { } pub use bindings_raw::*; - -pub const GFP_KERNEL: gfp_t = BINDINGS_GFP_KERNEL; -pub const __GFP_ZERO: gfp_t = BINDINGS___GFP_ZERO; diff --git a/rust/exports.c b/rust/exports.c index 83e2a7070cae88..3803c21d1403ef 100644 --- a/rust/exports.c +++ b/rust/exports.c @@ -11,7 +11,7 @@ * accidentally exposed. */ -#include +#include #define EXPORT_SYMBOL_RUST_GPL(sym) extern int sym; EXPORT_SYMBOL_GPL(sym) diff --git a/rust/helpers.c b/rust/helpers.c index 70e59efd92bc43..d146bbf25aec4a 100644 --- a/rust/helpers.c +++ b/rust/helpers.c @@ -23,12 +23,16 @@ #include #include #include +#include #include #include +#include #include #include #include +#include #include +#include #include #include @@ -139,6 +143,51 @@ void rust_helper_put_task_struct(struct task_struct *t) } EXPORT_SYMBOL_GPL(rust_helper_put_task_struct); +kuid_t rust_helper_task_uid(struct task_struct *task) +{ + return task_uid(task); +} +EXPORT_SYMBOL_GPL(rust_helper_task_uid); + +kuid_t rust_helper_task_euid(struct task_struct *task) +{ + return task_euid(task); +} +EXPORT_SYMBOL_GPL(rust_helper_task_euid); + +#ifndef CONFIG_USER_NS +uid_t rust_helper_from_kuid(struct user_namespace *to, kuid_t uid) +{ + return from_kuid(to, uid); +} +EXPORT_SYMBOL_GPL(rust_helper_from_kuid); +#endif /* CONFIG_USER_NS */ + +bool rust_helper_uid_eq(kuid_t left, kuid_t right) +{ + return uid_eq(left, right); +} +EXPORT_SYMBOL_GPL(rust_helper_uid_eq); + +kuid_t rust_helper_current_euid(void) +{ + return current_euid(); +} +EXPORT_SYMBOL_GPL(rust_helper_current_euid); + +struct user_namespace *rust_helper_current_user_ns(void) +{ + return current_user_ns(); +} +EXPORT_SYMBOL_GPL(rust_helper_current_user_ns); + +pid_t rust_helper_task_tgid_nr_ns(struct task_struct *tsk, + struct pid_namespace *ns) +{ + return task_tgid_nr_ns(tsk, ns); +} +EXPORT_SYMBOL_GPL(rust_helper_task_tgid_nr_ns); + struct kunit *rust_helper_kunit_get_current_test(void) { return kunit_get_current_test(); @@ -157,6 +206,51 @@ void rust_helper_init_work_with_key(struct work_struct *work, work_func_t func, } EXPORT_SYMBOL_GPL(rust_helper_init_work_with_key); +struct file *rust_helper_get_file(struct file *f) +{ + return get_file(f); +} +EXPORT_SYMBOL_GPL(rust_helper_get_file); + +const struct cred *rust_helper_get_cred(const struct cred *cred) +{ + return get_cred(cred); +} +EXPORT_SYMBOL_GPL(rust_helper_get_cred); + +void rust_helper_put_cred(const struct cred *cred) +{ + put_cred(cred); +} +EXPORT_SYMBOL_GPL(rust_helper_put_cred); + +#ifndef CONFIG_SECURITY +void rust_helper_security_cred_getsecid(const struct cred *c, u32 *secid) +{ + security_cred_getsecid(c, secid); +} +EXPORT_SYMBOL_GPL(rust_helper_security_cred_getsecid); + +int rust_helper_security_secid_to_secctx(u32 secid, char **secdata, u32 *seclen) +{ + return security_secid_to_secctx(secid, secdata, seclen); +} +EXPORT_SYMBOL_GPL(rust_helper_security_secid_to_secctx); + +void rust_helper_security_release_secctx(char *secdata, u32 seclen) +{ + security_release_secctx(secdata, seclen); +} +EXPORT_SYMBOL_GPL(rust_helper_security_release_secctx); +#endif + +void rust_helper_init_task_work(struct callback_head *twork, + task_work_func_t func) +{ + init_task_work(twork, func); +} +EXPORT_SYMBOL_GPL(rust_helper_init_task_work); + /* * `bindgen` binds the C `size_t` type as the Rust `usize` type, so we can * use it in contexts where Rust expects a `usize` like slice (array) indices. diff --git a/rust/kernel/allocator.rs b/rust/kernel/allocator.rs index a8f3d5be1af1e7..4b057e837358c0 100644 --- a/rust/kernel/allocator.rs +++ b/rust/kernel/allocator.rs @@ -21,7 +21,7 @@ unsafe fn krealloc_aligned(ptr: *mut u8, new_layout: Layout, flags: bindings::gf let mut size = layout.size(); - if layout.align() > bindings::BINDINGS_ARCH_SLAB_MINALIGN { + if layout.align() > bindings::ARCH_SLAB_MINALIGN { // The alignment requirement exceeds the slab guarantee, thus try to enlarge the size // to use the "power-of-two" size/alignment guarantee (see comments in `kmalloc()` for // more information). diff --git a/rust/kernel/cred.rs b/rust/kernel/cred.rs new file mode 100644 index 00000000000000..fbc749788bfa3c --- /dev/null +++ b/rust/kernel/cred.rs @@ -0,0 +1,73 @@ +// SPDX-License-Identifier: GPL-2.0 + +//! Credentials management. +//! +//! C header: [`include/linux/cred.h`](../../../../include/linux/cred.h) +//! +//! Reference: + +use crate::{ + bindings, + task::Kuid, + types::{AlwaysRefCounted, Opaque}, +}; + +/// Wraps the kernel's `struct cred`. +/// +/// # Invariants +/// +/// Instances of this type are always ref-counted, that is, a call to `get_cred` ensures that the +/// allocation remains valid at least until the matching call to `put_cred`. +#[repr(transparent)] +pub struct Credential(pub(crate) Opaque); + +// SAFETY: By design, the only way to access a `Credential` is via an immutable reference or an +// `ARef`. This means that the only situation in which a `Credential` can be accessed mutably is +// when the refcount drops to zero and the destructor runs. It is safe for that to happen on any +// thread, so it is ok for this type to be `Send`. +unsafe impl Send for Credential {} + +// SAFETY: It's OK to access `Credential` through shared references from other threads because +// we're either accessing properties that don't change or that are properly synchronised by C code. +unsafe impl Sync for Credential {} + +impl Credential { + /// Creates a reference to a [`Credential`] from a valid pointer. + /// + /// # Safety + /// + /// The caller must ensure that `ptr` is valid and remains valid for the lifetime of the + /// returned [`Credential`] reference. + pub unsafe fn from_ptr<'a>(ptr: *const bindings::cred) -> &'a Credential { + // SAFETY: The safety requirements guarantee the validity of the dereference, while the + // `Credential` type being transparent makes the cast ok. + unsafe { &*ptr.cast() } + } + + /// Get the id for this security context. + pub fn get_secid(&self) -> u32 { + let mut secid = 0; + // SAFETY: The invariants of this type ensures that the pointer is valid. + unsafe { bindings::security_cred_getsecid(self.0.get(), &mut secid) }; + secid + } + + /// Returns the effective UID of the given credential. + pub fn euid(&self) -> Kuid { + // SAFETY: By the type invariant, we know that `self.0` is valid. + Kuid::from_raw(unsafe { (*self.0.get()).euid }) + } +} + +// SAFETY: The type invariants guarantee that `Credential` is always ref-counted. +unsafe impl AlwaysRefCounted for Credential { + fn inc_ref(&self) { + // SAFETY: The existence of a shared reference means that the refcount is nonzero. + unsafe { bindings::get_cred(self.0.get()) }; + } + + unsafe fn dec_ref(obj: core::ptr::NonNull) { + // SAFETY: The safety requirements guarantee that the refcount is nonzero. + unsafe { bindings::put_cred(obj.cast().as_ptr()) }; + } +} diff --git a/rust/kernel/error.rs b/rust/kernel/error.rs index 032b6454395395..4f0c1edd63b7a8 100644 --- a/rust/kernel/error.rs +++ b/rust/kernel/error.rs @@ -2,7 +2,7 @@ //! Kernel errors. //! -//! C header: [`include/uapi/asm-generic/errno-base.h`](../../../include/uapi/asm-generic/errno-base.h) +//! C header: [`include/uapi/asm-generic/errno-base.h`](srctree/include/uapi/asm-generic/errno-base.h) use crate::str::CStr; @@ -335,3 +335,7 @@ where Err(e) => T::from(e.to_errno() as i16), } } + +/// Error message for calling a default function of a [`#[vtable]`](macros::vtable) trait. +pub const VTABLE_DEFAULT_ERROR: &str = + "This function must not be called, see the #[vtable] documentation."; diff --git a/rust/kernel/file.rs b/rust/kernel/file.rs new file mode 100644 index 00000000000000..eba96af4bdf76c --- /dev/null +++ b/rust/kernel/file.rs @@ -0,0 +1,431 @@ +// SPDX-License-Identifier: GPL-2.0 + +//! Files and file descriptors. +//! +//! C headers: [`include/linux/fs.h`](../../../../include/linux/fs.h) and +//! [`include/linux/file.h`](../../../../include/linux/file.h) + +use crate::{ + bindings, + cred::Credential, + error::{code::*, Error, Result}, + types::{ARef, AlwaysRefCounted, NotThreadSafe, Opaque}, +}; +use alloc::boxed::Box; +use core::{alloc::AllocError, marker::PhantomData, mem, ptr}; + +/// Flags associated with a [`File`]. +pub mod flags { + /// File is opened in append mode. + pub const O_APPEND: u32 = bindings::O_APPEND; + + /// Signal-driven I/O is enabled. + pub const O_ASYNC: u32 = bindings::FASYNC; + + /// Close-on-exec flag is set. + pub const O_CLOEXEC: u32 = bindings::O_CLOEXEC; + + /// File was created if it didn't already exist. + pub const O_CREAT: u32 = bindings::O_CREAT; + + /// Direct I/O is enabled for this file. + pub const O_DIRECT: u32 = bindings::O_DIRECT; + + /// File must be a directory. + pub const O_DIRECTORY: u32 = bindings::O_DIRECTORY; + + /// Like [`O_SYNC`] except metadata is not synced. + pub const O_DSYNC: u32 = bindings::O_DSYNC; + + /// Ensure that this file is created with the `open(2)` call. + pub const O_EXCL: u32 = bindings::O_EXCL; + + /// Large file size enabled (`off64_t` over `off_t`). + pub const O_LARGEFILE: u32 = bindings::O_LARGEFILE; + + /// Do not update the file last access time. + pub const O_NOATIME: u32 = bindings::O_NOATIME; + + /// File should not be used as process's controlling terminal. + pub const O_NOCTTY: u32 = bindings::O_NOCTTY; + + /// If basename of path is a symbolic link, fail open. + pub const O_NOFOLLOW: u32 = bindings::O_NOFOLLOW; + + /// File is using nonblocking I/O. + pub const O_NONBLOCK: u32 = bindings::O_NONBLOCK; + + /// Also known as `O_NDELAY`. + /// + /// This is effectively the same flag as [`O_NONBLOCK`] on all architectures + /// except SPARC64. + pub const O_NDELAY: u32 = bindings::O_NDELAY; + + /// Used to obtain a path file descriptor. + pub const O_PATH: u32 = bindings::O_PATH; + + /// Write operations on this file will flush data and metadata. + pub const O_SYNC: u32 = bindings::O_SYNC; + + /// This file is an unnamed temporary regular file. + pub const O_TMPFILE: u32 = bindings::O_TMPFILE; + + /// File should be truncated to length 0. + pub const O_TRUNC: u32 = bindings::O_TRUNC; + + /// Bitmask for access mode flags. + /// + /// # Examples + /// + /// ``` + /// use kernel::file; + /// # fn do_something() {} + /// # let flags = 0; + /// if (flags & file::flags::O_ACCMODE) == file::flags::O_RDONLY { + /// do_something(); + /// } + /// ``` + pub const O_ACCMODE: u32 = bindings::O_ACCMODE; + + /// File is read only. + pub const O_RDONLY: u32 = bindings::O_RDONLY; + + /// File is write only. + pub const O_WRONLY: u32 = bindings::O_WRONLY; + + /// File can be both read and written. + pub const O_RDWR: u32 = bindings::O_RDWR; +} + +/// Wraps the kernel's `struct file`. +/// +/// # Invariants +/// +/// Instances of this type are always ref-counted, that is, a call to `get_file` ensures that the +/// allocation remains valid at least until the matching call to `fput`. +#[repr(transparent)] +pub struct File(Opaque); + +// SAFETY: By design, the only way to access a `File` is via an immutable reference or an `ARef`. +// This means that the only situation in which a `File` can be accessed mutably is when the +// refcount drops to zero and the destructor runs. It is safe for that to happen on any thread, so +// it is ok for this type to be `Send`. +unsafe impl Send for File {} + +// SAFETY: All methods defined on `File` that take `&self` are safe to call even if other threads +// are concurrently accessing the same `struct file`, because those methods either access immutable +// properties or have proper synchronization to ensure that such accesses are safe. +unsafe impl Sync for File {} + +impl File { + /// Constructs a new `struct file` wrapper from a file descriptor. + /// + /// The file descriptor belongs to the current process. + pub fn fget(fd: u32) -> Result, BadFdError> { + // SAFETY: FFI call, there are no requirements on `fd`. + let ptr = ptr::NonNull::new(unsafe { bindings::fget(fd) }).ok_or(BadFdError)?; + + // SAFETY: `fget` either returns null or a valid pointer to a file, and we checked for null + // above. + // + // INVARIANT: `fget` increments the refcount before returning. + Ok(unsafe { ARef::from_raw(ptr.cast()) }) + } + + /// Creates a reference to a [`File`] from a valid pointer. + /// + /// # Safety + /// + /// The caller must ensure that `ptr` points at a valid file and that its refcount does not + /// reach zero during the lifetime 'a. + pub unsafe fn from_ptr<'a>(ptr: *const bindings::file) -> &'a File { + // SAFETY: The caller guarantees that the pointer is not dangling and stays valid for the + // duration of 'a. The cast is okay because `File` is `repr(transparent)`. + // + // INVARIANT: The safety requirements guarantee that the refcount does not hit zero during + // 'a. + unsafe { &*ptr.cast() } + } + + /// Returns a raw pointer to the inner C struct. + #[inline] + pub fn as_ptr(&self) -> *mut bindings::file { + self.0.get() + } + + /// Returns the credentials of the task that originally opened the file. + pub fn cred(&self) -> &Credential { + // SAFETY: Since the caller holds a reference to the file, it is guaranteed that its + // refcount does not hit zero during this function call. + // + // It's okay to read the `f_cred` field without synchronization as `f_cred` is never + // changed after initialization of the file. + let ptr = unsafe { (*self.as_ptr()).f_cred }; + + // SAFETY: The signature of this function ensures that the caller will only access the + // returned credential while the file is still valid, and the C side ensures that the + // credential stays valid at least as long as the file. + unsafe { Credential::from_ptr(ptr) } + } + + /// Returns the flags associated with the file. + /// + /// The flags are a combination of the constants in [`flags`]. + pub fn flags(&self) -> u32 { + // This `read_volatile` is intended to correspond to a READ_ONCE call. + // + // SAFETY: The file is valid because the shared reference guarantees a nonzero refcount. + // + // TODO: Replace with `read_once` when available on the Rust side. + unsafe { core::ptr::addr_of!((*self.as_ptr()).f_flags).read_volatile() } + } +} + +// SAFETY: The type invariants guarantee that `File` is always ref-counted. +unsafe impl AlwaysRefCounted for File { + fn inc_ref(&self) { + // SAFETY: The existence of a shared reference means that the refcount is nonzero. + unsafe { bindings::get_file(self.as_ptr()) }; + } + + unsafe fn dec_ref(obj: ptr::NonNull) { + // SAFETY: The safety requirements guarantee that the refcount is nonzero. + unsafe { bindings::fput(obj.cast().as_ptr()) } + } +} + +/// A file descriptor reservation. +/// +/// This allows the creation of a file descriptor in two steps: first, we reserve a slot for it, +/// then we commit or drop the reservation. The first step may fail (e.g., the current process ran +/// out of available slots), but commit and drop never fail (and are mutually exclusive). +/// +/// Dropping the reservation happens in the destructor of this type. +/// +/// # Invariants +/// +/// The fd stored in this struct must correspond to a reserved file descriptor of the current task. +pub struct FileDescriptorReservation { + fd: u32, + /// Prevent values of this type from being moved to a different task. + /// + /// The `fd_install` and `put_unused_fd` functions assume that the value of `current` is + /// unchanged since the call to `get_unused_fd_flags`. By adding this marker to this type, we + /// prevent it from being moved across task boundaries, which ensures that `current` does not + /// change while this value exists. + _not_send: NotThreadSafe, +} + +impl FileDescriptorReservation { + /// Creates a new file descriptor reservation. + pub fn get_unused_fd_flags(flags: u32) -> Result { + // SAFETY: FFI call, there are no safety requirements on `flags`. + let fd: i32 = unsafe { bindings::get_unused_fd_flags(flags) }; + if fd < 0 { + return Err(Error::from_errno(fd)); + } + Ok(Self { + fd: fd as u32, + _not_send: PhantomData, + }) + } + + /// Returns the file descriptor number that was reserved. + pub fn reserved_fd(&self) -> u32 { + self.fd + } + + /// Commits the reservation. + /// + /// The previously reserved file descriptor is bound to `file`. This method consumes the + /// [`FileDescriptorReservation`], so it will not be usable after this call. + pub fn fd_install(self, file: ARef) { + // SAFETY: `self.fd` was previously returned by `get_unused_fd_flags`, and `file.ptr` is + // guaranteed to have an owned ref count by its type invariants. + unsafe { bindings::fd_install(self.fd, file.0.get()) }; + + // `fd_install` consumes both the file descriptor and the file reference, so we cannot run + // the destructors. + core::mem::forget(self); + core::mem::forget(file); + } +} + +impl Drop for FileDescriptorReservation { + fn drop(&mut self) { + // SAFETY: `self.fd` was returned by a previous call to `get_unused_fd_flags`. + unsafe { bindings::put_unused_fd(self.fd) }; + } +} + +/// Helper used for closing file descriptors in a way that is safe even if the file is currently +/// held using `fdget`. +/// +/// Additional motivation can be found in commit 80cd795630d6 ("binder: fix use-after-free due to +/// ksys_close() during fdget()") and in the comments on `binder_do_fd_close`. +pub struct DeferredFdCloser { + inner: Box, +} + +/// SAFETY: This just holds an allocation with no real content, so there's no safety issue with +/// moving it across threads. +unsafe impl Send for DeferredFdCloser {} +unsafe impl Sync for DeferredFdCloser {} + +#[repr(C)] +struct DeferredFdCloserInner { + twork: mem::MaybeUninit, + file: *mut bindings::file, +} + +impl DeferredFdCloser { + /// Create a new [`DeferredFdCloser`]. + pub fn new() -> Result { + Ok(Self { + inner: Box::try_new(DeferredFdCloserInner { + twork: mem::MaybeUninit::uninit(), + file: core::ptr::null_mut(), + })?, + }) + } + + /// Schedule a task work that closes the file descriptor when this task returns to userspace. + /// + /// Fails if this is called from a context where we cannot run work when returning to + /// userspace. (E.g., from a kthread.) + pub fn close_fd(self, fd: u32) -> Result<(), DeferredFdCloseError> { + use bindings::task_work_notify_mode_TWA_RESUME as TWA_RESUME; + + // In this method, we schedule the task work before closing the file. This is because + // scheduling a task work is fallible, and we need to know whether it will fail before we + // attempt to close the file. + + // SAFETY: Getting a pointer to current is always safe. + let current = unsafe { bindings::get_current() }; + + // SAFETY: Accessing the `flags` field of `current` is always safe. + let is_kthread = (unsafe { (*current).flags } & bindings::PF_KTHREAD) != 0; + if is_kthread { + return Err(DeferredFdCloseError::TaskWorkUnavailable); + } + + // This disables the destructor of the box, so the allocation is not cleaned up + // automatically below. + let inner = Box::into_raw(self.inner); + + // The `callback_head` field is first in the struct, so this cast correctly gives us a + // pointer to the field. + let callback_head = inner.cast::(); + // SAFETY: This pointer offset operation does not go out-of-bounds. + let file_field = unsafe { core::ptr::addr_of_mut!((*inner).file) }; + + // SAFETY: The `callback_head` pointer is compatible with the `do_close_fd` method. + unsafe { bindings::init_task_work(callback_head, Some(Self::do_close_fd)) }; + // SAFETY: The `callback_head` pointer points at a valid and fully initialized task work + // that is ready to be scheduled. + // + // If the task work gets scheduled as-is, then it will be a no-op. However, we will update + // the file pointer below to tell it which file to fput. + let res = unsafe { bindings::task_work_add(current, callback_head, TWA_RESUME) }; + + if res != 0 { + // SAFETY: Scheduling the task work failed, so we still have ownership of the box, so + // we may destroy it. + unsafe { drop(Box::from_raw(inner)) }; + + return Err(DeferredFdCloseError::TaskWorkUnavailable); + } + + // SAFETY: Just an FFI call. This is safe no matter what `fd` is. + let file = unsafe { bindings::close_fd_get_file(fd) }; + if file.is_null() { + // We don't clean up the task work since that might be expensive if the task work queue + // is long. Just let it execute and let it clean up for itself. + return Err(DeferredFdCloseError::BadFd); + } + + // SAFETY: The `file` pointer points at a valid file. + unsafe { bindings::get_file(file) }; + + // SAFETY: Due to the above `get_file`, even if the current task holds an `fdget` to + // this file right now, the refcount will not drop to zero until after it is released + // with `fdput`. This is because when using `fdget`, you must always use `fdput` before + // returning to userspace, and our task work runs after any `fdget` users have returned + // to userspace. + // + // Note: fl_owner_t is currently a void pointer. + unsafe { bindings::filp_close(file, (*current).files as bindings::fl_owner_t) }; + + // We update the file pointer that the task work is supposed to fput. + // + // SAFETY: Task works are executed on the current thread once we return to userspace, so + // this write is guaranteed to happen before `do_close_fd` is called, which means that a + // race is not possible here. + // + // It's okay to pass this pointer to the task work, since we just acquired a refcount with + // the previous call to `get_file`. Furthermore, the refcount will not drop to zero during + // an `fdget` call, since we defer the `fput` until after returning to userspace. + unsafe { *file_field = file }; + + Ok(()) + } + + // SAFETY: This function is an implementation detail of `close_fd`, so its safety comments + // should be read in extension of that method. + unsafe extern "C" fn do_close_fd(inner: *mut bindings::callback_head) { + // SAFETY: In `close_fd` we use this method together with a pointer that originates from a + // `Box`, and we have just been given ownership of that allocation. + let inner = unsafe { Box::from_raw(inner as *mut DeferredFdCloserInner) }; + if !inner.file.is_null() { + // SAFETY: This drops a refcount we acquired in `close_fd`. Since this callback runs in + // a task work after we return to userspace, it is guaranteed that the current thread + // doesn't hold this file with `fdget`, as `fdget` must be released before returning to + // userspace. + unsafe { bindings::fput(inner.file) }; + } + // Free the allocation. + drop(inner); + } +} + +/// Represents a failure to close an fd in a deferred manner. +#[derive(Copy, Clone, Eq, PartialEq)] +pub enum DeferredFdCloseError { + /// Closing the fd failed because we were unable to schedule a task work. + TaskWorkUnavailable, + /// Closing the fd failed because the fd does not exist. + BadFd, +} + +impl From for Error { + fn from(err: DeferredFdCloseError) -> Error { + match err { + DeferredFdCloseError::TaskWorkUnavailable => ESRCH, + DeferredFdCloseError::BadFd => EBADF, + } + } +} + +impl core::fmt::Debug for DeferredFdCloseError { + fn fmt(&self, f: &mut core::fmt::Formatter<'_>) -> core::fmt::Result { + Error::from(*self).fmt(f) + } +} + +/// Represents the `EBADF` error code. +/// +/// Used for methods that can only fail with `EBADF`. +#[derive(Copy, Clone, Eq, PartialEq)] +pub struct BadFdError; + +impl From for Error { + fn from(_: BadFdError) -> Error { + EBADF + } +} + +impl core::fmt::Debug for BadFdError { + fn fmt(&self, f: &mut core::fmt::Formatter<'_>) -> core::fmt::Result { + f.pad("EBADF") + } +} diff --git a/rust/kernel/init/macros.rs b/rust/kernel/init/macros.rs index cb6e61b6c50bda..624e9108e3b423 100644 --- a/rust/kernel/init/macros.rs +++ b/rust/kernel/init/macros.rs @@ -538,6 +538,7 @@ macro_rules! __pin_data { ), @impl_generics($($impl_generics:tt)*), @ty_generics($($ty_generics:tt)*), + @decl_generics($($decl_generics:tt)*), @body({ $($fields:tt)* }), ) => { // We now use token munching to iterate through all of the fields. While doing this we @@ -560,6 +561,9 @@ macro_rules! __pin_data { @impl_generics($($impl_generics)*), // The 'ty generics', the generics that will need to be specified on the impl blocks. @ty_generics($($ty_generics)*), + // The 'decl generics', the generics that need to be specified on the struct + // definition. + @decl_generics($($decl_generics)*), // The where clause of any impl block and the declaration. @where($($($whr)*)?), // The remaining fields tokens that need to be processed. @@ -585,6 +589,7 @@ macro_rules! __pin_data { @name($name:ident), @impl_generics($($impl_generics:tt)*), @ty_generics($($ty_generics:tt)*), + @decl_generics($($decl_generics:tt)*), @where($($whr:tt)*), // We found a PhantomPinned field, this should generally be pinned! @fields_munch($field:ident : $($($(::)?core::)?marker::)?PhantomPinned, $($rest:tt)*), @@ -607,6 +612,7 @@ macro_rules! __pin_data { @name($name), @impl_generics($($impl_generics)*), @ty_generics($($ty_generics)*), + @decl_generics($($decl_generics)*), @where($($whr)*), @fields_munch($($rest)*), @pinned($($pinned)* $($accum)* $field: ::core::marker::PhantomPinned,), @@ -623,6 +629,7 @@ macro_rules! __pin_data { @name($name:ident), @impl_generics($($impl_generics:tt)*), @ty_generics($($ty_generics:tt)*), + @decl_generics($($decl_generics:tt)*), @where($($whr:tt)*), // We reached the field declaration. @fields_munch($field:ident : $type:ty, $($rest:tt)*), @@ -640,6 +647,7 @@ macro_rules! __pin_data { @name($name), @impl_generics($($impl_generics)*), @ty_generics($($ty_generics)*), + @decl_generics($($decl_generics)*), @where($($whr)*), @fields_munch($($rest)*), @pinned($($pinned)* $($accum)* $field: $type,), @@ -656,6 +664,7 @@ macro_rules! __pin_data { @name($name:ident), @impl_generics($($impl_generics:tt)*), @ty_generics($($ty_generics:tt)*), + @decl_generics($($decl_generics:tt)*), @where($($whr:tt)*), // We reached the field declaration. @fields_munch($field:ident : $type:ty, $($rest:tt)*), @@ -673,6 +682,7 @@ macro_rules! __pin_data { @name($name), @impl_generics($($impl_generics)*), @ty_generics($($ty_generics)*), + @decl_generics($($decl_generics)*), @where($($whr)*), @fields_munch($($rest)*), @pinned($($pinned)*), @@ -689,6 +699,7 @@ macro_rules! __pin_data { @name($name:ident), @impl_generics($($impl_generics:tt)*), @ty_generics($($ty_generics:tt)*), + @decl_generics($($decl_generics:tt)*), @where($($whr:tt)*), // We found the `#[pin]` attr. @fields_munch(#[pin] $($rest:tt)*), @@ -705,6 +716,7 @@ macro_rules! __pin_data { @name($name), @impl_generics($($impl_generics)*), @ty_generics($($ty_generics)*), + @decl_generics($($decl_generics)*), @where($($whr)*), @fields_munch($($rest)*), // We do not include `#[pin]` in the list of attributes, since it is not actually an @@ -724,6 +736,7 @@ macro_rules! __pin_data { @name($name:ident), @impl_generics($($impl_generics:tt)*), @ty_generics($($ty_generics:tt)*), + @decl_generics($($decl_generics:tt)*), @where($($whr:tt)*), // We reached the field declaration with visibility, for simplicity we only munch the // visibility and put it into `$accum`. @@ -741,6 +754,7 @@ macro_rules! __pin_data { @name($name), @impl_generics($($impl_generics)*), @ty_generics($($ty_generics)*), + @decl_generics($($decl_generics)*), @where($($whr)*), @fields_munch($field $($rest)*), @pinned($($pinned)*), @@ -757,6 +771,7 @@ macro_rules! __pin_data { @name($name:ident), @impl_generics($($impl_generics:tt)*), @ty_generics($($ty_generics:tt)*), + @decl_generics($($decl_generics:tt)*), @where($($whr:tt)*), // Some other attribute, just put it into `$accum`. @fields_munch(#[$($attr:tt)*] $($rest:tt)*), @@ -773,6 +788,7 @@ macro_rules! __pin_data { @name($name), @impl_generics($($impl_generics)*), @ty_generics($($ty_generics)*), + @decl_generics($($decl_generics)*), @where($($whr)*), @fields_munch($($rest)*), @pinned($($pinned)*), @@ -789,6 +805,7 @@ macro_rules! __pin_data { @name($name:ident), @impl_generics($($impl_generics:tt)*), @ty_generics($($ty_generics:tt)*), + @decl_generics($($decl_generics:tt)*), @where($($whr:tt)*), // We reached the end of the fields, plus an optional additional comma, since we added one // before and the user is also allowed to put a trailing comma. @@ -802,7 +819,7 @@ macro_rules! __pin_data { ) => { // Declare the struct with all fields in the correct order. $($struct_attrs)* - $vis struct $name <$($impl_generics)*> + $vis struct $name <$($decl_generics)*> where $($whr)* { $($fields)* diff --git a/rust/kernel/ioctl.rs b/rust/kernel/ioctl.rs index c49e1a8d3fd075..f1d42ab6997278 100644 --- a/rust/kernel/ioctl.rs +++ b/rust/kernel/ioctl.rs @@ -2,7 +2,7 @@ //! ioctl() number definitions //! -//! C header: [`include/asm-generic/ioctl.h`](../../../../include/asm-generic/ioctl.h) +//! C header: [`include/asm-generic/ioctl.h`](srctree/include/asm-generic/ioctl.h) #![allow(non_snake_case)] diff --git a/rust/kernel/kunit.rs b/rust/kernel/kunit.rs index 722655b2d62df7..0ba77276ae7ef2 100644 --- a/rust/kernel/kunit.rs +++ b/rust/kernel/kunit.rs @@ -2,7 +2,7 @@ //! KUnit-based macros for Rust unit tests. //! -//! C header: [`include/kunit/test.h`](../../../../../include/kunit/test.h) +//! C header: [`include/kunit/test.h`](srctree/include/kunit/test.h) //! //! Reference: diff --git a/rust/kernel/lib.rs b/rust/kernel/lib.rs index 7ac39874aeac36..d55d065642f059 100644 --- a/rust/kernel/lib.rs +++ b/rust/kernel/lib.rs @@ -14,7 +14,6 @@ #![no_std] #![feature(allocator_api)] #![feature(coerce_unsized)] -#![feature(const_maybe_uninit_zeroed)] #![feature(dispatch_from_dyn)] #![feature(new_uninit)] #![feature(offset_of)] @@ -34,7 +33,9 @@ extern crate self as kernel; #[cfg(not(testlib))] mod allocator; mod build_assert; +pub mod cred; pub mod error; +pub mod file; pub mod init; pub mod ioctl; #[cfg(CONFIG_KUNIT)] @@ -43,12 +44,14 @@ pub mod kunit; pub mod net; pub mod prelude; pub mod print; +pub mod security; mod static_assert; #[doc(hidden)] pub mod std_vendor; pub mod str; pub mod sync; pub mod task; +pub mod time; pub mod types; pub mod workqueue; diff --git a/rust/kernel/print.rs b/rust/kernel/print.rs index f48926e3e9fe32..9b13aca832c2df 100644 --- a/rust/kernel/print.rs +++ b/rust/kernel/print.rs @@ -2,7 +2,7 @@ //! Printing facilities. //! -//! C header: [`include/linux/printk.h`](../../../../include/linux/printk.h) +//! C header: [`include/linux/printk.h`](srctree/include/linux/printk.h) //! //! Reference: @@ -48,7 +48,7 @@ pub mod format_strings { /// The format string is always the same for a given level, i.e. for a /// given `prefix`, which are the kernel's `KERN_*` constants. /// - /// [`_printk`]: ../../../../include/linux/printk.h + /// [`_printk`]: srctree/include/linux/printk.h const fn generate(is_cont: bool, prefix: &[u8; 3]) -> [u8; LENGTH] { // Ensure the `KERN_*` macros are what we expect. assert!(prefix[0] == b'\x01'); @@ -97,7 +97,7 @@ pub mod format_strings { /// The format string must be one of the ones in [`format_strings`], and /// the module name must be null-terminated. /// -/// [`_printk`]: ../../../../include/linux/_printk.h +/// [`_printk`]: srctree/include/linux/_printk.h #[doc(hidden)] #[cfg_attr(not(CONFIG_PRINTK), allow(unused_variables))] pub unsafe fn call_printk( @@ -120,7 +120,7 @@ pub unsafe fn call_printk( /// /// Public but hidden since it should only be used from public macros. /// -/// [`_printk`]: ../../../../include/linux/printk.h +/// [`_printk`]: srctree/include/linux/printk.h #[doc(hidden)] #[cfg_attr(not(CONFIG_PRINTK), allow(unused_variables))] pub fn call_printk_cont(args: fmt::Arguments<'_>) { diff --git a/rust/kernel/security.rs b/rust/kernel/security.rs new file mode 100644 index 00000000000000..6545bfa2fd72e9 --- /dev/null +++ b/rust/kernel/security.rs @@ -0,0 +1,79 @@ +// SPDX-License-Identifier: GPL-2.0 + +//! Linux Security Modules (LSM). +//! +//! C header: [`include/linux/security.h`](../../../../include/linux/security.h). + +use crate::{ + bindings, + error::{to_result, Result}, +}; + +/// A security context string. +/// +/// # Invariants +/// +/// The `secdata` and `seclen` fields correspond to a valid security context as returned by a +/// successful call to `security_secid_to_secctx`, that has not yet been destroyed by calling +/// `security_release_secctx`. +pub struct SecurityCtx { + secdata: *mut core::ffi::c_char, + seclen: usize, +} + +impl SecurityCtx { + /// Get the security context given its id. + pub fn from_secid(secid: u32) -> Result { + let mut secdata = core::ptr::null_mut(); + let mut seclen = 0u32; + // SAFETY: Just a C FFI call. The pointers are valid for writes. + unsafe { + to_result(bindings::security_secid_to_secctx( + secid, + &mut secdata, + &mut seclen, + ))?; + } + + // INVARIANT: If the above call did not fail, then we have a valid security context. + Ok(Self { + secdata, + seclen: seclen as usize, + }) + } + + /// Returns whether the security context is empty. + pub fn is_empty(&self) -> bool { + self.seclen == 0 + } + + /// Returns the length of this security context. + pub fn len(&self) -> usize { + self.seclen + } + + /// Returns the bytes for this security context. + pub fn as_bytes(&self) -> &[u8] { + let ptr = self.secdata; + if ptr.is_null() { + // We can't pass a null pointer to `slice::from_raw_parts` even if the length is zero. + debug_assert_eq!(self.seclen, 0); + return &[]; + } + + // SAFETY: The call to `security_secid_to_secctx` guarantees that the pointer is valid for + // `seclen` bytes. Furthermore, if the length is zero, then we have ensured that the + // pointer is not null. + unsafe { core::slice::from_raw_parts(ptr.cast(), self.seclen) } + } +} + +impl Drop for SecurityCtx { + fn drop(&mut self) { + // SAFETY: This frees a pointer that came from a successful call to + // `security_secid_to_secctx` and has not yet been destroyed by `security_release_secctx`. + unsafe { + bindings::security_release_secctx(self.secdata, self.seclen as u32); + } + } +} diff --git a/rust/kernel/str.rs b/rust/kernel/str.rs index c41607b2e4fe93..7d848b83add482 100644 --- a/rust/kernel/str.rs +++ b/rust/kernel/str.rs @@ -608,6 +608,12 @@ impl<'a> TryFrom<&'a CStr> for CString { } } +impl fmt::Debug for CString { + fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result { + fmt::Debug::fmt(&**self, f) + } +} + /// A convenience alias for [`core::format_args`]. #[macro_export] macro_rules! fmt { diff --git a/rust/kernel/sync.rs b/rust/kernel/sync.rs index d219ee518eff15..84726f80c4069c 100644 --- a/rust/kernel/sync.rs +++ b/rust/kernel/sync.rs @@ -11,6 +11,7 @@ mod arc; mod condvar; pub mod lock; mod locked_by; +pub mod poll; pub use arc::{Arc, ArcBorrow, UniqueArc}; pub use condvar::CondVar; diff --git a/rust/kernel/sync/arc.rs b/rust/kernel/sync/arc.rs index 77cdbcf7bd2eba..97e7146a7b1fa0 100644 --- a/rust/kernel/sync/arc.rs +++ b/rust/kernel/sync/arc.rs @@ -277,26 +277,35 @@ impl Arc { impl ForeignOwnable for Arc { type Borrowed<'a> = ArcBorrow<'a, T>; + // Mutable access to the `Arc` does not give any extra abilities over + // immutable access. + type BorrowedMut<'a> = ArcBorrow<'a, T>; fn into_foreign(self) -> *const core::ffi::c_void { ManuallyDrop::new(self).ptr.as_ptr() as _ } + unsafe fn from_foreign(ptr: *const core::ffi::c_void) -> Self { + // SAFETY: By the safety requirement of this function, we know that `ptr` came from + // a previous call to `Arc::into_foreign`, which guarantees that `ptr` is valid and + // holds a reference count increment that is transferrable to us. + unsafe { Self::from_inner(NonNull::new_unchecked(ptr as _)) } + } + unsafe fn borrow<'a>(ptr: *const core::ffi::c_void) -> ArcBorrow<'a, T> { // SAFETY: By the safety requirement of this function, we know that `ptr` came from // a previous call to `Arc::into_foreign`. - let inner = NonNull::new(ptr as *mut ArcInner).unwrap(); + let inner = unsafe { NonNull::new_unchecked(ptr as *mut ArcInner) }; - // SAFETY: The safety requirements of `from_foreign` ensure that the object remains alive - // for the lifetime of the returned value. + // SAFETY: The safety requirements ensure that we will not give up our + // foreign-owned refcount while the `ArcBorrow` is still live. unsafe { ArcBorrow::new(inner) } } - unsafe fn from_foreign(ptr: *const core::ffi::c_void) -> Self { - // SAFETY: By the safety requirement of this function, we know that `ptr` came from - // a previous call to `Arc::into_foreign`, which guarantees that `ptr` is valid and - // holds a reference count increment that is transferrable to us. - unsafe { Self::from_inner(NonNull::new(ptr as _).unwrap()) } + unsafe fn borrow_mut<'a>(ptr: *const core::ffi::c_void) -> ArcBorrow<'a, T> { + // SAFETY: The safety requirements for `borrow_mut` are a superset of the safety + // requirements for `borrow`. + unsafe { Self::borrow(ptr) } } } diff --git a/rust/kernel/sync/condvar.rs b/rust/kernel/sync/condvar.rs index b679b6f6dbebc0..7f2b78e4abc71a 100644 --- a/rust/kernel/sync/condvar.rs +++ b/rust/kernel/sync/condvar.rs @@ -6,8 +6,18 @@ //! variable. use super::{lock::Backend, lock::Guard, LockClassKey}; -use crate::{bindings, init::PinInit, pin_init, str::CStr, types::Opaque}; +use crate::{ + bindings, + init::PinInit, + pin_init, + str::CStr, + task::{MAX_SCHEDULE_TIMEOUT, TASK_INTERRUPTIBLE, TASK_NORMAL, TASK_UNINTERRUPTIBLE}, + time::Jiffies, + types::Opaque, +}; +use core::ffi::{c_int, c_long}; use core::marker::PhantomPinned; +use core::ptr; use macros::pin_data; /// Creates a [`CondVar`] initialiser with the given name and a newly-created lock class. @@ -50,7 +60,7 @@ macro_rules! new_condvar { /// fn wait_for_value(e: &Example, v: u32) { /// let mut guard = e.value.lock(); /// while *guard != v { -/// e.value_changed.wait_uninterruptible(&mut guard); +/// e.value_changed.wait(&mut guard); /// } /// } /// @@ -69,7 +79,7 @@ macro_rules! new_condvar { /// } /// ``` /// -/// [`struct wait_queue_head`]: ../../../include/linux/wait.h +/// [`struct wait_queue_head`]: srctree/include/linux/wait.h #[pin_data] pub struct CondVar { #[pin] @@ -102,7 +112,12 @@ impl CondVar { }) } - fn wait_internal(&self, wait_state: u32, guard: &mut Guard<'_, T, B>) { + fn wait_internal( + &self, + wait_state: c_int, + guard: &mut Guard<'_, T, B>, + timeout_in_jiffies: c_long, + ) -> c_long { let wait = Opaque::::uninit(); // SAFETY: `wait` points to valid memory. @@ -110,49 +125,75 @@ impl CondVar { // SAFETY: Both `wait` and `wait_list` point to valid memory. unsafe { - bindings::prepare_to_wait_exclusive(self.wait_list.get(), wait.get(), wait_state as _) + bindings::prepare_to_wait_exclusive(self.wait_list.get(), wait.get(), wait_state) }; - // SAFETY: No arguments, switches to another thread. - guard.do_unlocked(|| unsafe { bindings::schedule() }); + // SAFETY: Switches to another thread. The timeout can be any number. + let ret = guard.do_unlocked(|| unsafe { bindings::schedule_timeout(timeout_in_jiffies) }); // SAFETY: Both `wait` and `wait_list` point to valid memory. unsafe { bindings::finish_wait(self.wait_list.get(), wait.get()) }; + + ret } - /// Releases the lock and waits for a notification in interruptible mode. + /// Releases the lock and waits for a notification in uninterruptible mode. /// /// Atomically releases the given lock (whose ownership is proven by the guard) and puts the /// thread to sleep, reacquiring the lock on wake up. It wakes up when notified by - /// [`CondVar::notify_one`] or [`CondVar::notify_all`], or when the thread receives a signal. - /// It may also wake up spuriously. + /// [`CondVar::notify_one`] or [`CondVar::notify_all`]. Note that it may also wake up + /// spuriously. + pub fn wait(&self, guard: &mut Guard<'_, T, B>) { + self.wait_internal(TASK_UNINTERRUPTIBLE, guard, MAX_SCHEDULE_TIMEOUT); + } + + /// Releases the lock and waits for a notification in interruptible mode. + /// + /// Similar to [`CondVar::wait`], except that the wait is interruptible. That is, the thread may + /// wake up due to signals. It may also wake up spuriously. /// /// Returns whether there is a signal pending. - #[must_use = "wait returns if a signal is pending, so the caller must check the return value"] - pub fn wait(&self, guard: &mut Guard<'_, T, B>) -> bool { - self.wait_internal(bindings::TASK_INTERRUPTIBLE, guard); + #[must_use = "wait_interruptible returns if a signal is pending, so the caller must check the return value"] + pub fn wait_interruptible(&self, guard: &mut Guard<'_, T, B>) -> bool { + self.wait_internal(TASK_INTERRUPTIBLE, guard, MAX_SCHEDULE_TIMEOUT); crate::current!().signal_pending() } - /// Releases the lock and waits for a notification in uninterruptible mode. + /// Releases the lock and waits for a notification in interruptible mode. /// - /// Similar to [`CondVar::wait`], except that the wait is not interruptible. That is, the - /// thread won't wake up due to signals. It may, however, wake up supirously. - pub fn wait_uninterruptible(&self, guard: &mut Guard<'_, T, B>) { - self.wait_internal(bindings::TASK_UNINTERRUPTIBLE, guard) + /// Atomically releases the given lock (whose ownership is proven by the guard) and puts the + /// thread to sleep. It wakes up when notified by [`CondVar::notify_one`] or + /// [`CondVar::notify_all`], or when a timeout occurs, or when the thread receives a signal. + #[must_use = "wait_interruptible_timeout returns if a signal is pending, so the caller must check the return value"] + pub fn wait_interruptible_timeout( + &self, + guard: &mut Guard<'_, T, B>, + jiffies: Jiffies, + ) -> CondVarTimeoutResult { + let jiffies = jiffies.try_into().unwrap_or(MAX_SCHEDULE_TIMEOUT); + let res = self.wait_internal(TASK_INTERRUPTIBLE, guard, jiffies); + + match (res as Jiffies, crate::current!().signal_pending()) { + (jiffies, true) => CondVarTimeoutResult::Signal { jiffies }, + (0, false) => CondVarTimeoutResult::Timeout, + (jiffies, false) => CondVarTimeoutResult::Woken { jiffies }, + } } - /// Calls the kernel function to notify the appropriate number of threads with the given flags. - fn notify(&self, count: i32, flags: u32) { + /// Calls the kernel function to notify the appropriate number of threads. + fn notify(&self, count: c_int) { // SAFETY: `wait_list` points to valid memory. - unsafe { - bindings::__wake_up( - self.wait_list.get(), - bindings::TASK_NORMAL, - count, - flags as _, - ) - }; + unsafe { bindings::__wake_up(self.wait_list.get(), TASK_NORMAL, count, ptr::null_mut()) }; + } + + /// Calls the kernel function to notify one thread synchronously. + /// + /// This method behaves like `notify_one`, except that it hints to the scheduler that the + /// current thread is about to go to sleep, so it should schedule the target thread on the same + /// CPU. + pub fn notify_sync(&self) { + // SAFETY: `wait_list` points to valid memory. + unsafe { bindings::__wake_up_sync(self.wait_list.get(), TASK_NORMAL) }; } /// Wakes a single waiter up, if any. @@ -160,7 +201,7 @@ impl CondVar { /// This is not 'sticky' in the sense that if no thread is waiting, the notification is lost /// completely (as opposed to automatically waking up the next waiter). pub fn notify_one(&self) { - self.notify(1, 0); + self.notify(1); } /// Wakes all waiters up, if any. @@ -168,6 +209,22 @@ impl CondVar { /// This is not 'sticky' in the sense that if no thread is waiting, the notification is lost /// completely (as opposed to automatically waking up the next waiter). pub fn notify_all(&self) { - self.notify(0, 0); + self.notify(0); } } + +/// The return type of `wait_timeout`. +pub enum CondVarTimeoutResult { + /// The timeout was reached. + Timeout, + /// Somebody woke us up. + Woken { + /// Remaining sleep duration. + jiffies: Jiffies, + }, + /// A signal occurred. + Signal { + /// Remaining sleep duration. + jiffies: Jiffies, + }, +} diff --git a/rust/kernel/sync/lock.rs b/rust/kernel/sync/lock.rs index f12a684bc95794..149a5259d431a8 100644 --- a/rust/kernel/sync/lock.rs +++ b/rust/kernel/sync/lock.rs @@ -139,7 +139,7 @@ pub struct Guard<'a, T: ?Sized, B: Backend> { unsafe impl Sync for Guard<'_, T, B> {} impl Guard<'_, T, B> { - pub(crate) fn do_unlocked(&mut self, cb: impl FnOnce()) { + pub(crate) fn do_unlocked(&mut self, cb: impl FnOnce() -> U) -> U { // SAFETY: The caller owns the lock, so it is safe to unlock it. unsafe { B::unlock(self.lock.state.get(), &self.state) }; @@ -147,7 +147,7 @@ impl Guard<'_, T, B> { let _relock = ScopeGuard::new(|| unsafe { B::relock(self.lock.state.get(), &mut self.state) }); - cb(); + cb() } } diff --git a/rust/kernel/sync/lock/mutex.rs b/rust/kernel/sync/lock/mutex.rs index 09276fedc091b8..8c524a3ec45af1 100644 --- a/rust/kernel/sync/lock/mutex.rs +++ b/rust/kernel/sync/lock/mutex.rs @@ -84,7 +84,7 @@ macro_rules! new_mutex { /// } /// ``` /// -/// [`struct mutex`]: ../../../../include/linux/mutex.h +/// [`struct mutex`]: srctree/include/linux/mutex.h pub type Mutex = super::Lock; /// A kernel `struct mutex` lock backend. diff --git a/rust/kernel/sync/lock/spinlock.rs b/rust/kernel/sync/lock/spinlock.rs index 91eb2c9e9123f7..068535ce1b29f9 100644 --- a/rust/kernel/sync/lock/spinlock.rs +++ b/rust/kernel/sync/lock/spinlock.rs @@ -82,7 +82,7 @@ macro_rules! new_spinlock { /// } /// ``` /// -/// [`spinlock_t`]: ../../../../include/linux/spinlock.h +/// [`spinlock_t`]: srctree/include/linux/spinlock.h pub type SpinLock = super::Lock; /// A kernel `spinlock_t` lock backend. diff --git a/rust/kernel/sync/poll.rs b/rust/kernel/sync/poll.rs new file mode 100644 index 00000000000000..e1dded9b7b9d69 --- /dev/null +++ b/rust/kernel/sync/poll.rs @@ -0,0 +1,103 @@ +// SPDX-License-Identifier: GPL-2.0 + +//! Utilities for working with `struct poll_table`. + +use crate::{ + bindings, + file::File, + prelude::*, + sync::{CondVar, LockClassKey}, + types::Opaque, +}; +use core::ops::Deref; + +/// Creates a [`PollCondVar`] initialiser with the given name and a newly-created lock class. +#[macro_export] +macro_rules! new_poll_condvar { + ($($name:literal)?) => { + $crate::file::PollCondVar::new($crate::optional_name!($($name)?), $crate::static_lock_class!()) + }; +} + +/// Wraps the kernel's `struct poll_table`. +#[repr(transparent)] +pub struct PollTable(Opaque); + +impl PollTable { + /// Creates a reference to a [`PollTable`] from a valid pointer. + /// + /// # Safety + /// + /// The caller must ensure that for the duration of 'a, the pointer will point at a valid poll + /// table, and that it is only accessed via the returned reference. + pub unsafe fn from_ptr<'a>(ptr: *mut bindings::poll_table) -> &'a mut PollTable { + // SAFETY: The safety requirements guarantee the validity of the dereference, while the + // `PollTable` type being transparent makes the cast ok. + unsafe { &mut *ptr.cast() } + } + + fn get_qproc(&self) -> bindings::poll_queue_proc { + let ptr = self.0.get(); + // SAFETY: The `ptr` is valid because it originates from a reference, and the `_qproc` + // field is not modified concurrently with this call since we have an immutable reference. + unsafe { (*ptr)._qproc } + } + + /// Register this [`PollTable`] with the provided [`PollCondVar`], so that it can be notified + /// using the condition variable. + pub fn register_wait(&mut self, file: &File, cv: &PollCondVar) { + if let Some(qproc) = self.get_qproc() { + // SAFETY: The pointers to `self` and `file` are valid because they are references. + // + // Before the wait list is destroyed, the destructor of `PollCondVar` will clear + // everything in the wait list, so the wait list is not used after it is freed. + unsafe { qproc(file.as_ptr() as _, cv.wait_list.get(), self.0.get()) }; + } + } +} + +/// A wrapper around [`CondVar`] that makes it usable with [`PollTable`]. +/// +/// # Invariant +/// +/// If `needs_synchronize_rcu` is false, then there is nothing registered with `register_wait`. +/// +/// [`CondVar`]: crate::sync::CondVar +#[pin_data(PinnedDrop)] +pub struct PollCondVar { + #[pin] + inner: CondVar, +} + +impl PollCondVar { + /// Constructs a new condvar initialiser. + pub fn new(name: &'static CStr, key: &'static LockClassKey) -> impl PinInit { + pin_init!(Self { + inner <- CondVar::new(name, key), + }) + } +} + +// Make the `CondVar` methods callable on `PollCondVar`. +impl Deref for PollCondVar { + type Target = CondVar; + + fn deref(&self) -> &CondVar { + &self.inner + } +} + +#[pinned_drop] +impl PinnedDrop for PollCondVar { + fn drop(self: Pin<&mut Self>) { + // Clear anything registered using `register_wait`. + // + // SAFETY: The pointer points at a valid wait list. + unsafe { bindings::__wake_up_pollfree(self.inner.wait_list.get()) }; + + // Wait for epoll items to be properly removed. + // + // SAFETY: Just an FFI call. + unsafe { bindings::synchronize_rcu() }; + } +} diff --git a/rust/kernel/task.rs b/rust/kernel/task.rs index b2299bc7ac1ff5..6b8c2f3d7d58b9 100644 --- a/rust/kernel/task.rs +++ b/rust/kernel/task.rs @@ -2,10 +2,27 @@ //! Tasks (threads and processes). //! -//! C header: [`include/linux/sched.h`](../../../../include/linux/sched.h). +//! C header: [`include/linux/sched.h`](srctree/include/linux/sched.h). use crate::{bindings, types::Opaque}; -use core::{marker::PhantomData, ops::Deref, ptr}; +use core::{ + ffi::{c_int, c_long, c_uint}, + cmp::{Eq, PartialEq}, + marker::PhantomData, + ops::Deref, + ptr, +}; + +/// A sentinal value used for infinite timeouts. +pub const MAX_SCHEDULE_TIMEOUT: c_long = c_long::MAX; + +/// Bitmask for tasks that are sleeping in an interruptible state. +pub const TASK_INTERRUPTIBLE: c_int = bindings::TASK_INTERRUPTIBLE as c_int; +/// Bitmask for tasks that are sleeping in an uninterruptible state. +pub const TASK_UNINTERRUPTIBLE: c_int = bindings::TASK_UNINTERRUPTIBLE as c_int; +/// Convenience constant for waking up tasks regardless of whether they are in interruptible or +/// uninterruptible sleep. +pub const TASK_NORMAL: c_uint = bindings::TASK_NORMAL as c_uint; /// Returns the currently running task. #[macro_export] @@ -78,6 +95,12 @@ unsafe impl Sync for Task {} /// The type of process identifiers (PIDs). type Pid = bindings::pid_t; +/// The type of user identifiers (UIDs). +#[derive(Copy, Clone)] +pub struct Kuid { + kuid: bindings::kuid_t, +} + impl Task { /// Returns a task reference for the currently executing task/thread. /// @@ -113,11 +136,16 @@ impl Task { } } + /// Returns a raw pointer to the underlying C task struct. + pub fn as_raw(&self) -> *mut bindings::task_struct { + self.0.get() + } + /// Returns the group leader of the given task. pub fn group_leader(&self) -> &Task { - // SAFETY: By the type invariant, we know that `self.0` is a valid task. Valid tasks always + // SAFETY: By the type invariant, we know that `self.as_raw()` is a valid task. Valid tasks always // have a valid group_leader. - let ptr = unsafe { *ptr::addr_of!((*self.0.get()).group_leader) }; + let ptr = unsafe { *ptr::addr_of!((*self.as_raw()).group_leader) }; // SAFETY: The lifetime of the returned task reference is tied to the lifetime of `self`, // and given that a task has a reference to its group leader, we know it must be valid for @@ -127,26 +155,135 @@ impl Task { /// Returns the PID of the given task. pub fn pid(&self) -> Pid { - // SAFETY: By the type invariant, we know that `self.0` is a valid task. Valid tasks always + // SAFETY: By the type invariant, we know that `self.as_raw()` is a valid task. Valid tasks always // have a valid pid. - unsafe { *ptr::addr_of!((*self.0.get()).pid) } + unsafe { *ptr::addr_of!((*self.as_raw()).pid) } + } + + /// Returns the UID of the given task. + pub fn uid(&self) -> Kuid { + // SAFETY: By the type invariant, we know that `self.as_raw()` is valid. + Kuid::from_raw(unsafe { bindings::task_uid(self.as_raw()) }) + } + + /// Returns the effective UID of the given task. + pub fn euid(&self) -> Kuid { + // SAFETY: By the type invariant, we know that `self.as_raw()` is valid. + Kuid::from_raw(unsafe { bindings::task_euid(self.as_raw()) }) + } + + /// Returns the UID of the given task. + pub fn uid(&self) -> Kuid { + // SAFETY: By the type invariant, we know that `self.0` is valid. + Kuid::from_raw(unsafe { bindings::task_uid(self.0.get()) }) + } + + /// Returns the effective UID of the given task. + pub fn euid(&self) -> Kuid { + // SAFETY: By the type invariant, we know that `self.0` is valid. + Kuid::from_raw(unsafe { bindings::task_euid(self.0.get()) }) } /// Determines whether the given task has pending signals. pub fn signal_pending(&self) -> bool { - // SAFETY: By the type invariant, we know that `self.0` is valid. - unsafe { bindings::signal_pending(self.0.get()) != 0 } + // SAFETY: By the type invariant, we know that `self.as_raw()` is valid. + unsafe { bindings::signal_pending(self.as_raw()) != 0 } + } + + /// Returns the given task's pid in the current pid namespace. + pub fn pid_in_current_ns(&self) -> Pid { + let current = current!(); + // SAFETY: Calling `task_active_pid_ns` with the current task is always safe. + let namespace = unsafe { bindings::task_active_pid_ns(current.as_raw()) }; + // SAFETY: We know that `self.raw()` is valid by the type invariant. + unsafe { bindings::task_tgid_nr_ns(self.as_raw(), namespace) } + } + + /// Returns the given task's pid in the current pid namespace. + pub fn pid_in_current_ns(&self) -> Pid { + // SAFETY: Calling `task_active_pid_ns` with the current task is always safe. + let namespace = unsafe { bindings::task_active_pid_ns(bindings::get_current()) }; + // SAFETY: We know that `self.0.get()` is valid by the type invariant. + unsafe { bindings::task_tgid_nr_ns(self.0.get(), namespace) } } /// Wakes up the task. pub fn wake_up(&self) { - // SAFETY: By the type invariant, we know that `self.0.get()` is non-null and valid. + // SAFETY: By the type invariant, we know that `self.raw()` is non-null and valid. // And `wake_up_process` is safe to be called for any valid task, even if the task is // running. - unsafe { bindings::wake_up_process(self.0.get()) }; + unsafe { bindings::wake_up_process(self.as_raw()) }; + } +} + +impl Kuid { + /// Get the current euid. + pub fn current_euid() -> Kuid { + // SAFETY: Just an FFI call. + Self::from_raw(unsafe { bindings::current_euid() }) + } + + /// Create a `Kuid` given the raw C type. + pub fn from_raw(kuid: bindings::kuid_t) -> Self { + Self { kuid } + } + + /// Turn this kuid into the raw C type. + pub fn into_raw(self) -> bindings::kuid_t { + self.kuid + } + + /// Converts this kernel UID into a userspace UID. + /// + /// Uses the namespace of the current task. + pub fn into_uid_in_current_ns(self) -> bindings::uid_t { + // SAFETY: Just an FFI call. + unsafe { bindings::from_kuid(bindings::current_user_ns(), self.kuid) } + } +} + +impl PartialEq for Kuid { + fn eq(&self, other: &Kuid) -> bool { + // SAFETY: Just an FFI call. + unsafe { bindings::uid_eq(self.kuid, other.kuid) } + } +} + +impl Kuid { + /// Get the current euid. + pub fn current_euid() -> Kuid { + // SAFETY: Just an FFI call. + Self::from_raw(unsafe { bindings::current_euid() }) + } + + /// Create a `Kuid` given the raw C type. + pub fn from_raw(kuid: bindings::kuid_t) -> Self { + Self { kuid } + } + + /// Turn this kuid into the raw C type. + pub fn into_raw(self) -> bindings::kuid_t { + self.kuid + } + + /// Converts this kernel UID into a userspace UID. + /// + /// Uses the namespace of the current task. + pub fn into_uid_in_current_ns(self) -> bindings::uid_t { + // SAFETY: Just an FFI call. + unsafe { bindings::from_kuid(bindings::current_user_ns(), self.kuid) } + } +} + +impl PartialEq for Kuid { + fn eq(&self, other: &Kuid) -> bool { + // SAFETY: Just an FFI call. + unsafe { bindings::uid_eq(self.kuid, other.kuid) } } } +impl Eq for Kuid {} + // SAFETY: The type invariants guarantee that `Task` is always ref-counted. unsafe impl crate::types::AlwaysRefCounted for Task { fn inc_ref(&self) { diff --git a/rust/kernel/time.rs b/rust/kernel/time.rs new file mode 100644 index 00000000000000..25a896eed4689f --- /dev/null +++ b/rust/kernel/time.rs @@ -0,0 +1,20 @@ +// SPDX-License-Identifier: GPL-2.0 + +//! Time related primitives. +//! +//! This module contains the kernel APIs related to time and timers that +//! have been ported or wrapped for usage by Rust code in the kernel. + +/// The time unit of Linux kernel. One jiffy equals (1/HZ) second. +pub type Jiffies = core::ffi::c_ulong; + +/// The millisecond time unit. +pub type Msecs = core::ffi::c_uint; + +/// Converts milliseconds to jiffies. +#[inline] +pub fn msecs_to_jiffies(msecs: Msecs) -> Jiffies { + // SAFETY: The `__msecs_to_jiffies` function is always safe to call no + // matter what the argument is. + unsafe { bindings::__msecs_to_jiffies(msecs) } +} diff --git a/rust/kernel/types.rs b/rust/kernel/types.rs index fdb778e65d79d3..c0940af1239b1b 100644 --- a/rust/kernel/types.rs +++ b/rust/kernel/types.rs @@ -20,66 +20,111 @@ use core::{ /// This trait is meant to be used in cases when Rust objects are stored in C objects and /// eventually "freed" back to Rust. pub trait ForeignOwnable: Sized { - /// Type of values borrowed between calls to [`ForeignOwnable::into_foreign`] and - /// [`ForeignOwnable::from_foreign`]. + /// Type used to immutably borrow a value that is currently foreign-owned. type Borrowed<'a>; + /// Type used to mutably borrow a value that is currently foreign-owned. + type BorrowedMut<'a>; + /// Converts a Rust-owned object to a foreign-owned one. /// /// The foreign representation is a pointer to void. fn into_foreign(self) -> *const core::ffi::c_void; - /// Borrows a foreign-owned object. + /// Converts a foreign-owned object back to a Rust-owned one. + /// + /// # Safety + /// + /// The provided pointer must have been returned by a previous call to [`into_foreign`], and it + /// must not be passed to `from_foreign` more than once. + /// + /// [`into_foreign`]: Self::into_foreign + unsafe fn from_foreign(ptr: *const core::ffi::c_void) -> Self; + + /// Borrows a foreign-owned object immutably. + /// + /// This method provides a way to access a foreign-owned value from Rust immutably. It provides + /// you with exactly the same abilities as an `&Self` when the value is Rust-owned. /// /// # Safety /// - /// `ptr` must have been returned by a previous call to [`ForeignOwnable::into_foreign`] for - /// which a previous matching [`ForeignOwnable::from_foreign`] hasn't been called yet. + /// The provided pointer must have been returned by a previous call to [`into_foreign`], and if + /// the pointer is ever passed to [`from_foreign`], then that call must happen after the end of + /// the lifetime 'a. + /// + /// [`into_foreign`]: Self::into_foreign + /// [`from_foreign`]: Self::from_foreign unsafe fn borrow<'a>(ptr: *const core::ffi::c_void) -> Self::Borrowed<'a>; - /// Converts a foreign-owned object back to a Rust-owned one. + /// Borrows a foreign-owned object mutably. + /// + /// This method provides a way to access a foreign-owned value from Rust mutably. It provides + /// you with exactly the same abilities as an `&mut Self` when the value is Rust-owned, except + /// that this method does not let you swap the foreign-owned object for another. (That is, it + /// does not let you change the address of the void pointer that the foreign code is storing.) + /// + /// Note that for types like [`Arc`], an `&mut Arc` only gives you immutable access to the + /// inner value, so this method also only provides immutable access in that case. + /// + /// In the case of `Box`, this method gives you the ability to modify the inner `T`, but it + /// does not let you change the box itself. That is, you cannot change which allocation the box + /// points at. /// /// # Safety /// - /// `ptr` must have been returned by a previous call to [`ForeignOwnable::into_foreign`] for - /// which a previous matching [`ForeignOwnable::from_foreign`] hasn't been called yet. - /// Additionally, all instances (if any) of values returned by [`ForeignOwnable::borrow`] for - /// this object must have been dropped. - unsafe fn from_foreign(ptr: *const core::ffi::c_void) -> Self; + /// The provided pointer must have been returned by a previous call to [`into_foreign`], and if + /// the pointer is ever passed to [`from_foreign`], then that call must happen after the end of + /// the lifetime 'a. + /// + /// The lifetime 'a must not overlap with the lifetime of any other call to [`borrow`] or + /// `borrow_mut` on the same object. + /// + /// [`into_foreign`]: Self::into_foreign + /// [`from_foreign`]: Self::from_foreign + /// [`borrow`]: Self::borrow + /// [`Arc`]: crate::sync::Arc + unsafe fn borrow_mut<'a>(ptr: *const core::ffi::c_void) -> Self::BorrowedMut<'a>; } impl ForeignOwnable for Box { type Borrowed<'a> = &'a T; + type BorrowedMut<'a> = &'a mut T; fn into_foreign(self) -> *const core::ffi::c_void { Box::into_raw(self) as _ } - unsafe fn borrow<'a>(ptr: *const core::ffi::c_void) -> &'a T { - // SAFETY: The safety requirements for this function ensure that the object is still alive, - // so it is safe to dereference the raw pointer. - // The safety requirements of `from_foreign` also ensure that the object remains alive for - // the lifetime of the returned value. - unsafe { &*ptr.cast() } - } - unsafe fn from_foreign(ptr: *const core::ffi::c_void) -> Self { // SAFETY: The safety requirements of this function ensure that `ptr` comes from a previous // call to `Self::into_foreign`. unsafe { Box::from_raw(ptr as _) } } + + unsafe fn borrow<'a>(ptr: *const core::ffi::c_void) -> &'a T { + // SAFETY: The safety requirements of this method ensure that the object remains alive and + // immutable for the duration of 'a. + unsafe { &*ptr.cast() } + } + + unsafe fn borrow_mut<'a>(ptr: *const core::ffi::c_void) -> &'a mut T { + // SAFETY: The safety requirements of this method ensure that the pointer is valid and that + // nothing else will access the value for the duration of 'a. + unsafe { &mut *ptr.cast_mut().cast() } + } } impl ForeignOwnable for () { type Borrowed<'a> = (); + type BorrowedMut<'a> = (); fn into_foreign(self) -> *const core::ffi::c_void { core::ptr::NonNull::dangling().as_ptr() } - unsafe fn borrow<'a>(_: *const core::ffi::c_void) -> Self::Borrowed<'a> {} - unsafe fn from_foreign(_: *const core::ffi::c_void) -> Self {} + + unsafe fn borrow<'a>(_: *const core::ffi::c_void) -> Self::Borrowed<'a> {} + unsafe fn borrow_mut<'a>(_: *const core::ffi::c_void) -> Self::BorrowedMut<'a> {} } /// Runs a cleanup function/closure when dropped. @@ -387,3 +432,13 @@ pub enum Either { /// Constructs an instance of [`Either`] containing a value of type `R`. Right(R), } + +/// Zero-sized type to mark types not [`Send`]. +/// +/// Add this type as a field to your struct if your type should not be sent to a different task. +/// Since [`Send`] is an auto trait, adding a single field that is `!Send` will ensure that the +/// whole type is `!Send`. +/// +/// If a type is `!Send` it is impossible to give control over an instance of the type to another +/// task. This is useful when a type stores task-local information for example file descriptors. +pub type NotThreadSafe = PhantomData<*mut ()>; diff --git a/rust/kernel/workqueue.rs b/rust/kernel/workqueue.rs index b67fb1ba168ed2..7f394e682f3feb 100644 --- a/rust/kernel/workqueue.rs +++ b/rust/kernel/workqueue.rs @@ -132,7 +132,7 @@ //! } //! ``` //! -//! C header: [`include/linux/workqueue.h`](../../../../include/linux/workqueue.h) +//! C header: [`include/linux/workqueue.h`](srctree/include/linux/workqueue.h) use crate::{bindings, prelude::*, sync::Arc, sync::LockClassKey, types::Opaque}; use alloc::alloc::AllocError; @@ -334,8 +334,10 @@ pub trait WorkItem { /// Wraps the kernel's C `struct work_struct`. /// /// This is a helper type used to associate a `work_struct` with the [`WorkItem`] that uses it. +#[pin_data] #[repr(transparent)] pub struct Work { + #[pin] work: Opaque, _inner: PhantomData, } @@ -357,21 +359,22 @@ impl Work { where T: WorkItem, { - // SAFETY: The `WorkItemPointer` implementation promises that `run` can be used as the work - // item function. - unsafe { - kernel::init::pin_init_from_closure(move |slot| { - let slot = Self::raw_get(slot); - bindings::init_work_with_key( - slot, - Some(T::Pointer::run), - false, - name.as_char_ptr(), - key.as_ptr(), - ); - Ok(()) - }) - } + pin_init!(Self { + work <- Opaque::ffi_init(|slot| { + // SAFETY: The `WorkItemPointer` implementation promises that `run` can be used as + // the work item function. + unsafe { + bindings::init_work_with_key( + slot, + Some(T::Pointer::run), + false, + name.as_char_ptr(), + key.as_ptr(), + ) + } + }), + _inner: PhantomData, + }) } /// Get a pointer to the inner `work_struct`. diff --git a/rust/macros/helpers.rs b/rust/macros/helpers.rs index afb0f2e3a36a9d..3f50a5c847c8cc 100644 --- a/rust/macros/helpers.rs +++ b/rust/macros/helpers.rs @@ -1,6 +1,6 @@ // SPDX-License-Identifier: GPL-2.0 -use proc_macro::{token_stream, Group, Punct, Spacing, TokenStream, TokenTree}; +use proc_macro::{token_stream, Group, TokenStream, TokenTree}; pub(crate) fn try_ident(it: &mut token_stream::IntoIter) -> Option { if let Some(TokenTree::Ident(ident)) = it.next() { @@ -70,8 +70,40 @@ pub(crate) fn expect_end(it: &mut token_stream::IntoIter) { } } +/// Parsed generics. +/// +/// See the field documentation for an explanation what each of the fields represents. +/// +/// # Examples +/// +/// ```rust,ignore +/// # let input = todo!(); +/// let (Generics { decl_generics, impl_generics, ty_generics }, rest) = parse_generics(input); +/// quote! { +/// struct Foo<$($decl_generics)*> { +/// // ... +/// } +/// +/// impl<$impl_generics> Foo<$ty_generics> { +/// fn foo() { +/// // ... +/// } +/// } +/// } +/// ``` pub(crate) struct Generics { + /// The generics with bounds and default values (e.g. `T: Clone, const N: usize = 0`). + /// + /// Use this on type definitions e.g. `struct Foo<$decl_generics> ...` (or `union`/`enum`). + pub(crate) decl_generics: Vec, + /// The generics with bounds (e.g. `T: Clone, const N: usize`). + /// + /// Use this on `impl` blocks e.g. `impl<$impl_generics> Trait for ...`. pub(crate) impl_generics: Vec, + /// The generics without bounds and without default values (e.g. `T, N`). + /// + /// Use this when you use the type that is declared with these generics e.g. + /// `Foo<$ty_generics>`. pub(crate) ty_generics: Vec, } @@ -81,6 +113,8 @@ pub(crate) struct Generics { pub(crate) fn parse_generics(input: TokenStream) -> (Generics, Vec) { // `impl_generics`, the declared generics with their bounds. let mut impl_generics = vec![]; + // The generics with bounds and default values. + let mut decl_generics = vec![]; // Only the names of the generics, without any bounds. let mut ty_generics = vec![]; // Tokens not related to the generics e.g. the `where` token and definition. @@ -90,10 +124,17 @@ pub(crate) fn parse_generics(input: TokenStream) -> (Generics, Vec) { let mut toks = input.into_iter(); // If we are at the beginning of a generic parameter. let mut at_start = true; - for tt in &mut toks { + let mut skip_until_comma = false; + while let Some(tt) = toks.next() { + if nesting == 1 && matches!(&tt, TokenTree::Punct(p) if p.as_char() == '>') { + // Found the end of the generics. + break; + } else if nesting >= 1 { + decl_generics.push(tt.clone()); + } match tt.clone() { TokenTree::Punct(p) if p.as_char() == '<' => { - if nesting >= 1 { + if nesting >= 1 && !skip_until_comma { // This is inside of the generics and part of some bound. impl_generics.push(tt); } @@ -105,49 +146,70 @@ pub(crate) fn parse_generics(input: TokenStream) -> (Generics, Vec) { break; } else { nesting -= 1; - if nesting >= 1 { + if nesting >= 1 && !skip_until_comma { // We are still inside of the generics and part of some bound. impl_generics.push(tt); } - if nesting == 0 { - break; - } } } - tt => { + TokenTree::Punct(p) if skip_until_comma && p.as_char() == ',' => { if nesting == 1 { - // Here depending on the token, it might be a generic variable name. - match &tt { - // Ignore const. - TokenTree::Ident(i) if i.to_string() == "const" => {} - TokenTree::Ident(_) if at_start => { - ty_generics.push(tt.clone()); - // We also already push the `,` token, this makes it easier to append - // generics. - ty_generics.push(TokenTree::Punct(Punct::new(',', Spacing::Alone))); - at_start = false; - } - TokenTree::Punct(p) if p.as_char() == ',' => at_start = true, - // Lifetimes begin with `'`. - TokenTree::Punct(p) if p.as_char() == '\'' && at_start => { - ty_generics.push(tt.clone()); - } - _ => {} - } + impl_generics.push(TokenTree::Punct(p.clone())); + ty_generics.push(TokenTree::Punct(p)); + skip_until_comma = false; } - if nesting >= 1 { - impl_generics.push(tt); - } else if nesting == 0 { + } + tt if !skip_until_comma => { + match nesting { // If we haven't entered the generics yet, we still want to keep these tokens. - rest.push(tt); + 0 => rest.push(tt), + 1 => { + // Here depending on the token, it might be a generic variable name. + match tt { + TokenTree::Ident(i) if at_start && i.to_string() == "const" => { + let Some(name) = toks.next() else { + // Parsing error. + break; + }; + impl_generics.push(TokenTree::Ident(i)); + impl_generics.push(name.clone()); + ty_generics.push(name.clone()); + decl_generics.push(name); + at_start = false; + } + tt @ TokenTree::Ident(_) if at_start => { + impl_generics.push(tt.clone()); + ty_generics.push(tt); + at_start = false; + } + TokenTree::Punct(p) if p.as_char() == ',' => { + impl_generics.push(TokenTree::Punct(p.clone())); + ty_generics.push(TokenTree::Punct(p)); + at_start = true; + } + // Lifetimes begin with `'`. + TokenTree::Punct(p) if p.as_char() == '\'' && at_start => { + ty_generics.push(TokenTree::Punct(p.clone())); + impl_generics.push(TokenTree::Punct(p)); + } + // Generics can have default values, we skip these. + TokenTree::Punct(p) if p.as_char() == '=' => { + skip_until_comma = true; + } + tt => impl_generics.push(tt), + } + } + _ => impl_generics.push(tt), } } + _ => {} } } rest.extend(toks); ( Generics { impl_generics, + decl_generics, ty_generics, }, rest, diff --git a/rust/macros/lib.rs b/rust/macros/lib.rs index c42105c2ff9635..f489f315738323 100644 --- a/rust/macros/lib.rs +++ b/rust/macros/lib.rs @@ -20,7 +20,7 @@ use proc_macro::TokenStream; /// The `type` argument should be a type which implements the [`Module`] /// trait. Also accepts various forms of kernel metadata. /// -/// C header: [`include/linux/moduleparam.h`](../../../include/linux/moduleparam.h) +/// C header: [`include/linux/moduleparam.h`](srctree/include/linux/moduleparam.h) /// /// [`Module`]: ../kernel/trait.Module.html /// @@ -87,27 +87,49 @@ pub fn module(ts: TokenStream) -> TokenStream { /// implementation could just return `Error::EINVAL`); Linux typically use C /// `NULL` pointers to represent these functions. /// -/// This attribute is intended to close the gap. Traits can be declared and -/// implemented with the `#[vtable]` attribute, and a `HAS_*` associated constant -/// will be generated for each method in the trait, indicating if the implementor -/// has overridden a method. +/// This attribute closes that gap. A trait can be annotated with the +/// `#[vtable]` attribute. Implementers of the trait will then also have to +/// annotate the trait with `#[vtable]`. This attribute generates a `HAS_*` +/// associated constant bool for each method in the trait that is set to true if +/// the implementer has overridden the associated method. +/// +/// For a trait method to be optional, it must have a default implementation. +/// This is also the case for traits annotated with `#[vtable]`, but in this +/// case the default implementation will never be executed. The reason for this +/// is that the functions will be called through function pointers installed in +/// C side vtables. When an optional method is not implemented on a `#[vtable]` +/// trait, a NULL entry is installed in the vtable. Thus the default +/// implementation is never called. Since these traits are not designed to be +/// used on the Rust side, it should not be possible to call the default +/// implementation. This is done to ensure that we call the vtable methods +/// through the C vtable, and not through the Rust vtable. Therefore, the +/// default implementation should call `kernel::build_error`, which prevents +/// calls to this function at compile time: +/// +/// ```compile_fail +/// # use kernel::error::VTABLE_DEFAULT_ERROR; +/// kernel::build_error(VTABLE_DEFAULT_ERROR) +/// ``` +/// +/// Note that you might need to import [`kernel::error::VTABLE_DEFAULT_ERROR`]. /// -/// This attribute is not needed if all methods are required. +/// This macro should not be used when all functions are required. /// /// # Examples /// /// ```ignore +/// use kernel::error::VTABLE_DEFAULT_ERROR; /// use kernel::prelude::*; /// /// // Declares a `#[vtable]` trait /// #[vtable] /// pub trait Operations: Send + Sync + Sized { /// fn foo(&self) -> Result<()> { -/// Err(EINVAL) +/// kernel::build_error(VTABLE_DEFAULT_ERROR) /// } /// /// fn bar(&self) -> Result<()> { -/// Err(EINVAL) +/// kernel::build_error(VTABLE_DEFAULT_ERROR) /// } /// } /// @@ -125,6 +147,8 @@ pub fn module(ts: TokenStream) -> TokenStream { /// assert_eq!(::HAS_FOO, true); /// assert_eq!(::HAS_BAR, false); /// ``` +/// +/// [`kernel::error::VTABLE_DEFAULT_ERROR`]: ../kernel/error/constant.VTABLE_DEFAULT_ERROR.html #[proc_macro_attribute] pub fn vtable(attr: TokenStream, ts: TokenStream) -> TokenStream { vtable::vtable(attr, ts) @@ -254,8 +278,8 @@ pub fn pinned_drop(args: TokenStream, input: TokenStream) -> TokenStream { /// Within the `paste!` macro, identifiers inside `[<` and `>]` are concatenated together to form a /// single identifier. /// -/// This is similar to the [`paste`] crate, but with pasting feature limited to identifiers -/// (literals, lifetimes and documentation strings are not supported). There is a difference in +/// This is similar to the [`paste`] crate, but with pasting feature limited to identifiers and +/// literals (lifetimes and documentation strings are not supported). There is a difference in /// supported modifiers as well. /// /// # Example @@ -337,6 +361,24 @@ pub fn pinned_drop(args: TokenStream, input: TokenStream) -> TokenStream { /// assert_eq!(br_ok(), binder_driver_return_protocol_BR_OK); /// ``` /// +/// # Literals +/// +/// Literals can also be concatenated with other identifiers: +/// +/// ```ignore +/// macro_rules! create_numbered_fn { +/// ($name:literal, $val:literal) => { +/// kernel::macros::paste! { +/// fn []() -> u32 { $val } +/// } +/// }; +/// } +/// +/// create_numbered_fn!("foo", 100); +/// +/// assert_eq!(some_foo_fn100(), 100) +/// ``` +/// /// [`paste`]: https://docs.rs/paste/ #[proc_macro] pub fn paste(input: TokenStream) -> TokenStream { diff --git a/rust/macros/paste.rs b/rust/macros/paste.rs index 385a78434224f6..f40d42b35b5869 100644 --- a/rust/macros/paste.rs +++ b/rust/macros/paste.rs @@ -9,7 +9,15 @@ fn concat(tokens: &[TokenTree], group_span: Span) -> TokenTree { loop { match tokens.next() { None => break, - Some(TokenTree::Literal(lit)) => segments.push((lit.to_string(), lit.span())), + Some(TokenTree::Literal(lit)) => { + // Allow us to concat string literals by stripping quotes + let mut value = lit.to_string(); + if value.starts_with('"') && value.ends_with('"') { + value.remove(0); + value.pop(); + } + segments.push((value, lit.span())); + } Some(TokenTree::Ident(ident)) => { let mut value = ident.to_string(); if value.starts_with("r#") { diff --git a/rust/macros/pin_data.rs b/rust/macros/pin_data.rs index 6d58cfda9872e6..1d4a3547c684b3 100644 --- a/rust/macros/pin_data.rs +++ b/rust/macros/pin_data.rs @@ -10,6 +10,7 @@ pub(crate) fn pin_data(args: TokenStream, input: TokenStream) -> TokenStream { let ( Generics { impl_generics, + decl_generics, ty_generics, }, rest, @@ -76,6 +77,7 @@ pub(crate) fn pin_data(args: TokenStream, input: TokenStream) -> TokenStream { @sig(#(#rest)*), @impl_generics(#(#impl_generics)*), @ty_generics(#(#ty_generics)*), + @decl_generics(#(#decl_generics)*), @body(#last), }); quoted.extend(errs); diff --git a/rust/macros/zeroable.rs b/rust/macros/zeroable.rs index 0d605c46ab3b40..cfee2cec18d5f5 100644 --- a/rust/macros/zeroable.rs +++ b/rust/macros/zeroable.rs @@ -7,6 +7,7 @@ pub(crate) fn derive(input: TokenStream) -> TokenStream { let ( Generics { impl_generics, + decl_generics: _, ty_generics, }, mut rest, diff --git a/scripts/Makefile b/scripts/Makefile index 576cf64be6677c..c85d130a4125bd 100644 --- a/scripts/Makefile +++ b/scripts/Makefile @@ -11,12 +11,14 @@ hostprogs-always-$(CONFIG_MODULE_SIG_FORMAT) += sign-file hostprogs-always-$(CONFIG_SYSTEM_EXTRA_CERTIFICATE) += insert-sys-cert hostprogs-always-$(CONFIG_RUST_KERNEL_DOCTESTS) += rustdoc_test_builder hostprogs-always-$(CONFIG_RUST_KERNEL_DOCTESTS) += rustdoc_test_gen -always-$(CONFIG_RUST) += target.json +ifdef CONFIG_X86_64 +always-$(CONFIG_RUST) += target.json filechk_rust_target = $< < include/config/auto.conf $(obj)/target.json: scripts/generate_rust_target include/config/auto.conf FORCE $(call filechk,rust_target) +endif hostprogs += generate_rust_target generate_rust_target-rust := y diff --git a/scripts/Makefile.build b/scripts/Makefile.build index 9a3063735e711b..dae447a1ad30c4 100644 --- a/scripts/Makefile.build +++ b/scripts/Makefile.build @@ -275,6 +275,7 @@ rust_common_cmd = \ --extern alloc --extern kernel \ --crate-type rlib -L $(objtree)/rust/ \ --crate-name $(basename $(notdir $@)) \ + --sysroot=/dev/null \ --out-dir $(dir $@) --emit=dep-info=$(depfile) # `--emit=obj`, `--emit=asm` and `--emit=llvm-ir` imply a single codegen unit diff --git a/scripts/generate_rust_target.rs b/scripts/generate_rust_target.rs index 3c6cbe2b278d30..ec5ef35dbe52db 100644 --- a/scripts/generate_rust_target.rs +++ b/scripts/generate_rust_target.rs @@ -148,7 +148,9 @@ fn main() { let mut ts = TargetSpec::new(); // `llvm-target`s are taken from `scripts/Makefile.clang`. - if cfg.has("X86_64") { + if cfg.has("ARM64") { + panic!("arm64 uses the builtin rustc aarch64-unknown-none target"); + } else if cfg.has("X86_64") { ts.push("arch", "x86_64"); ts.push( "data-layout", diff --git a/scripts/min-tool-version.sh b/scripts/min-tool-version.sh index fd5ffdb81bab70..bcc7d4247290d4 100755 --- a/scripts/min-tool-version.sh +++ b/scripts/min-tool-version.sh @@ -31,7 +31,7 @@ llvm) fi ;; rustc) - echo 1.73.0 + echo 1.75.0 ;; bindgen) echo 0.65.1