Skip to content

Generic transmutability #1359

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
joshlf opened this issue May 30, 2024 · 0 comments · Fixed by #2408
Open

Generic transmutability #1359

joshlf opened this issue May 30, 2024 · 0 comments · Fixed by #2408

Comments

@joshlf
Copy link
Member

joshlf commented May 30, 2024

See also: #1122

Currently, we have a number of traits that express some aspect of transmutability:

These, in turn, are used to bound certain Ptr transmutation methods, e.g.:

Both transparent_wrapper_into_inner and as_bytes implement special cases of a generic Ptr<T> to Ptr<U> transmutation. This suggests that we could in theory support a more generic Ptr::transmute method, supported by a new TransmutableFrom trait. This trait would share some similarities with the built-in transmutability trait. However, as TransparentWrapper's support for invariant variance shows, it might be more powerful than that trait in certain ways.

In its most basic form, this API would look something like:

unsafe trait TransmutableFrom<T> {}

impl<'a, T, A: Aliasing> Ptr<'a, T, (A, Aligned, Valid)> {
    fn transmute<U: TransmutableFrom<T>>(self) -> Ptr<'a, U, (A, Aligned, Valid)>
}

However, we will likely want to relax the Aligned and Valid requirements and post-conditions, and support a generic framework for mapping source invariants (on T) to destination invariants (on U) - essentially a generalization of what's currently supported with TransparentWrapper's invariant variance concept.

There may be multiple overlapping reasons to support transmuting a particular pair of types. For example, we could imagine the following impls:

unsafe impl<T: IntoBytes, U: FromBytes> TransmutableFrom<T> for U {}
unsafe impl<T> TransmutableFrom<T> for MaybeUninit<T> {}

Currently, these would result in an impl conflict. If #[marker] traits are stabilized, this won't be an issue - we can just mark TransmutableFrom as a #[marker] trait. Alternatively, we could use a dummy "reason" parameter to disambiguate, as we do with AliasingSafe today:

unsafe trait TransmutableFrom<T, R> {}

enum BecauseIntoBytesFromBytes {}
enum BecauseMaybeUninit {}

unsafe impl<T: IntoBytes, U: FromBytes> TransmutableFrom<T, BecauseIntoBytesFromBytes> for U {}
unsafe impl<T> TransmutableFrom<T> for MaybeUninit<T, BecauseMaybeUninit> {}
@joshlf joshlf mentioned this issue Oct 18, 2024
9 tasks
github-merge-queue bot pushed a commit that referenced this issue Feb 17, 2025
Previously, validity was stored as part of the `I: Invariants` type
parameter on a `Ptr<T, I>`. In this commit, we migrate the validity to
being stored as the `V` in `Ptr<V, I>`; `I` now only stores aliasing and
alignment.

Bit validity is subtle, in that the pair of referent type and validity
invariant define a set of bit patterns which may appear in the `Ptr`'s
referent. By encoding the validity in the referent type instead of in
the `I` parameter, we ensure that the validity of the referent is
captured entirely by a single type parameter rather than by a pair of
two separate type parameters. This makes future changes (e.g. #1359)
easier to model.

This idea was originally proposed in #1866 (comment)

Makes progress on #1866

gherrit-pr-id: Ia73ca4e5dfb3b20f71ed72ffda24dd7450c427ba
joshlf added a commit that referenced this issue Mar 6, 2025
TODO

This commit also takes a different approach from the one originally
envisioned in #1945. In particular, it turns out that we don't need a
full type-level mapping concept. Instead, we need a *predicate* over
transitions to determine which ones are valid (e.g., it is valid to go
from a `Valid` `MaybeUninit<T>` to an `Uninit` `MaybeUninit<T>`). By
contrast, the invariant mapping concept suggests that each source
validity has *exactly one* destination validity.

This commit makes progress on #1940 by supporting unsized
transmutations, but we don't yet support size shrinking or expanding
transmutations.

This commit obsoletes #1359, as that issue was predicated upon the
existence of `TransparentWrapper`, which this commit removes.

This commit closes #1226, which suggests supporting `UnsafeCell`
agreement.

Closes #1945
Closes #1359
Closes #2226
Closes #1226
Makes progress on #1359

Co-authored-by: Jack Wrenn <[email protected]>
gherrit-pr-id: Iad14813bc6d933312bc8d7a1ddcf1aafc7126938
joshlf added a commit that referenced this issue Mar 6, 2025
This commit removes the `TransparentWrapper` trait and the
`Ptr::transparent_wrapper_into_inner` method. It replaces them with a
new family of transmutation traits which encode more generic
transmutation (from any `T` to any `U`) and a set of `Ptr` methods which
use those traits to bound transmutation operations.

In particular:
- `Dst: TransmuteFrom<Src>` denotes that a by-value transmutation is
  sound
- `Dst: TryTransmuteFromPtr<Src>` denotes that a transmutation is sound
  so long as it can be guaranteed that the source is bit-valid for the
  destination; this is used by e.g. `Ptr::try_into_valid`, which
  performs runtime validation of bit validity
- `Dst: TransmuteFromPtr<Src>` is equivalent to `TransmuteFrom<Src> +
  TryTransmuteFromPtr<Src>`

Some type arguments are omitted in this summary. In particular, all
three traits also take validity invariant parameters for both the source
and destination types. Also, the `[Try]TransmuteFromPtr` traits take an
aliasing parameter.

In order to support these traits, we introduce a generalization of
`Read` known as `MutationCompatible`. `T: MutationCompatible<U, A>`
denotes that *either* `T: Read<A>` and `U: Read<A>` *or* `T` and `U`
have the same interior mutation semantics (formally, it is sound for
`&T` and `&U` to reference the same referent - safe code operating on
these references cannot cause undefined behavior). This is a refinement
of the "`UnsafeCell` agreement" concept that we have used before, but it
supports types which store but don't actually use `UnsafeCell`s. For
example, given a hypothetical `ReadOnly<T>`, the following bound holds:

  usize: MutationCompatible<ReadOnly<AtomicUsize>, Exclusive>

This commit also takes a different approach from the one originally
envisioned in #1945. In particular, it turns out that we don't need a
full type-level mapping concept. Instead, we need a *predicate* over
transitions to determine which ones are valid (e.g., it is valid to go
from a `Valid` `MaybeUninit<T>` to an `Uninit` `MaybeUninit<T>`). By
contrast, the invariant mapping concept suggests that each source
validity has *exactly one* destination validity.

This commit makes progress on #1940 by supporting unsized
transmutations, but we don't yet support size shrinking or expanding
transmutations.

This commit obsoletes #1359, as that issue was predicated upon the
existence of `TransparentWrapper`, which this commit removes.

This commit closes #1226, which suggests supporting `UnsafeCell`
agreement.

Closes #1945
Closes #1359
Closes #2226
Closes #1226
Closes #1866
Makes progress on #1359

Co-authored-by: Jack Wrenn <[email protected]>
gherrit-pr-id: Iad14813bc6d933312bc8d7a1ddcf1aafc7126938
joshlf added a commit that referenced this issue Mar 6, 2025
This commit removes the `TransparentWrapper` trait and the
`Ptr::transparent_wrapper_into_inner` method. It replaces them with a
new family of transmutation traits which encode more generic
transmutation (from any `T` to any `U`) and a set of `Ptr` methods which
use those traits to bound transmutation operations.

In particular:
- `Dst: TransmuteFrom<Src>` denotes that a by-value transmutation is
  sound
- `Dst: TryTransmuteFromPtr<Src>` denotes that a transmutation is sound
  so long as it can be guaranteed that the source is bit-valid for the
  destination; this is used by e.g. `Ptr::try_into_valid`, which
  performs runtime validation of bit validity
- `Dst: TransmuteFromPtr<Src>` is equivalent to `TransmuteFrom<Src> +
  TryTransmuteFromPtr<Src>`

Some type arguments are omitted in this summary. In particular, all
three traits also take validity invariant parameters for both the source
and destination types. Also, the `[Try]TransmuteFromPtr` traits take an
aliasing parameter.

In order to support these traits, we introduce a generalization of
`Read` known as `MutationCompatible`. `T: MutationCompatible<U, A>`
denotes that *either* `T: Read<A>` and `U: Read<A>` *or* `T` and `U`
have the same interior mutation semantics (formally, it is sound for
`&T` and `&U` to reference the same referent - safe code operating on
these references cannot cause undefined behavior). This is a refinement
of the "`UnsafeCell` agreement" concept that we have used before, but it
supports types which store but don't actually use `UnsafeCell`s. For
example, given a hypothetical `ReadOnly<T>`, the following bound holds:

  usize: MutationCompatible<ReadOnly<AtomicUsize>, Exclusive>

This commit also takes a different approach from the one originally
envisioned in #1945. In particular, it turns out that we don't need a
full type-level mapping concept. Instead, we need a *predicate* over
transitions to determine which ones are valid (e.g., it is valid to go
from a `Valid` `MaybeUninit<T>` to an `Uninit` `MaybeUninit<T>`). By
contrast, the invariant mapping concept suggests that each source
validity has *exactly one* destination validity.

This commit makes progress on #1940 by supporting unsized
transmutations, but we don't yet support size shrinking or expanding
transmutations.

This commit obsoletes #1359, as that issue was predicated upon the
existence of `TransparentWrapper`, which this commit removes.

This commit closes #1226, which suggests supporting `UnsafeCell`
agreement.

Closes #1945
Closes #1359
Closes #2226
Closes #1226
Closes #1866
Makes progress on #1359

Co-authored-by: Jack Wrenn <[email protected]>
gherrit-pr-id: Iad14813bc6d933312bc8d7a1ddcf1aafc7126938
github-merge-queue bot pushed a commit that referenced this issue Mar 6, 2025
This commit removes the `TransparentWrapper` trait and the
`Ptr::transparent_wrapper_into_inner` method. It replaces them with a
new family of transmutation traits which encode more generic
transmutation (from any `T` to any `U`) and a set of `Ptr` methods which
use those traits to bound transmutation operations.

In particular:
- `Dst: TransmuteFrom<Src>` denotes that a by-value transmutation is
  sound
- `Dst: TryTransmuteFromPtr<Src>` denotes that a transmutation is sound
  so long as it can be guaranteed that the source is bit-valid for the
  destination; this is used by e.g. `Ptr::try_into_valid`, which
  performs runtime validation of bit validity
- `Dst: TransmuteFromPtr<Src>` is equivalent to `TransmuteFrom<Src> +
  TryTransmuteFromPtr<Src>`

Some type arguments are omitted in this summary. In particular, all
three traits also take validity invariant parameters for both the source
and destination types. Also, the `[Try]TransmuteFromPtr` traits take an
aliasing parameter.

In order to support these traits, we introduce a generalization of
`Read` known as `MutationCompatible`. `T: MutationCompatible<U, A>`
denotes that *either* `T: Read<A>` and `U: Read<A>` *or* `T` and `U`
have the same interior mutation semantics (formally, it is sound for
`&T` and `&U` to reference the same referent - safe code operating on
these references cannot cause undefined behavior). This is a refinement
of the "`UnsafeCell` agreement" concept that we have used before, but it
supports types which store but don't actually use `UnsafeCell`s. For
example, given a hypothetical `ReadOnly<T>`, the following bound holds:

  usize: MutationCompatible<ReadOnly<AtomicUsize>, Exclusive>

This commit also takes a different approach from the one originally
envisioned in #1945. In particular, it turns out that we don't need a
full type-level mapping concept. Instead, we need a *predicate* over
transitions to determine which ones are valid (e.g., it is valid to go
from a `Valid` `MaybeUninit<T>` to an `Uninit` `MaybeUninit<T>`). By
contrast, the invariant mapping concept suggests that each source
validity has *exactly one* destination validity.

This commit makes progress on #1940 by supporting unsized
transmutations, but we don't yet support size shrinking or expanding
transmutations.

This commit obsoletes #1359, as that issue was predicated upon the
existence of `TransparentWrapper`, which this commit removes.

This commit closes #1226, which suggests supporting `UnsafeCell`
agreement.

Closes #1945
Closes #1359
Closes #2226
Closes #1226
Closes #1866
Makes progress on #1359


gherrit-pr-id: Iad14813bc6d933312bc8d7a1ddcf1aafc7126938

Co-authored-by: Jack Wrenn <[email protected]>
@joshlf joshlf reopened this Mar 6, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant