Description
Proposal
Problem statement
feature(ptr_metadata) is in feature limbo. Being able to split pointers into raw pointer and metadata unlocks a lot of cool abilities in library code, but as currently architected the metadata APIs expose more than is really comfortable to stabilize in the short term. Notably, Pointee is a very special lang-item, being automatically implemented for every type. Plus, it's unfortunately tied in identity to both "DynSized", extern type, and custom pointee metadata.
This API offers a way to expose a minimal feature(min_ptr_metadata) subset of feature(ptr_metadata) that unblocks the simple use cases of separating address from metadata, while remaining forward-compatible with any1 design for exposing more knobs later on.
In short: this splits feature(ptr_metadata) into granular subfeatures such that min_ptr_metadata might be stabilizable this year.
Motivation, use-cases
My main motivating use case is custom reference-like types.
Consider something like a generational arena. When using an arena, instead of using something like rc::Weak<Object>, you use Handle<Object>, where Handle is some index; perhaps { ix: u32, salt: u32 }. This works well when your arena stores a single sized type; you Arena::<Object>::resolve(&self, Handle<Object>) -> &'_ Object and all is good.
This is simple enough to extend to storing arbitrary sized objects in the arena as well, so long as you use some kind of allocation strategy which can handle the mixed-layout allocations rather than a simple array storage. This also means that your index changes from an element index to a byte index, and you need some way to ensure that a handle is only used for the arena that it came from.
But what about unsized types? For example, in an actor-based game engine (as opposed to an ECS-based solution), you might have Handle<PlayerPawn> and Handle<EnemyPawn>, and want to be able to store a handle to either one as Handle<dyn Pawn>. Without feature(ptr_metadata), such a handle is forced to store some ptr::NonNull<dyn Pawn>, where what actually wants to be stored is { ix: u32, salt: u32, meta: ptr::Metadata<dyn Pawn> }.
Being able to separate pointer metadata from the actual pointer is a space-saving mechanism for such designs in cases where the storage is known to be pinned, as resolving a handle could be implemented to validate the index, salt, address, etc. match and then dereference the stored pointer with its metadata. When the storage is not pinned, however, allowing dyn-erased handles is just not possible, as there is no (stable) way (yet) to graft one pointer's metadata onto another's.
Solution sketches
Due to the specifics of this proposal, the proposed API is a slight evolution of the existing API, and heavily split into multiple features. For continuity, all of these smaller features should be considered implied by the ptr_metadata feature.
// mod core::ptr
#[unstable(feature = "ptr_metadata_trait")]
pub trait Pointee {
type Metadata: MetadataOf<Self>;
}
// compiler-generated
impl<T: Sized> Pointee for T {
type Metadata = ();
}
// compiler-generated
impl<T: Sized> Pointee for [T] {
type Metadata = usize;
}
// compiler-generated
impl<trait Trait> Pointee for dyn Trait {
type Metadata = DynMetadata<dyn Trait>;
}
#[unstable(feature = "thin_ptr_metadata")]
pub trait Thin = Pointee<Metadata = ()>;
#[unstable(feature = "min_ptr_metadata")]
struct Metadata<T: ?Sized> {
/*priv*/ raw: <T as Pointee>::Metadata,
}
impl<T: ?Sized>
Copy, Clone, Send, Sync, Debug, Ord, PartialOrd, Eq, PartialEq, Hash, Unpin
for Metadata<T>;
// but not StructuralEq!
#[unstable(feature = "ptr_metadata_access")]
impl Metadata<T: ?Sized> {
pub const fn into_raw(self) -> <T as Pointee>::Metadata;
pub const fn from_raw(<T as Pointee>::Metadata) -> Self;
}
#[unstable(feature = "min_ptr_metadata")]
pub const fn metadata<T: ?Sized>(*const T) -> Metadata<T>;
#[unstable(feature = "from_ptr_metadata")]
pub const fn from_raw_parts<T: ?Sized>(*const (), Metadata<T>) -> *const T;
#[unstable(feature = "from_ptr_metadata")]
pub const fn from_raw_parts_mut<T: ?Sized>(*mut (), Metadata<T>) -> *mut T;
impl<T: ?Sized> NonNull<T> {
#[unstable(feature = "min_ptr_metadata")]
pub const fn metadata(self) -> Metadata<T>;
#[unstable(feature = "into_ptr_metadata")]
pub const fn to_raw_parts(self) -> (NonNull<()>, Metadata<T>);
#[unstable(feature = "from_ptr_metadata")]
pub const fn from_raw_parts(NonNull<()>, Metadata<T>) -> Self;
#[unstable(feature = "with_ptr_metadata")]
pub const fn with_metadata<U: ?Sized>(self, Metadata<U>) -> NonNull<U>;
#[unstable(feature = "with_ptr_metadata")] // currently feature(set_ptr_value)
pub const fn with_metadata_of<U: ?Sized>(self, *const U) -> NonNull<U>;
}
impl<T: ?Sized> *const T {
#[unstable(feature = "min_ptr_metadata")]
pub const fn metadata(self) -> Metadata<T>;
#[unstable(feature = "into_ptr_metadata")]
pub const fn to_raw_parts(self) -> (*const (), Metadata<T>);
#[unstable(feature = "from_ptr_metadata")]
pub const fn from_raw_parts(*const (), Metadata<T>) -> Self;
#[unstable(feature = "with_ptr_metadata")]
pub const fn with_metadata<U: ?Sized>(self, Metadata<U>) -> *const U;
#[unstable(feature = "with_ptr_metadata")] // currently feature(set_ptr_value)
pub const fn with_metadata_of<U: ?Sized>(self, *const U) -> *const U;
}
impl<T: ?Sized> *mut T {
#[unstable(feature = "min_ptr_metadata")]
pub const fn metadata(self) -> Metadata<T>;
#[unstable(feature = "into_ptr_metadata")]
pub const fn to_raw_parts(self) -> (*mut (), Metadata<T>);
#[unstable(feature = "from_ptr_metadata")]
pub const fn from_raw_parts(*mut (), Metadata<T>) -> Self;
#[unstable(feature = "with_ptr_metadata")]
pub const fn with_metadata<U: ?Sized>(self, Metadata<U>) -> *mut U;
#[unstable(feature = "with_ptr_metadata")] // currently feature(set_ptr_value)
pub const fn with_metadata_of<U: ?Sized>(self, *const U) -> *mut U;
}
#[unstable(/* indeterminate */)]
unsafe trait MetadataOf<T: ?Sized>: Send + Sync + Copy + Debug + Ord + Hash + Unpin {
// indeterminate
}
unsafe impl<T: Sized> MetadataOf<T> for () {}
unsafe impl<T: Sized> MetadataOf<[T]> for usize {}
#[unstable(feature = "dyn_ptr_metadata")]
pub struct DynMetadata<T: ?Sized> {
/*priv*/ raw: WellFormed<__CompilerGeneratedVtable<T>>,
}
impl<T>
Copy, Clone, Send, Sync, Debug, Ord, PartialOrd, Eq, PartialEq, Hash, Unpin
for DynMetadata<T>;
// but not StructuralEq!
// compiler-generated, if statically restricting to just dyn Trait objects
unsafe impl<trait Trait> MetadataOf<dyn Trait> for DynMetadata<T> {}
#[unstable(feature = "ptr_metadata_layout")]
impl<T: ?Sized> DynMetadata<T> {
pub fn size(self) -> usize;
pub fn align(self) -> usize;
pub fn layout(self) -> core::alloc::Layout;
}The key change is that rather than expose concrete metadata types, we only expose a single uniform opaque ptr::Metadata<T> type. This can have additional other benefit -- one key one I'm interested in is that ptr::Metadata<T> can CoerceUnsized, allowing pointer-like types wrapping ptr::Metadata to get unsizing coercions as if they were holding an actual pointer.
The other key removal of surface area of ptr_metadata in min_ptr_metadata is the moving of from_ptr_metadata into a separate feature. This means that the choice of using *mut () or *mut u8 for the untyped pointer is deferred to a separate feature. The alternative is with_metadata[_of].
Other notes on API design choices
- It is valid to (
as).cast()between pointers which have the same<T as Pointee>::Metadatatype.- This, along with the existing
(*const T, usize) -> *const [T]functions, is why<[T] as Pointee>::Metadataisusizerather than aSliceMetadata<T>type.
- This, along with the existing
core::ptr::WellFormedis a raw pointer that is guaranteed non-null and aligned, per Commit to safety rules for dyn trait upcasting rust#101336- The metadata
sizefunction probably wants to wait for the public exposing of aValidSizetype which is bound to0..=isize::MAX. - The metadata
alignfunction might want to wait for the public exposing of aValidAligntype which is restricted to powers of two. - The metadata
sizefunction is fallible because it's possible to safely construct*const [T]describing too-large objects. - The metadata
alignfunction is infallible because I don't see any utility in dynamic alignment requirements, but perhaps it should be fallible for uniformity? - The potential ability to add
Pointeeas a default bound and makeextern typebe!PointeeI find intriguing. Doing so might be able to unblock further experimentation withextern type, as the current direction seems to be forbidding the use ofextern typein generics which do not opt-in to supporting extern types, if opting in is allowed at all. In such a case, perhaps the trait really should evolve to beDynSized.
Links and related work
- Tracking Issue for pointer metadata APIs rust#81513
- Implement RFC 2580: Pointer metadata & VTable rust#81172
- RFC: Pointer metadata & VTable rfcs#2580
- Implement pointee metadata unsizing via a TypedMetadata<T> container rust#97052
What happens now?
This issue is part of the libs-api team API change proposal process. Once this issue is filed the libs-api team will review open proposals in its weekly meeting. You should receive feedback within a week or two.
Footnotes
-
Asserted without proof. ↩