What is `std::pin::Pin` in Rust?
14 points by vrongmeal
14 points by vrongmeal
std::pin::Pin is the Monad of the Rust world. Once you understand it you are forced to write a blogpost about it.
Might be good to address a couple of things that hit me (and others) when trying to understand Pin:
Unpin is not great (I don't think many would disagree, but no better alternatives were found). More-accurate, but otherwise bad, names would be MovableWhenPinned or PinIsNoOp.!Unpin in nightly looks weird. But it has to be that way, because to default existing types to the 99% case, an auto-trait had to be added (Unpin) that types can opt out of. (Imagine it as !MovableWhenPinned and it makes more sense.)PhantomPinned also has a not-great name, because being pinned is a temporary state caused by having a pinned reference, not a characteristic of a type. The alternative name would be PhantomNotMovableWhenPinned.Once I started doing these mental translations things made more sense to me. Of course, maybe I'm still confused and have just been lucky! :)
I feel I’ve asked this before and someone gave a thoughtful reply.. but I can’t remember it. Pin, as it was explained to me, came out of async where local variable references become self-references in the blob of data that represents the state machine for a given function.
Ok fine, if the async state were moved, all those local variable references would point to the old invalid location.
But, that’s only true because they are real pointers, like full absolute references. My question is: Why was the solution to this to remove the ability to move, instead of making the references be relative?
Is the answer mostly “because there is a million engineer years gone into the compiler and CPU and OS really, really, really getting pointers”, so pointers are just better in so many ways that its better to have to Pin everywhere, or is there some real hard reason that makes relative references not actually work as an alternative solution?
It's not just that local variables inside an async state can directly reference other local variables inside that same state - that would be fine since the compiler knows about all those locals and could make accesses relative. But imagine how tricky it gets when you have a reference deep inside one of those types which points to another value deep inside another type.
If those references were relative, those types would have to have different memory representations depending on whether they're used from inside an async state or not, and there would need to be some notion of a base pointer you also have to pass around to recover the actual pointer from those relative references.
Nested objects within a pinned reference can still be freely moved around even when the root object is pinned too, so these hypothetical relative references aren't even necessarily all relative to the same base pointer.
Ok so absolute pointers is a must, relative references can't really work out. Well, the Rust compiler knows about all the types here so what if you changed the approach and made objects moveable by tracing through that whole object graph and fixing up references to moved objects to point to the new location? Oops, you've just built a tracing GC :)
Then there's the problem that the Rust compiler doesn't know about all the types in an object graph - references can be passed through FFI and foreign libraries can retain those references. Fixing up moving references through the FFI boundary is basically an intractable problem.
So it's really tricky. It's also worth noting that moving objects is a (relatively) new technique. In most C and C++ programs all objects can be considered to be implicitly pinned. Pinning is not discussed as much there because objects simply don't move, or if they do move it's up to you as the programmer to make sure no dangling references are left behind.
Pin is also necessary for interop with other languages where rust cannot simply move memory around as if they're opaque buckets of bits. As I understand it, part of the problem with C++ interop is that objects are not simply buckets of bits that can be moved around freely, essentially requiring pinning for some large number of types, and the ergonomics around that are unpleasant.
But that comes from talking to some of the folk working on this at least 6 months ago, so I don't know how much that story has been improved.
Probably worth noting somewhere in this post that !UnPin is only expressible in nightly rust, which is the main reason PhantomPinned is a thing.
Overall I think this is a good explainer to have in addition to the official Rust documentation, eases into the issues a bit nicer.
However, I think leading with the self-referential struct makes things more confusing than if it had been omitted. Specifically, in the introduction:
Hence, we need a way to prevent moving
SelfRefonce those self-references have been established.
started me thinking about the wrong problem, preventing moves entirely, when really the core of it is explained much later:
Pindoes not physically prevent values from moving. Instead, it is a type-level guarantee that the value will not be moved through that pointer.
It is not possible to prevent moves, so we use Pin to expose self-referential data only behind exclusive references in a safe API. Maybe I understand Pin too much already, but maybe tweaking the presentation slightly could help readers not get lost.