Everybody's so Creative
56 points by weinzierl
56 points by weinzierl
Rust REALLY doesn’t like you hiding details. But libraries are all about abstracting away implementation details.
Something not mentioned in this post, but it’s related: Breaking changes, specifically interfaces. For all the safety rust gives you, library authors tend to be extremely conservative about changing interfaces. Some of this over-abstraction is guarding against internal changes leaking out to user breaking changes.
If you want simple interfaces (reduced abstraction), you need to be willing to accommodate more breaking changes from libraries. And as a community we need to find a balance point between “abstract enough” and “stable enough” and “clear enough.”
this is nowadays a fairly popular sentiment but it'd be more productive moving forward to give concrete examples of useless traits and what you'd replace them with. I found that half of the time when I am trying to think about how to simplify an API like that, I end up with very baroque early-Java-style patterns and optionally a prohibitive amount of trait objects. I'm sure somebody would complain about that too when some code isn't inlined properly.
There are some commonly used crates that are just notoriously overabstracted - base64 comically so, but also rand and the rust-crypto crates. Some of these have inspired simpler alternatives that satisfy most users' needs with basic functions (e.g. getrandom).
But another example that bothers me is parking_lot, an excellent crate with some bad API decisions. parking_lot::RwLock is a type alias for lock_api::RwLock parameterized by parking_lot::RawRwLock, which implements lock_api::RawRwLock - four different items from two different crates, reusing the same two names. lock_api is another package maintained by the same owner as parking_lot with the noble goal of re-using the code which translates low level locking operations into a lock which protects a specific piece of data, but the fact that parking_lot uses lock_api should be hidden by making parking_lot::RwLock a struct, not a type alias.
I see this pattern fairly often - the types people are meant to use are type aliases over basically the product of a couple library-internal components of those types. This is greater for you, the library author: you don't have to write a bunch of boilerplate methods on the type. But its terrible for me, the library consumer: now I need to understand all of the components of your library and how they fit together in order to follow your public API.
Rather than everyone being "too creative," I have the impression this is the problem with the Rust open source ecosystem: people are not being empathetic to users who don't already know how their library works and shouldn't need to learn all of it in order to know where to begin. Rust libraries are too often like remotes with too many buttons.
I think this gets at a really specific issue that I agree with: type aliases almost universally make libraries harder to understand. But type aliases are almost the opposite of making the code abstract!
Yes: the word abstraction is being used in a rather unclear way in this discussion. We’re actually talking about libraries that expose optionality for abstraction by their users, not libraries that provide abstractions.
To me the issue isn't having too many abstractions necessarily but in not providing a usability layer over the top. The rand crate is a good example in both a good way and a bad way. It is very flexible and I'm sure each of the lesser used distributions are well-motovated, even if less common. They do offer an easy path if you just want something like a string of characters uniformly sampled from a prng. However the way rust docs are formatted by default makes it somewhat difficult to see the easy path first. I would like to see some of these easy paths extracted into a crate that could be the crate directly depended on by most apps that have mainstream use cases, in which case users can more easily consume narrower documentation. Then only have to face the web of abstraction if their use case requires it.
The problem is that there's already enough pressure to reduce the number of dependencies. I could see a feature flag on the same crate, or a push for better rustdoc... but when I'm re-inventing minor wheels (eg. prettier/friendlier errors) to avoid an extra dependency while simultaneously arguing against people coming from C++ who want a Boost or a Qt, that's a data point.
However the way rust docs are formatted by default makes it somewhat difficult to see the easy path first.
What kind of rustdoc features would help with highlighting the simple, common-case API to the users?
As you say, the crate could be split into modules (or multiple crates) with different modules/crates for common-case simple usage and advanced usage. No support from rustdoc needed.
Also to be fair to this crate, they have a "Quick Start" section in the documentation that shows the simple usage.
There are simple things like generating graphs of type and trait relationships and being able to more quickly click through a long list of generic trait implementations to see if the constraints are compatible with a given use.
One nice feature would be documentation navigation prefixes that would work similar to how namespaces serve the documentation of other languages. There isn't going to be a 1:1 here because of the role of the crate as the unit of compilation, the interaction with traits, and the module tree. Whatever it was called, it would be nice to have modules within a crate tagged with additional navigation data. One of those could be "breadcrumbs" so that there could be a navigation thee separate from the module tree. The module tree might look like:
-lib
- models
- wind
- particulates
- solar
- measures
- temp
- fluidvelocity
- geo
- forecasts
But your navigation tree could be:
- Easy weather forecasting
- Do it yourself
Then whichever of those navigation nodes you click on would determine two things. Some types and functions could appear in only one of them or both but have a primary. When you navigate between them there would be a visual indication that you've left the easy path or accidentally got back on it. Any navigation lists could also use these in grouping and ordering. That way when I see a list of 20 trait implementations the one I want would be much easier to find.
I don't agree at all, in fact I find the opposite. When searching crates.io for some purpose, I constantly find libraries which technically do the thing but have all kinds of arbitrary choices of concrete types baked into their API which conflict with the requirements or design of my use case. It's like people are so used to using languages with terrible abstraction capabilities, that they have not yet realized that it's actually possible to write high quality libraries in Rust.
If performance and correctness are not important, maybe you don't need to use Rust?
Caveats:
Rust's cutesy OO-imitating traits sometimes necessitate strange contortions that would not be an issue if they were more like Haskell's type classes or ML's modules (neither of which attach to a single privileged type).
I think this is one of the unfortunate design decisions in Rust. My guess is they wanted to make traits look like OOP style interfaces/abstract classes (with a receiver type and argument, called with receiver.method(...) syntax) because that's familiar to many, but traits (typeclasses) can dispatch based on multiple types and the Rust syntax for multiple-parameter traits is somewhat confusing and it certainly seems quite weird that the first type parameter of a trait is special.
I copy many things from Rust in my programming language (including traits), but in trait and impl declarations and predicates (constraints) I use Haskell syntax instead of Rust.
I still have method call syntax, but it works for any function not just trait methods. In x.f(a, b, c) I find an f in scope with a matching first argument type and arity. When there's ambiguity ask the user to disambiguate.
I still have method call syntax, but it works for any function not just trait methods. In x.f(a, b, c) I find an f in scope with a matching first argument type and arity. When there's ambiguity ask the user to disambiguate.
I'd worry about the effects that would have on "fearless upgrades". Rust is already vulnerable to "one of the dependencies added a function to a trait and now the code won't compile anymore without resolving the ambiguity" and this sounds like it'd make that risk more common.
Yeah, additions becoming a breaking change is certainly annoying, but I think we may be able to avoid it for the most part.
I also don't know how else to solve the problems I'm trying to solve:
Problem 1: It's unclear when to make something a method vs. a function. (note: this is Rust-style methods, not OOP methods)
The only meaningful distinction between them is that they're called with different syntax (x.f() vs. f(x)), but the difference between these different syntax is mostly a matter of taste + syntactic things they allow. (e.g. chaining is easier with methods: x.f().g() vs. g(f(x)))
It's annoying that the library author can dictate how I should be writing my code. E.g. if you define things as methods I have to do x.f().g(). Otherwise I have to do g(f(x)). The user should be able to decide.
Problem 2: I can define a function that takes as argument a type that I don't define, but somehow defining the same function as method on the same type is for some reason not possible. I've never understood the reason why and couldn't find a good reason why this should be the case.
The UFCS-like soultion that I came up with (I don't know if it's the same as D's UFCS, we don't have classes in Fir) is: you allow every function (including trait methods) to be called as methods. It solves both of the issues:
Solves problem 1: you don't have to plan ahead of time how the users will want to call your function, as methods of functions.
Solves problem 2: you don't have the restriction that only the type's defining module (or package etc.) can define methods on it.
The problem with adding a new function becoming a breaking change can be solved by avoiding * imports (the syntax for importing everything from a module). So maybe we get rid of * syntax entirely and require listing all imported things. The tradeoff here is that it'll get verbose. Or just accept that additions are not considered breaking (by convention) and if you're using * imports you accept that a minor version bump can cause breakage.
Even without * syntax adding a trait method can potentially break, same as Rust.
My current thinking is that I'm willing to solve problems (1) and (2) with the UFCS-like thing I describe at the cost of making more things breaking changes. Then attack this problem with (1) community efforts (recommend avoiding * imports when possible, maybe have lints/warnings etc.) (2) tooling support: e.g. always link with the smallest versions of dependencies (Go style), rather than largest (Rust style).
For background: here are the issues where I discuss the problems mentioned above: 1, 2, also somewhat related: 3.
The only meaningful distinction between them is that they're called with different syntax
I vaguely remember "Any method can be called as a function but not all functions can be called as methods" imposing some kind of additional constraint on what semantics the compiler will OK for functions that are also methods.
Problem 2: I can define a function that takes as argument a type that I don't define, but somehow defining the same function as method on the same type is for some reason not possible. I've never understood the reason why and couldn't find a good reason why this should be the case.
Please clarify. To me, what you're describing just sounds like defining an extension trait, like how Rayon adds .par_iter() to standard library collections.
Beyond that, the orphan rule exists to prevent a trait-oriented version of the diamond inheritance problem. See this page for an example of what it avoids.
The problem with adding a new function becoming a breaking change can be solved by avoiding * imports (the syntax for importing everything from a module).
I never use * imports... but use foo::FooTrait as _; is a de facto * import for all the methods on a trait, so you're already at risk of collisions that break "fearless upgrades" if FooTrait adds a new method.
My current thinking is that I'm willing to solve problems (1) and (2) with the UFCS-like thing I describe at the cost of making more things breaking changes.
That's your choice. I came from Python for the fearless upgrades and the strong compile-time correctness, so I disagree. Minimize maintenance burden first and foremost.
Problem 2: I can define a function that takes as argument a type that I don't define, but somehow defining the same function as method on the same type is for some reason not possible. I've never understood the reason why and couldn't find a good reason why this should be the case.
Please clarify. To me, what you're describing just sounds like defining an extension trait, like how Rayon adds .par_iter() to standard library collections.
I can import a type T and then define (using Rust syntax)
fn foo(self_: T) { ... }
But if I want this foo to be a method on T that's out of question for some reason.
I'm saying that there's no reason that I can think of to allow defining the foo function above but not a foo method on T. They're the same thing, just called using different syntax.
Instead of allowing defining methods (which would solve problem (2) but leave problem (1)), I'm eliminating the function/method distinction altogether outside of traits (which solves both problems).
Trait methods can also be used as functions: Trait.method(receiver, arg2, arg3, ...).
Beyond that, the orphan rule exists to prevent a trait-oriented version of the diamond inheritance problem
I wrote Haskell for 8 years, I'm aware of the problems with orphan instances.
What I describe is completely orthogonal to trait implementation rules. I'm talking about non-trait methods and functions in Rust. For traits we can just copy the Rust's orphan implementation rules, I don't know. I'm not at that stage yet.
Minimize maintenance burden first and foremost
PL design is full of tradeoffs at every step. Function call syntax is one of the most basic things, and even there you can't win on all fronts. A language that optimizes maintainability over everything else would be a different language than mine.
That said, my guess is that this part of the language is not going to be a big problem for updates. I work on large OOP code bases, and if I searched for all function and method names (excluding the overloaded ones, those would be trait methods in Rust and my lang) I suspect I would get very little name conflicts, and among the ones that I get there would be very little, if any, functions/methods that have the same first argument type and the same arity. My guess is that's this part of the language is not going to be an issue.
But if I want this foo to be a method on T that's out of question for some reason.
Again, what are you describing that isn't covered by this?
trait Frobnicate {
fn frobnicate(&self);
}
impl Frobnicate for &str {
fn frobnicate(&self) {
println!("FROB: {}", self);
}
}
fn main() {
let a = "foo";
a.frobnicate();
}
What I describe is completely orthogonal to trait implementation rules. I'm talking about non-trait methods and functions in Rust. For traits we can just copy the Rust's orphan implementation rules, I don't know. I'm not at that stage yet.
By the time compilation is finished, there's no difference between trait and non-trait methods unless you're doing dynamic dispatch, which isn't the default. They're both just free functions taking references.
What's so special about non-trait methods that implementing a one-off extension trait isn't good enough?
That said, my guess is that this part of the language is not going to be a big problem for updates. I work on large OOP code bases, and if I searched for all function and method names (excluding the overloaded ones, those would be trait methods in Rust and my lang) I suspect I would get very little name conflicts, and among the ones that I get there would be very little, if any, functions/methods that have the same first argument type and the same arity. My guess is that's this part of the language is not going to be an issue.
Wait, you're doing what is essentially method overloading? (i.e. including arity in the method resolution?)
Rust disallows that because it doesn't compose well with Rust's variation of Hindley–Milner type inference and Rust's approach to time-of-definition-checked generics. For example, see scottmcm's answers in Why Rust doesn’t have function overloading? @ users.rust-lang.org.
Here’s my rule of thumb: keep "Go to Definition" useful.
I agree with the sentiment... but, even once I started using coc.nvim for rust-analyzer instead of just syntax highlighting and ALE's rustfmt-and-clippy-on-save support, I am so "I never use Go To Definition" that I never bothered to set up a keybinding for it. ...so I have no idea how dumb or smart it is. (I use rustdoc's source links and a mix of fzf.vim's buffer switcher and / and that's about it.)
That said, past experience with writing "enterprise OOP" in Python and then having to maintain it has led me to a YAGNI attitude. I think the main reason I "over"-abstract has more to do with wanting to be able to cargo test as much as possible, so I wind up using traits for mocking to avoid winding up at 3AM, discovering that I've written some kind of "Use a 0-day exploit to automatically download and install Docker or whatever as a side-effect of cargo test" abomination.
TL;DR: I agree... what are the DOs and DON'Ts of making "Go To Definition" not fall over? I don't use those newfangled things.
Here’s my rule of thumb: keep "Go to Definition" useful.
I really like this. This metric degraded terribly in Python too since type specs became the norm. Which is probably a sign of the times, devx improved in terms of code writing at the expense of understanding the libs we use.
Tell me more. How did type specs make code definitions harder to follow?
Before you jump into definition and got a function implementation, now you get multiple @overload -> ReturnType: ... declarations of the same function
Not sure what the issue with go to definition is. In RustRover I can find all the implementations easily. It can’t jump directly to the implementation when the type is not known statically, of course, but that seems impossible anyway?
Abstraction really fights with understandability when it comes to library crates providing data structures. You start with a dictionary/map structure or a small string over the specific types you need and think it can be more generic. But to get it generic correctly you’re suddenly doing arithmetic with size_of::<usize>() or hiding details in dependencies. Even if it doesn’t negatively impact “goto definition”, being generic seems to usually cost readability.