Stop using REST for state synchronization

Arguably, the example isn’t REST because it’s not using HATEOAS, it’s just RPC over HTTP. And this is important because if it were using HATEOAS, you wouldn’t have to worry about state synchronization. In the case of two requests arriving out of order, the client knows which one took effect last, because its UI is determined by the last piece of state to arrive back from the server. You might, indeed, overwrite B with A, but unlike in the example, your UI would be consistent, and show A to the user.

This probably does have implications for UI design, and quite possibly the designer’s preferred UI is not really compatible with REST, and they actually do have to do state synchronization, on their own. But that doesn’t have anything to do with REST, it’s a product of deciding not to use REST but then also not using something well suited for state synchronization.

My overall feedback to this would be to encourage reading https://htmx.org/essays/how-did-rest-come-to-mean-the-opposite-of-rest/

OP is not doing REST, they are just doing RPCs. It’s more accurate to call this post ‘Stop using RPC for state synchronization’, and subtitle ‘Start using real-time streaming instead’.

One solution to this problem is to require the user to hit a “submit” button, which we disable while requests are in-flight, so that we cannot send the second request before we’ve received the response to the first request. This is arguably bad UX though, as evidenced by most highly polished UIs (e.g. your browser’s preference page) not requiring you to do this.

Disabling the button while the request is in flight is a perfectly acceptable solution. The browser’s preference page has no bearing on it because it’s not making a web request.

hmmm yeah, that seems like a reasonable thing to want

I started reading this kind of predisposed to not like it, because I found the title inflammatory…. I guess I fell for clickbait, but I’m pleased that the actual thing it’s saying is reasonable

To ground this in an older theory, distributed transactions are hard, and distributed transactions with optimistic concurrency are even harder. To me, this code wants to have a transactional UX without doing any of the work needed to make the distributed transaction be transactional. The sync-based version of it would solve this by not doing a transaction at all, but doing an automatic merge instead. But in reality I don’t know that you can always do an automatic merge, so the UX will still be inadequate sometimes if the work is not done to address that case.

Obviously, hardly anyone ever solves this “correctly”, and yet the web seems to be pretty successful. It sure can be clunky at times, though.

Is part of the issue with the state “transfer” that it’s treated as unconditional?

Without going down the route of CRDTs, perhaps conditional updates is sufficient for most simple application cases. One approach is using ETAGs when retrieving content, then only permissing updates when passing the ETAG and failing if another modification has been made since the last load. Another simple approach is using PATCH semantics where you can target an update of a specific sub-property, again with a condition to only perform the update if the value is the previously observed value.

Every highly interactive, database-driven web application I’ve ever worked on has struggled with this problem. I advocated for GraphQL in one project but found that it introduced almost as many problems as it solved. React’s maintainer’s have been pushing server functions for this sort of thing lately, but I am not convinced that the benefits of a magical API that has to be invoked with a pragma outweigh the burden of its opacity.

Elm’s creator, Evan Czaplicki recently disclosed that he has spent a great deal of time lately thinking about and working on this problem. I realize the disclosure was accidental and is not in any way a commitment to a solution, but the fact that he’s thinking about it inspires me to wonder what an Elm-like solution might look like.

I think REST is a great base for state synchronization, having built a couple of systems like that.

Not by itself, but you don’t need that much else. Primarily, you also use In-Process-REST to organize your in-process state. Which mostly just means your internal data gets URIs.

You also need a channel to communicate changed URIs. This isn’t hard.

And then you map internal to external URIs and back. Also not hard.

With that, the sync becomes generic: something changes, send the URI of the thing that changed to the sync engine. It then pushes that data where it’s needed or communicates the changed URI to other interested parties that can pull the data.

Blackbird: A reference architecture for local-first connected mobile apps. Not limited to mobile apps.

So yes, you need some additional elements, but they are all based on what we get from REST in the first place.

Interesting topic!

There is actually a solution for this, although to the author’s point it increases the boilerplate needed:

Unfortunately, HTTP does not guarantee that the requests also arrive in this order at our server.

JavaScript fetch can take an AbortController[1] which will allow you to cancel a request in-flight.

[1] https://developer.mozilla.org/en-US/docs/Web/API/AbortController

I think it’s fine to be frustrated about using HTTP for state synchronization, but you’re leaving the reason for why we are here.

For real applications, these POST requests are doing a lot more than simply transferring state. They are running authentication, authorization, validation, and then potentially returning errors pertaining to this specific procedure call. They are transactional.

This means that we might first save “B” to our database and then overwrite it with the older value “A” even though the user meant the final value to be “B”.

You solve this for RPC with locking, no other way.

I do still think you’re on to something here, but it’s not replacing HTTP/RPC with state synchronization, it’s complementing it with state synchronization.

Keep the RPC, it’s unavoidable, but stop returning data that frontend state is updated from (you still need to return the errors with the RPC call though). Instead, you have a separate process to synchronize state, decoupled from RPC.

This isn’t novel, I’ve seen it implemented plenty of times, usually in the context of “real-time” data blasted out to multiple users, which is just a forcing function for decoupled state synchronization. If you have a piece of data you want real-time collaboration on, now you can start talking about CRDT. None of that can be transactional though, it must be decoupled from your RPC stuff.

I think this concept is what has me jazzed about SpacetimeDB. I’ve been tinkering with it and aside from me being bad at React, it has been super neat to work with. Obviously I have no idea how it scales and it’s far less-known than Postgres, but for my hobby projects it really fills this role.