What Functional Programmers Get Wrong About Systems
35 points by typesanitizer
35 points by typesanitizer
Sigh
This is just another rehashing of the very old 'but dynamic type systems model the real world better because the real world is dynamic' arguments that 'No, dynamic type systems are not inherently more open' does an excellent job of eviscerating.
The problem is in an axiom assumed by the blog post: type systems (which one can extend to things like functional purity) do not exist to model the real world, they exist to label it such that we can usefully start talking about it.
If your types / program flow is not sufficiently capturing the complexity that's present in your domain's data model / execution environment, it means you need to do more work modelling it properly! There's no free lunch, and throwing around any types won't solve your problems!
Dude, it’s basically what GPT spits out if you give a few bullet points of unbridled type theory enthusiasm. It’s eactly rehashing lol
One of the points are cross version correctness when deployments are rolling out or when a message in a work queue has been submitted on a different version then what is now processing it (either older or newer due to rollbacks and deployments).
I found TigerBeetle's take on the same problem interesting: They include the old logic when they upgrade so that a new binary can handle the N previous versions as well, and they stamp each operation with the version so that it's always processed by the same version. Details are available over at Jepsen: https://jepsen.io/analyses/tigerbeetle-0.16.11
The relevant section from Jepsen:
TigerBeetle also offers a noteworthy approach to upgrades. Each TigerBeetle binary includes the code not just for that particular version, but several previous versions. For example, the 0.16.21 binary can run 0.16.17, 0.16.18, and so on through 0.16.21. To upgrade, one simply replaces the binary on disk. TigerBeetle loads the new binary, but continues running with the current version. It then coordinates across the cluster to smoothly roll out each successive version, until all nodes are running the latest version available. This approach does not require operators to carefully sequence the upgrade process. Instead, upgrades are performed automatically, and coupled to the replicated state machine. This also allows TigerBeetle to ensure that an operation which commits on version x will never commit on any other version—guarding against state divergence.
I have a hint the long text was not written by a human.
Humans have been writing long texts about topics of deep interest for a long time, what's new here.
I don't feel any obvious slop, the article has a good and coherent idea, with multiple examples and non-obvious interconnections, the author is clearly passionate about the topic. If it's slop, it must be a very new variety. If it's not, you've just casually dismissed a lot of human effort.
(Submitter here): I think the timing of several long blog posts under /engineering (https://www.iankduncan.com/engineering) on the blog provides a fairly strong hint that AI assistance was used (along with some of the writing tics). However, the examples + citations provided in the blog post made me think/connect a few different ideas, hence I decided to share the post here.
Here's the author's response to that, fwiw. https://news.ycombinator.com/item?id=46957262
quite a backlash here. https://news.ycombinator.com/item?id=46953491
It would explain why the information density is so low. I got about halfway through hoping for something clever, then started scanning, then just gave up. 10k words for "migrating code and data is a headache". Yeah. Next.
You read a series of solid bullet points fleshed out semi-competently using AI, and given a slight human touch. It should be a quarter of this length at least.
This falls super flat for me, because the LLM that wrote this spends all of its time seemingly attacking a straw man.
Note how the author didn't deny it was written/rewritten by an LLM. The following paragraph gave it away though:
This is where the F1/Erlang “two versions apart” assumption breaks down entirely. You aren’t dealing with a bounded version window. You’re dealing with an unbounded archaeological record of every serialization format your system has ever used.
Here's one of his posts from 2021. https://www.iankduncan.com/personal/2021-10-04-garbage-in-never-out.
It is entirely possible that this is AI written. If so, it's less overt than most AI writing. But I also find it plausible that the guy who inserted Tao Te Ching quotes in his writeup of Haskell space leaks and adds headlines like "When Helping Hurts" just has a style that regrettably looks a bit like AI.
As someone with a small child, about to go on parental leave in a few months, it's hard to see how you can get more time to finish technical writing compared to a tech job. Weird
Great post. Systemic thinking is really the key to software engineering.
I'm going to go out on a limb: this is a very good point, and it has nothing to do with functional programming (and it sounds like the author also agrees). We build systems out of independent programs which get mapped to independent processes at runtime. Every language is like this, it's not just FP.
This is why TLA+ was such a breath of fresh air when I started looking into it. One of the main books about it is called Specifying Systems. In it, you learn how to model things like asynchronous memory access, which is directly applicable to modern web applications.
This is what led me to explore more system-level integration tests, like as described in a post I wrote about model-based testing web applications. This aligns with the trend that Antithesis and other folks are pushing around deterministic simulation testing. One of the core beliefs of DST is that you should test your system as end to end as possible. This is how you can check for system-level invariants, which (as this post says) is where much of the complexity lies.
Of course, as soon as you try to do this, you see why we focus on programs: individual programs are easier to control. Once a full system is deployed, it's a whole other beast entirely. But: it can be tamed, if you move all non-determinism outside of the system.
Got about 2/3 through and didn't feel like it was getting to an actual problem. Seems like the only thing that would satisfy the "problem" is an omniscient and omnipresent computer.
It's not exactly "wrong", it's more about a typical type enthusiast having little idea of distributed systems in practice.
Clickbait title aside, I agree that some kind of "system-wide" and "version-wide" typing would be nice, and protobuf is not it. Erlang gets away with gradual typing, Unison does try something new. Maybe some kind of CTL invariants with "until" and "forever" could help. Maybe a richer data constraint language.
18 pages of text (based on print preview), with no summary whatsoever. I find it hard to invest this much time without any clue if it will be worth it.
Here is an AI summary, if you are like me and would rather know in advance what you are looking at.
Prompt: Please summarize this blog post in 5 sentences: [...]
Please help me understand what’s wrong with this comment of mine. There clearly is as I see ‘2 off-topic, 6 spam, 1 troll’. I genuinely miss summaries for long articles as I have no clue otherwise what it is about. We are clearly not banning titles either, so there must be some value in having a summary.
So what is the problem? The summary? That it was AI generated? My message preciding it?
Appreciate any directions, again, I must be blind to the obvious.
Don't post slop.
Slop enthusiasts can readily generate their own; for everyone else you're posting the equivalent of an unasked-for "let me google that for you" that simultaneously demonstrates you couldn't be bothered to read the article yourself.
Ah I see. Thanks for pointing it out, makes sense I guess.
It’s not fair to say I couldn’t be bothered though. You can’t read everything, it’s about picking what you do read.
Thanks again!
You are in for serious surprise if all your reading is summarized by AI. Specially if you drop it on ChatGPT or another customized AI. It won’t tell the difference between your own ideas and what’s in the weights, and you will get subtly different readings from what the author intended. And it’s not what delleuze meant