Bespoke software is the future
19 points by abhin4v
19 points by abhin4v
there’s a running joke in the financial sector that cryptocurrencies are a grand experiment in learning the history of financial regulation from first principles.
i always thought that was funny but also like, okay maybe a bunch of software engineers got excited about what they viewed as a simpler model of finance unfettered by an arcane and confusing regulatory apparatus that felt unnecessary in the course of their interactions with it.
so it’s been breathtaking to see this play out in the context of LLMs ushering in an era of “bespoke” software and a rugged individualism free from the restrictions of abstraction and the lessons learned over the lifetime of a project.
it’s one thing to not understand the nuance of another area of subject matter expertise, but seeing other folks in my industry make the same mistake for an even shorter window of time in an area that they are intimately familiar with is kind of bleak!
EDIT: and just to be clear, this isn’t me complaining about folks who want to use LLMs as a way to learn something new that might otherwise feel too daunting to approach on their own, which is how linking and loading definitely come across at first blush.
this is about a style of rhetoric about software engineering that appears to be cropping up which points to projects like LLVM and says “wow that’s so complicated, would that we could just build something a little simpler” and then proceeds to reinvent the world based on their own (often narrow) perspective of the problem domain.
this is about a style of rhetoric about software engineering that appears to be cropping up which points to projects like LLVM and says “wow that’s so complicated, would that we could just build something a little simpler” and then proceeds to reinvent the world based on their own (often narrow) perspective of the problem domain.
Quite frankly, it is not a surprise to me that this mindset is so common, and it is not a surprise to me that this mindset is so prevalent in the LLM-space.The roots of the modern "AI hype" come not just from the technology, but also from people funding the technology — Sam Altman, Elon Musk, and so on, and you can trace the ideological thread back to the origin — Eliezer Yudkowsky. Yudkowsky, in his Less Wrong group, essentially laid a lot of the ideological frameworks for a lot of modern Silicon Valley thinking, including attitudes to AI — that Evil AI is dangerous, and will eat the world, for example. His group is where the concept of "Roko's Baselisk" came from, and the Effective Altruism movement urged rich people such as Elon Musk to consider the "threat of AI" over things like poverty, systemic problems, etc. This is why you get Sam Altman up on a stage talking about the possibility of "Evil AI", and OpenAI as a whole was founded from the idea that they needed to make a "Good AI" first.
The baseline tl;dr of Less Wrong is — "Computer scientists misunderstand philosophy and reinvent it wholesale from scratch based on the writings of one person". You may notice certain names in that article, such as Scott Alexander, etc. being very prominent within the tech community, and also within right-wing circles. This is also unsurprising to me, given how deeply all of these people are embedded (either in the periphery of, or wholesale) within the Neo-reactionary Movement — from what I can tell, there's an almost direct line between Elon Musk falling into the Less Wrong orbit, socialising with those groups, and subsequently getting on stage and heiling Hitler. Many figures within the Neo-reactionary movement, were also on the periphery of Less Wrong, or writing posts in that space.
That the prevailing ideology within these groups is "This is so complex, we are smarter, we can do it better and simpler", is also unsurprising. Less Wrong did (does?) a lot of that. These were, after all, the people that thought MetaMed ("Medical consultancy with no actual medical experts involved"), was a good idea. Many of them thought Cryptocurrency and NFTs were a good idea. It really doesn't surprise me that they think that Generative AI is a good idea, at this point — it is completely predictable and in-line with their sociocultural, and subsequent ideological threads.
The thought experiment is this: If you had a compiler that only targeted a single architecture, libc, userland and OS. Could you devise something different? Right now the economics are that we all use general purpose built tooling.
I think LLMs help change the economics so companies can pursue tooling more purpose built and explore the space more.
economics is not the reason why we use general-purpose tooling except in the sense that economics is as much a social science as a technical one, and as such economic theory can apply generally to how we collectively solve coordination problems as an industry.
EDIT: or, as a friend put it:
r: llvm was a reaction to it being like that
r: it was already like that and everyone hated it being like that because as soon as they had to interact with multiple platforms they had to do gymnastics
I agree -- Economics is "a" reason vs "the" reason.
My thought experiment just changes the scope of the collective solving is my thought experiment. There does not need to be a grand-unifying theory on how a single compiler works well for all cases. I was just trying to articulate the old "square peg, round hole" idea.
it was seen as more sustainable in that context to pool resources to produce a single polyglot compiler backend rather than each compiler rolling their own.
If any such "bespoke" experiment will need to support multiple platforms, they will learn the same lessons of LLVM the hard way (precisely as your cryptocurrency example)
however I find the mere possibility of making a very specific piece of sw in a short time to be quite exciting too, by itself.
I am amazed at people saying „I couldn’t write the software myself, but I‘m perfectly capable of auditing and debugging it.“
Honestly, this can be true - the frame of mind required to build and create is kind of different to the frame of mind that reads critically and looks for exceptions in everything.
Maybe a bit? I remain doubtful.
Reviewing like the shallow PR review in an enterprise team workflow? Yeah, that works.
Reviewing as in making sure that everything will work according to spec (however informal)? Much harder and time intensive and most likely to get wrong.
Debugging? This will be absolute hell.
I think there’s a parallel between algorithm driven feeds and vibe coded bespoke software. You may get exactly what you want from the spectrum of options the algorithm / LLM can produce, but it will be limited to that spectrum and prone to outside influences like ads. I think it’s also going to be interesting from a social aspect to see if we get “echo chambers” of bespoke apps where fewer people have shared digital experiences. More options, more individual choices, but less mutual understanding.
Instead, I leveraged LLMs to help me design and write µld, a tiny opinionated linker in Rust that only targets ELF, x86_64, static linking and barebone feature-set.
It shouldn’t be a surprise to anyone that the end result is a codebase that I can audit, educate myself and can easily grow to support additional improvements and optimizations.
The surprising bit, especially to me, was how easy it was to author and write within a very short period of time (1-2 days).
That means smaller companies, without the coffer of similar FAANG companies, can also pursue bespoke custom tailored software for their needs
Assuming that the author means this seriously, when considering the given example of a linker, why does it make sense for a small company to bet on a bespoke linker for x86_64 Linux only that doesn't support debug info etc. instead of using a standard OSS linker?
The argument about "additional improvements and optimizations" doesn't apply to small companies -- the vast majority of them have enough on their plate that they're not interested in contributing to foundational infra like compilers/linkers/databases etc (and that they can manage without making these changes is a strength of existing tools!).
It's going to be an unpopular opinion based on the comments, but the article raises good points in that "cooking software out of cans" often brings more abstractions than necessary and reduces overall cohesion. A CTO friend said to me that most of the time the price in overhead and architectural discrepencies is not worth it when you can spend a couple of days (or even weeks) building a custom solution that fits your needs like a glove. LLMs are indeed liberating here as they can help draft a solution, but also maintain and document it. Now the counterpoint to that is that when tools are designed to do one job and do it well, and are easily compostable (UNIX style), then there's little reason to not reuse them. Anyway, I'm quite biased as I do prefer control over the code, and have most often than not regretted reusing a third party library/framework as opposed to building custom. This is definitely not an opinion shared by the majority of engineers, but I sometimes wonder if it's simply because they can't create easily, and the prospect of doing so scares them. I am much more scared of being locked in and ending up having to rewrite because I've reached a dead end due to third party code.
A CTO friend said to me that most of the time the price in overhead and architectural discrepencies is not worth it when you can spend a couple of days (or even weeks) building a custom solution that fits your needs like a glove
There's another aspect to this (also potentially unpopular). If the software ecosystem fractures significantly due to more people writing smaller code bases that fit their specific problem (and that code is kept in-house), it could lead to a higher cost on the part of attackers to understand and find the right vulnerabilities. If today you can find a vuln in libVulnerable v 1.2.3 and hit every Fortune 500 company at once, it makes sense to put the effort in. If you have to go against 500 separate iterations of libVuln, the cost to the attacker goes up and the payoff goes down.
On the flip side, if the attackers are already targeting AI (either by cache poisoning or just understanding where it might make mistakes), maybe there isn't a huge change. Also, each company trying to vibecode its own solutions will now be at risk should the AI provider go offline, so their resiliency is going to go down.
It will be interesting to watch.
My career has followed a trajectory of NIH to a stable balance and on to a kind of "invented here syndrome" where software developers would seemingly rather find a vendor than write bespoke code for our organization. (I'm not saying Software Engineering as a whole has followed this arc; much of it is the kinds of companies I've worked at, although I do suspect SE has followed this arc if you zoom out enough).
I think that often good advice like "don't build your own crypto" / "don't write a database" / "just use AWS" have somehow moved from being good defaults to being dogma. I've always considered there to be an implied "unless you know what you're doing" after these.
The start of my career was bespoke software development; literally writing ecommerce websites from a blank directory and composing a new "Cart.pm" or "Cart.php" which was specific to the customer's business. Was that a great use of time? Possibly not! But now if you do anything in ecommerce you have to push hard against "just use Shopify" as if no one should do their own commerce. I expect this from product managers, but I hear it from software developers. There's a host of reasons that's bad, but given the article I'm just sad about the self-imposed limits of what we think we should be doing in our craft.
I do think a lot of people think we should be cooking from cans. Once we have one flavour of library that does something why would we need anything else?
I think that often good advice like "don't build your own crypto" / "don't write a database" / "just use AWS" have somehow moved from being good defaults to being dogma. I've always considered there to be an implied "unless you know what you're doing" after these.
The implied part should really be, “Unless you know what you’re doing, and can maintain it in the long run.” I’ve seen too many custom solutions become problematic because there is nobody left with the skills and knowledge to make changes or enhancements. That often also means you don’t have the skills needed to move away from that custom solution.
100% agree, although I also think that this applies (almost) equally to depending on other people's software. Software needs maintenance no matter who wrote it.
A few recent examples for me. I'm not trying to "but, actually" here; I think you're broadly correct but just like everything else: it's depends. I still maintain that the push towards using other people's code can be just as short-sighted (who's maintaining that in the long run).
I was at Internet Archive and had two "weird" databases.
"trough" is a distributed SQLite-over-HDFS service written in Python. It's maybe two thousand lines long. It was custom developed by two folks who left and core to a service I was working on. After years of minimal changes and putting up with actual bugs, it took a pretty intensive two months for me to fix a lot of the issues with it, because (despite minimal documentation) it only actually does what IA needed it to do. It was: unmaintained, custom, but small. While we intended to replace trough, it could be fixed in the mean time because it fits inside one developer's head. It had minimum dependencies and bringing it from Python 3.5 to Python 3.10 was a week or two's work.
"RethinkDB" was also used for two services. It's a massive C++ codebase that was donated to the ASF and is on minimal life-support. Before then it was a VC funded competitor to MongoDB that had several full time devs with experience in this space. They simply lost the NoSQL wars. The original authors have moved on and the volunteers who support is now mostly fix broken builds caused by dependency updates. It's technically maintained, generic and massive.
It also sometimes segfaults under load, triggering multi-hour automatic rebalance operations. There's basically no stabilizing it because it's is very complex. IA only uses about 10% of its functionality, but we're running 100% of its code. The plan (when I left) was to take on moving to an entirely different database.
At my current place, we have a Rails monolith with many dependencies. Rails itself is a great call, but some of these dependencies are big and we use a small subset of their functionality. Some of these deps are coming up on a decade out of date and the task of updating the whole thing is massive. That's our fault: we didn't do regular maintenance.
But part of the reason for that is that none of this maintenance brought (significant) benefit to the company, so it wasn't prioritised. This is both reckless and logical. If the cost of importing a library that kinda-does what you want (or does what you want but also 10 things you don't care about) is ongoing and increasing maintenance, I think it's good to think "could we build this ourselves and then we only care about fixes to bugs that affect us" and not be urgently importing a bugfix for code that we never use, just so we can stay on the right side of PCI-DSS auditors.
This really resonated "somehow moved from being good defaults to being dogma", as well as "self-imposed limits". Maybe it's a cultural/generational thing (I'm getting old), and that people who wrote software before the internet and tons of open source projects have grown up with no other option than rolling your own?
Your last question made me think of Web UIs and React: it's so prevalent that now it does feel like a heresy to even consider something else -- and the vast majority of frameworks are aligning to a similar model and API (eg. Solid). Like you, I do find it a bit sad, and sort of hope that LLMs will unlock new possibilities for the new generations of engineers.
All these years wasted on concepts like "secure software", "less error prone programming practices", and in one fell swoop now everyone can make their own app that stores their credit card information and passwords in plain text! amazing /s
The blog post is about low level tooling like linkers, I don't see where credit card info comes into that
Edit: My apologies, I thought this was another thread!
Here's a proper response.
Linkers, loaders, compilers, and other low level tooling are important for the rest of the stack to function properly. They are tools that might, later down the line, end up used in critical performance applications — in financial software, in aerospace software. Even small bugs in regular consumer-grade software can end up in payment systems or in social security systems, to say nothing of surveillance software. Even minor errors that crop up in tools like this can bubble up the stack and cause unpredictable behaviour, that may result in human harm in unexpected ways. This isn't quite Therac-25, but it still might, down the line, result in someone's social security being mishandled, their payment data mishandled, their password that they use for everything breached, etc. All because a symbol wasn't imported, or the wrong symbol was selected for dynamic dispatch, or something else. We have previous examples of even minor software errors severely fucking over people's lives:
https://en.wikipedia.org/wiki/British_Post_Office_scandal — bugs in a processing system resulted in several suicides and people becoming unhirable.
https://www.buzzfeednews.com/article/charliewarzel/it-gets-worse-the-newest-sony-data-breach-exposes-thousands — improperly designed system leaked hundreds of thousands of passwords
https://www.motorbiscuit.com/null-license-plate-dmv-tickets/ — null handling error results in thousands of dollars of fines
https://tech.yahoo.com/computing/articles/woman-whose-last-name-null-164558254.html — same here
All of these are improperly designed systems, or systems where input data was mishandled.
The utopic improvement to security, if it's to be believed, is a practical path away from our unsafe languages, unsafe auth schemes, unsafe ambient privileges, etc.
Brb writing my own database that does exactly what I need and nothing I don’t. Maintenance will be easy so I can also make my own compiler.
There’s always a balance to be struck between too much integration and too much bespoke. That’s engineering.
How will anyone be able to learn anything anymore? If you use a bespoke web server, database, programming language for everything it might fit the problem domain perfectly, but how are you going to learn the quirks of a system? Also you cannot easily hire new software developers, because they use a different set of bespoke software.
I think what the author misses is when to draw the line. I much rather program a few hundred lines myself than adding another dependency and sometimes I just copy over the code if I just need a specific function, but I don‘t think it works as general concept with LLMs.
This feature is well-suited for tooling such as Nix.
Maybe it was a bad fit for us, but Nix was a terrible way to build software for a small team working on a multi-language repository.
I shudder every time I remember dealing with it: from the syntax, to flakes, to incorrect caching causing havoc, everything being slow, almost zero community support for anything we needed... the list goes on and on.
Thought provoking idea, thanks.
The problem statement is that today's tech has lots of accidental complexity, which I expect we all agree with.
The problem with these generalized solutions is that, well, they are generalized and rarely fit well together.
I agree. As an example, I'm looking for a WebDAV server that does auth via JWT, with static JSON config, and ideally in a language I can write in case I want to change it.
I can find projects that meet some requirements, but not all. I will likely end up compromising, resulting in weak integration with the rest of my system. Even though I've not written the program, the integration is very much DIY.
Meanwhile, the ideal program isn't that complex: there are decent WebDAV and JWT libraries, and their composition isn't too fiddly.
Further, I'd rather be composing those libraries than shrinking a program to my fit infra, because:
But I will likely pick an off-the-shelf solution because it's easier to get started and the payoff is more certain.
If development was 10x faster, I might DIY.
Will LLMs deliver 10x faster development? No idea!
I find this argument is best for personal software.
For example while messing with my dotfiles I noticed a certain program didn't play nice with my setup. With a few prompts I was able to make a patch. Like the author said since I use nixos applying the patch was trivial and just works.