There continue to be reasons for software to be slow
11 points by typesanitizer
11 points by typesanitizer
Slow software is usually a consequence of the org chart and conway's law IMO. It's extremely hard to make software that is fast when the team is more than 3 people.
I'm not really sure it's that simple. For example, many databases (Umbra, DuckDB, Clickhouse to name a few) have excellent performance and have been worked on by a large number of people as far as I know. The other examples I can think of are observability tooling backends and search engines.
I know you said "usually", but I think with this kind of generalization, it's useful to try to come up with realistic counter-examples.
For Umbra in particular I can say that a lot more than 50% of the code has been written by a single person. While there have been contributions from a few dozen PhD students, most of these were rather focused, and several projects were also never merged. Nerd sniping and benchmark hunting is also part of the culture that helps building fast systems.
Thanks for adding context. I would've assumed the ratio to be less imbalanced given the size of the research group.
In my own experience it is also hard to make software that is fast in a lot of companies. There are so many times where I have identified a slowdown (often when working on something new, but also when refactoring old code) and opt to improve the speed. Those changes have pretty much always been rejected because they "don't fit in" or "aren't pretty". Things like iterating with loops and exiting early... yes really.
Until we value good software over more software, I don't think much can change.
It really matters how you sell these types of changes.
If you can tie it to some user experience metric, like "this cuts 700ms from our response times", then it's generally easier to get buy in for ugly optimizations.
Microbenchmarks are a bare minimum, but usually insufficient since optimizing something that isn't a big bottleneck usually has imperceptive performance implications.
I tend to agree, but I also think that a good core architecture (ideally one that somewhat discourages inefficient designs), a fast initial baseline developed by a small team, and continuous performance testing with fairly strict reversion on regressions can result in fast systems. LLVM is not an ideal example of this, as the baseline when compile-time tracking started was very bad, but it certainly helped a lot to prevent regressions and force people to find better designs.
What if we took our unscientific opinions and mathematised them?
Some people have said that now with LLMs you can make software that is faster and less buggy. @typesanitizer says no you can’t. No-one really has numbers or significant examples.
It’s clear that llm-driven agents can do some tasks with minimal supervision and some software delivery is faster. Clearly the people who bother software engineers about speed of delivery (and I’d argue it’s mostly ourselves) will still be happier if a project is delivered faster than it was before; and some of the time not spent on delivering the bare minimum can be spent on making it be better than it otherwise would be. If that newly spare time is enough to make a significant difference to the bugs or speed dimension then engineers can choose to have less bugs or more speed. If not then they can’t.
Not all projects, products, organizations, engineers, and markets are the same. So, different people will likely have different experiences of this. Variance will likely increase until the relevant tools become more widely adopted due to some form of it becoming really really easy to adopt them.
Just to clarify my position, I definitely think you can make software faster and less buggy with LLMs.
My point is that can != will. To simplify, (1) the short-term cost of producing slower, buggier software has likely gone down further compared to the cost of producing faster, robust software and (2) costs of producing software is only one out of many possible inputs into decision-making.
I don't think it's possible to mathematise software engineering outcomes to any real extent, at least, not presently.
The reason I think this is that there is due to economics. Economists continually make incorrect predictions about systems of significant scale and enormous impact, with similar seeming human-level complexities and definitional challenges, despite having a much more rigorously developed scientific culture. I don't see why software could be measured and understood any better, given the undeveloped scientific tradition we have as a field.
Performance is not always part of the product
In fact it almost never is, because neither the people making the buying decision nor the product managers use it.
You’re assuming the user is not the one making the buying decision. That's often true of enterprise software, but not of consumer apps or operating systems. I can think of several times I’ve switched from one app to a competitor to get better performance, like Photoshop Elements to Acorn or Symantec C++ to Metrowerks or Netscape 4 to MSIE.
Right that happens but the user writing the cheque on the basis of performance is the least common case. Which is why I characterize it as almost never.
The other point that’s brought up is that supposedly people don’t write JITs (in the context of databases) because that’s too difficult, but LLMs make that accessible.
It's not JIT compilation itself that's difficult (simply generating C/C++ is easy enough to boost large queries), it's the lack of knowledge of the engineers (who would have guessed that database experts are often no compiler experts?) that makes maintenance a risk and the difficulty of integrating query compilation in an often existing database that clearly isn't designed with that in mind. HyPer was centered around LLVM from the start, but later on they encountered some architectural problems that were addressed with the rewrite in the form of Umbra by making some different design decisions from the start. Rewriting a complex production system is realistically a no-go in a serious company.
IME query compilation is largely not adopted for the (fear of) complexity and the maintenance risk (what happens if something goes wrong in prod?). LLMs may or may not help with the fear, but certainly don't help with the maintenance risk if none of the engineers has a clue of what's going on.
On the point about other people not writing JITs, there are at least a few database engines which implement JITs.
The list misses the biggest compiling engines: Salesforce Hyper, SAP HANA, SQL Server (Hekaton), ...