Why do software developers love complexity?
18 points by MatheusRich
18 points by MatheusRich
Simplicity is not actually possible, especially in the face of certain constraints. Performance being the prime example.
Simplicity is a false idol. You just have to write the logic that’s necessary and try to have it not collapse under its own weight as much as possible.
The more common thing I see is that bad performance is caused by complexity, and in particular layers of dependencies
It’s less common that I see you need more complex code to make something fast. It’s true the optimal algorithms / data structures take more code than the simple ones, but that’s usually +200 lines
Whereas the complexity I see is more like +2000 or +20,000 lines
I respectfully disagree. All else being equal, a simple solution is better than a complex solution and you should strive for simplicity without taking shortcuts. A good example would be the raft vs paxos consensus algorithms; they solve the same problem, but one is way simpler than the other.
The problem is of course that there are many different axis of simplicity-complexity in software. For example I often see people make bad api’s because they push complexity out of their solution and on to their clients. Favoring local simplicity over system simplicity like this is in my opinion a very bad choice, but unfortunately very common.
The reality is that designing software solutions is in fact an optimization problem across all the axis of simplicity-complexity, in addition to multiple other factors like performance, development time, future proofing etc. There is a lot of uncertainty in designing these solutions, so you sometimes need to make bets. Failed bets can lead to adding complexity to solve problems that never becomes necessary, or having to rewrite parts of the solution because you underestimated the future requirements. Sometimes complexity caused by wildly wrong assumptions, such as “we need a big data pipeline to process a couple thousand rows per day”, or “we need redundant, multi-cloud k8s clusters to handle 1000 daily users”.
I think the question is, what logic is necessary? If that is ill defined or assumed it must be a certain way, it can easily spiral out of control and collapse under its own weight, as you say. It takes a lot of iteration and refinement of consistently asking “is this necessary?” Far too often the first pass is considered the necessary pass, and it rarely is - in my experience.
The author is making the error of subjectivism; I think you are staking out an opposed argument of objectivism. All else being equal I think your position is more correct, but equally one-sided. We must consider objective factors and subjective factors together, how they relate to one another, and how this relation changes over time.
For instance: simple code that receives simple updates over time may experience a qualitative leap to become complex code, even if the programmer is attempting simplicity the entire time. Quantitative addition of simple code creates qualitative leaps in complexity. The opposing force to this is to refactor; by qualitatively changing the architecture of the code, we can minimize the cognitive cost of additive changes.
What I call the actually existing complexity, the real-world behavior we need to model and account for, constitutes the floor of how simple we can make it.
At $dayjob, it’s simple: Engineers loved complexity because it was expected and rewarded. The career skill matrix had dozens of positive mentions regarding building and handling “complex” software and “complexity”, especially in the Senior+ levels, and zero mentions of simplicity. This led to absurd and ridiculously overengineered solutions by engineers looking to fill their promotion dossier.
I’m tired of people lazily blaming software complexity on things like “marketing” and “developer boredom” as of it’s 100% personal preference. The biggest reason software is complex is because it’s solving complex problems. Oh you’re just making a simple CRUD app? Compliance needs you to store a complete record of every change in the database, but also permanently delete the whole record on a GDPR request. Bam, complexity.
It’s telling that his example of a “real complex” thing is a stone pyramid. The high rise I live in is unimaginably more complex than a pyramid. That’s because it houses a thousand living people in half an acre of land. The great pyramid is simpler, but houses only one dead person in 13 acres. Complexity serves a purpose!
The biggest reason software is complex is because it’s solving complex problems
citation needed :-)
I don’t think that’s true … I’d offer these as some alternative things to think about:
I do think the concept of essential vs inherent complexity is useful though. Sometimes you really didn’t need those five layers of abstraction to solve the complex problem. Sometimes people do things because they heard it was a good practice not because the problem calls for it, and that is worth pushing back against.
I don’t think “essential vs accidental complexity”, at least as it was originally proposed, is a great analogy here. No Silver Bullet gives three examples of accidental complexity:
The first two are things solved by adding more layers of abstraction. Arguably, React reduces the accidental complexity of having to do everything in JavaScript! Brooks in fact says that “Buy versus build” is one of the best ways of reducing essential complexity. I think using lots of third party libraries counts here, too.
(To be fair I don’t think I agree fully with this definition. One of the examples of essential complexity he gives is “you have to interface with a different software that uses a different interface”. He calls this “arbitrary complexity” but puts it in the essential complexity category. It sounds like there are actually three kinds of complexity: essential, arbitrary, and accidental.)
The biggest reason software is complex is because it’s solving complex problems.
This should be the main motivation for software complexity, but at the macro scale, it rarely is. If your experience has been different, then you’ve been extraordinarily lucky, and that experience has been not at all representational, I think.
I don’t think this is about complexity, it’s about complication
Complexity happens because problems are hard and multifaceted, complication happens when the solution to one problem starts to accrete solutions to other problems.
That’s a good distinction and I generally see it referred to as accidental vs essential/inherent complexity.
I like complication as a word, and French has “complexe” vs “compliqué” (complicated) which is the same idea but colloquially are conflated.
The article manages to mention the existence of complex evolving requirements … but then kind of ignores that this has even been mentioned?
The additional detail that not changing existing workflows while adding new features has value (and this also leads to complexity) is fully skipped, though.
Of course, some kinds of complexity hurts the stability and often even leads to externally-forced reprioritisation between requirements — but I don’t believe that the distinction can be discussed without recognising that some of complexity is good.
Time to self flagellate over the sin of stacked software
I’m reminded of this study showing people inherently prefer additive solutions even when they are suboptimal https://www.weforum.org/stories/2021/04/brains-prefer-adding-sustainability/
The simple alternative is just around the corner: sprinkle vanilla JavaScript where it’s needed and don’t build your identity around a framework.
I am a certified React hater, but people use React for a reason! The reason is that React shifts complexity out of your system and into theirs. Yes, for some cases React also adds additional complexity that isn’t needed, but overall, if you have a medium sized site you can win the tradeoff by making Facebook think about these problems instead of doing it yourself. A site of even modest complexity will need to have conventions to make the JavaScript manageable, and the conventions eventually approximate a framework.
I think simplicity is often “just” a perspective. I mean, what’s got the fewest moving parts: a bicycle, a tricycle, or a unicycle? The unicycle! So simple, so elegant. But it’s hard as hell to use it correctly. Triple the number of wheels, though, and even a toddler can’t go wrong. I’ve never ridden a Segway but they sure look simple to ride. Complicated to build, sure, but simple to ride.
Maybe that’s a trite example. But I do feel some things called simple are easier to misuse. There are times I wish I had grabbed scissors instead of a knife. You could say scissors handle the edge cases a little more safely. We don’t all have the faultless bladework of a fishmonger or a sushi chef.
IME the problem has very little to do with people liking complexity. It’s just that it’s much easier to get over the line by adding complexity. I think the way people are taught software engineering sometimes makes things worse, but even then, it’s not as though anyone has a binary choice between making things needlessly complex or not. Making things simpler is hard and they haven’t practised it, and in a typical business setting it also takes a huge amount of time nobody is going to give you.
Once again strangle loop conferences have a talk which perfectly answers this I feel, by rich hickey - simple made easy (https://www.youtube.com/watch?v=SxdOUGdseq4)
Love complexity?
Of course now we have AI that only knows the same.
What are the other ways you allude to?
Well what movements occurred in order to solve the problems inherent in functional decomposition? What alternatives are there to just modeling the transformation of data? You’re a student from your nick? What do your teachers / studies tell you?
Are you just talking about the design methodologies associated with the major programming paradigms - object oriented, (not functional), concurrency oriented, relational, logic? I’d love to know what you think are the major methods in this space.
I think “simple” gets a bad rap. Maybe because it has a bunch of different meanings, some with bad connotations. I’ve always thought of simplicity being the absolute core essence of a particular thing.
There’s a Picasso piece Steve Jobs allegedly used to use to teach his viewpoint on product design that’s also been a staple of my personal engineering philosophy: Le Taureau. It has several drawings of a bull, each progressively removing detail. The very last one is very minimal figure that you could still look at it and go
yeah that’s a bull.
That very last drawing is what we want. Simple is easy to reason about, maintain, extend, remove when necessary, and probably most importantly: easy to sell. It’s not easy to get to, either. It takes a lot of careful thought to put the right things in the right place to maintain the essence of whatever it is you are trying to make.
I’m a lead and am pretty adamant on being mindful of simplicity, but have definitely worked in environments where it wasn’t appreciated as much.
Simple is easy to reason about, maintain, extend, remove when necessary, and probably most importantly: easy to sell.
In a world where people who don’t actually end up using the product do the shopping, it might not be easy to sell. If the competition has more features, it often wins. And even if the end-user is doing the shopping, they might go with the more complex and featureful product because “what if we need it later”.
If the competition has more features, it often wins.
If and only if the competition has features that the customer actually wants and needs. In my experience, engineers have a huge disconnect between what they want to build and what the customers actually want. There’s actually a hot reddit post about exactly that right now.
I’ve had the exact opposite experience, where customers want something that’s reasonable to ask for but very difficult to implement and engineers wind up not being able to push back enough to avoid it.
I actually find quite the opposite. In my experience on operations teams supporting developers we joke that developers are like water, taking the path of least resistance. Everyone gets burned in their career at some point adding too much complexity to a project and then paying the price forever, so there is an almost pathological desire to avoid that complexity when they can see it coming.
The issue is that the problems developers are asked to solve are rarely well-defined, which is why I don’t have a lot of fear about LLMs. If management was capable of explaining what they wanted to do and responding to requests for decisions inside of like one markdown file that lived over the course of a project, then an LLM might be a real risk to me. But because I’ve never worked at a place that actually gathered all the requirements before building and held to those requirements in the face of business/sales pressure, instead you end up with a constantly shifting set of priorities that inevitably ends in complexity.