We Are Not Special (2021)
27 points by WilhelmVonWeiner
27 points by WilhelmVonWeiner
I agree with a lot of the article but I also do think it is being overly optimistic about rigor in software engineering. Specifically this line
Plenty would kill to get the same kind of automated testing we treat as a given.
I have worked in a multitude of organizations and automated testing is still in this day and age not a given. Specifically robust automated tests at various levels and not part of the process. Software engineers all too often see themselves as the only checkpoint that is really needed and organizations try to eliminate parts of the processes they see as roadblocks.
I have written a a comment about this before so rather than repeating myself entirely I am going to include it below. In addition to that we as a software industry seem to collective suffer from short term memory loss where we are simply refusing to actually codify learned best practices and insist on reinventing the wheel every few years. I can already hear people scuff and point out that a lot of things have standardized over time and a lot of practices are shared between companies. But in my experience what has been standardized is mostly a shared vocabulary on loosely defined things that have wildly different meanings between companies and testing. Terminology like agile should be familiar to anyone. But also when we are talking about "testing" in my experience people think about wildly different highly specific things, not the holistic testing approach as a whole (see my comment below). Most developers will think about unit tests, some might think about a slightly expanded scope with component/integration unit tests. In some companies that is all the testing that will be done before things are yeeted to production even.
Anyway, no we certainly aren't special. But, we aren't the same either even though we probably should be a lot more similar in my view.
It also annoyed me but for a different reason as you are saying. The point of doing a code review, in a healthy organization, should be that it seen as one of many processes that together are responsible for good quality process. Not just a singular gateway that is responsible for something going to production. In fact, in many cases it shouldn't even be the last gateway depending on the complexity of the landscape, integration with other applications, etc. But, unfortunately for many teams and companies the QA process after code is merged is something that might as well not exist anymore.
Anyway, I am a firm believer of everyone being responsible who is involved in the process. However not to diffuse responsibility but because quality should be a integral part of the process before code is even written. Basic boring stuff like three amigo sessions (or whatever method used to get various disciplines involved in talking about spec and requirements), test driven development methods, basic static code analysis tooling integrated in IDEs and various checkpoints, QA/Test automation expertise other than developers themselves integrated in teams and much more.
I know that for many people this is not a reality they ever encounter. But if we are making the comparison to civil engineering. The developer shouldn't be both the engineer and construction worker for the bridge in many cases. Even if they are, in civil engineering there also never is one person who is responsible as the practices I mentioned are very much part of the process there as well. An engineer needs to document everything, make all the necessary calculations for load bearing, have others re-check those calculations, then often get approvals from relevant government bodies who will double check various things. Then during construction there are also many more points where things are audited, checked, approved, etc. Even for simpler construction like that of a single family house this is still the case. Plans need to be drawn up, someone needs to sign of based on calculations, permits need to be given based on those plans and at various points in the process inspections need to be done.
Yes, there are people who have more responsibility than other but overall there never is a single person who has the only responsibility. In fact whenever there is a engineering disaster you often see that it has been a failure in the overall process where at multiple points things should have been stopped but were not.
We like to compare ourselves in software development with civil engineering. But if we do that we need to realize that our processes very often are just not on the same level at all. At least not as far as constructing bridges goes. In the best case scenario we often find ourselves at the level of housing developers who build the cheapest possible houses and try to work around a lot of the legally requirements.
You are right!
But also I think you do not realise how few testing actually happens in engineering and how painful integration hell is.
Oh I am well aware, see my quoted comment I included near the bottom for example. Tests are just a small part of the overall quality process anyway. More importantly, when I hear software engineers compare themselves to civil engineering and the sort of projects where quality is interwoven in the overall process (where tests themselves are only a small part of it).
I really recommend reading the three parts, they're great! They have also been discussed here in the past, if you want to read the comments:
https://lobste.rs/s/fv8swh/crossover_project (project announcement)
https://lobste.rs/s/lmvroa/are_we_really_engineers (an encore)
https://lobste.rs/s/8j8sdc/are_we_really_engineers (another encore)
I believe the link is in there somewhere, but this talk from Glenn Vanderburg is very relevant as well.
This is the nature of our material. And it’s also why other engineering fields are progressively using more software by creating these design tools and simulations, engineers can prototype their ideas with software before implementing them.
That is I think the biggest difference between software development and trad engineering - the degree of friction. In our (software) case, our great advantage is the (relative) cheapness of a change and flexibility which opens up a whole new world of possibilities. The more simulations we can create for trad engineering, the more possibilities of a change more akin to ours they would have (physics & chemistry limits included of course).
I cannot also stop myself from quoting, sorry!
If I give you a sorting function, you can expect it to sort. You do not expect it to sort a given list of (nonpathological) numbers only 95% of the time. That would be ridiculous.
In the context off LLM-generated code, it actually is true; it works - most of the time, but not always.