Perl's decline was cultural
81 points by LeahNeukirchen
81 points by LeahNeukirchen
This is a wonderful retrospective on 20 years of programming language mindshare and the culture of programming that is about more than just Perl. I’d particularly like to highlight this aside:
The truth of the matter is, that the majority of programming work is broadly similar, and the specific implementation details of the language you use don't matter all that much, if it's a good fit for the circumstances.
This is the sort of thing that I think most programmers know in their bones after a few years of real work and after they’ve learned their 5th or 6th language to elementary competence. That threads on languages continue to be so overheated can’t be entirely blamed on novices and converts hyped up on the thing they learned most recently.
That may be kinda true for me now, when I am more experienced
But honestly I first touched Perl in ~2000 and Python in 2003, and Python was way better for me ... it's still the language I use most often
But in practice Python had a nice syntax for me in 2003. (I had used C and Java before, and Lisp for college) And it turns out it can do basically anything else, and I just kept using it
And I don't think I'm alone -- probably hundreds of thousands of people experienced Perl before Python, but settled on Python
Nothing about using Perl in 2000 made me say, "I want to keep using this", but that's what happened with Python
I wanted to learn a scripting language in 1998; I was familiar with Pascal, C, C++. So I already had Perl on my home linux installation, man page tutorials and such. I didn't have internet at home yet so the man pages mattered. I thought built in hashmaps and growable vectors were amazing. But I couldn't wrap my head around it easily. I knew about Python as an option because I read an article about scripting languages in a magazine. One, two weeks later, I went online, printed out the Python tutorial, read it at night, and the next day I could write Python. Not well, but good enough. For the next few decades it was my primary programming language.
Language does matter. If it didn't, I would have continued with Perl - it was right there, installed on my system with nice offline man pages.
FWIW I think perl still shines at what it’s targeted at: being a better shell script with built in text handling.
Python shines much more at being an easy to read and learn programming language, but I think it’s moderately uncomfortable for that “better bash” role.
YSH is for Python and JS users who avoid shell :-)
I do think there are a large number of people who have never seen or touched Perl, but know Python/JS, and touch shell
That includes me -- the last Perl I wrote was probably 2000, and I didn't really learn shell until 2007 or later, after I had written a ton of Python.
Shell is the #8 or #9 programming language according to multiple sources (Github and IEEE I think), and it's growing fast. A large part of that is due to the cloud and VMs and containers (which barely existed in 2007, which is funny to me)
(Idiomatic YSH is very different than Perl too -- it's less "string-ish", despite being a shell)
In a similar vein of "better shell script with builtin text handling" is Fish's scripting language. I haven't touched Oils -- probably should, but only so much time in a day -- so I don't know how Fish compares to it, but in relation to bash I consider Fish's language to be somewhat more of a shell scripting language in all the "shell scripting" design aspects.
The two major changes / improvements here are first-class arrays (everything is an array) and more uniform notation (almost everything is a function call). A lot of special syntax in Bash (variable definitions, comparisons, etc) are normal function syntax in Fish. And since the ergonomics of strings and the like are a lot better when you treat everything as arrays, string operations are also provided through a builtin string command that functions just as any other function or command.
...I bring this up mostly for the "better text handling" point, because once you get away from POSIX, there's a lot of good options for a better shell script with built in text handling, IMO. Which is excellent -- I do love a diverse ecosystem of programming (here, shell scripting) languages. (and yeah python doesn't cut it -- not self-extensible in the manner shell languages are)
Fish seems to be stronger as an interactive shell
There are zillions of people who use fish as their default shell, yet I've seen almost zero fish scripts in the wild
There aren't any distros that use fish scripts rather than shell scripts (e.g. Alpine Arch Fedora ...)
I'd like to see any 500+ line fish scripts people have written
Python is a very imperfect "better bash", but good enough for me that it's not worthwhile relearning the perl I've forgotten over the past 25 years since I used it professionally.
FWIW it's a good enough text & data processing language. In my experience its often significantly faster than equivalent Rust code for wrangling large JSON docs.
With Python, what I really struggle with is where it seems to go out of its way to be difficult for use-cases where shell, perl (and ruby) are strong. I’m thinking how awkward sub processes can be compared to backticks, no regular expression operator (and with regular expressions in strings we bump into how many lexical string types in python? 5 now?)
Even lambdas are syntactically clunky and bolted-on.
The truth of the matter is, that the majority of programming work is broadly similar, and the specific implementation details of the language you use don't matter all that much, if it's a good fit for the circumstances.
I understand this in the micro in some sense (though even in the micro how you grasp at problems is pretty language dependent) but I do think that the language choice on a larger project will greatly influence how the superstructure works.
No amount of library API design work gives your Python code straightforward monadic transformations that work well with the rest of the language. No amount of library API design work makes your Haskell code have straightforward type inference for anonymous records. No amount of library API design work makes your JS codebase get decent operator overloading. No amount of library API design work gives your Python code staticly checked ownership invariants.
And while the difference between add(a, b) and a + b is very slight... if you're working on a bunch of data analytics code all the time after a while you start missing that kind of stuff!
So you can solve all your problems in all your languages, but at one point you'll end up wishing for things to make your job easier, more straightforward, safer, etc. And your code will end up shaped very differently depending on the language, after acreting so much stuff that works with what the language does and doesn't give you.
Having said all that... at the end of the day you can still just suffer through all the issues. I don't think it doesn't matter, but it also is unlikely to be a massive blocker. The quality of problems you need to solve will be different is all.
I agree with everything you say here and your examples are very good ones. I wouldn't want to say that all languages are equivalent because you can solve any business problem in all of them and I don't think the author is saying so either. I think the real point is that bit about "if it's a good fit for the circumstances", because circumstances (taste, history, trends, pique, corporate machination, group dynamics) wind up dictating what language code is written in more than some hypothesized fully rational decision made by Homo economicus. And I think it is also a good and subtle corrective nudge against the cliché of "using the right tool for the job". It's more fun to use the wrong tool for the job anyway.
That's totally legit, we rarely are presented with a scenario where we have every option (PHP being such a good example here: PHP usage is defined almost entirely by being able to use it on shared hosting). That makes a lot of sense
I would disagree although maybe you can just consider Perl a "bad fit".
I was a Perl user in the early 90s and moved off of Perl about 1995 onto Python (just barely at 1.0). Perl had some ferociously nasty bugs that I always somehow managed to hit and Perl 5 didn't manage to fix (the one that sticks in my brain to this day was that regexes of exactly 512 characters would fail in mysterious ways--511 and 513 characters were fine--yeah debugging that one was a NIGHTMARE). Combine this with the fact that maintaining Perl scripts was simply too painful once you tried to deploy it for any task more complicated than basic system administration stuff, and I was looking to move pretty much anywhere else. Python dropped onto that with a syntax that other people used to C and FORTRAN could read and enforced whitespace indentation in an era where autoformatting wasn't commonplace.
My biggest example of this was always taking two printouts of code doing the same task to VLSI job interviews--one Perl and one Python. The goal with the Perl one was to prove I "knew Perl" and to force the unfortunate interviewer into MY subset of Perl instead of theirs. The goal of the Python one was to talk about the VLSI task and the solution. The goal of both together was to fiercely intimidate any interviewer who wasn't very good at programming.
The upshot of the Perl one was always "Wow, I don't know Perl as well as I thought I did." The upshot of the Python one was always "What a neat way to solve that task"--even if they didn't know anything about Python.
Basically every job interview re-validated my choice to leave Perl behind for Python.
Reminds me of what Rob Pike said about the Unix philosophy of small tools written in C glued together by shell scripts and pipelines:
"Those days are dead and gone and the eulogy was delivered by Perl."
Perl showed that integrating features from standard tools like sed, awk, and Bourne shell into a single "real" programming language could be far more powerful than using those tools in isolation.
I wonder what Rob Pike would say about this quote from the last minute of this video:
If we can get those developers to move from coding the volume of this problem space to coding the perimeter, like they did with UNIX, then they'd be exponentially more efficient and that would allow Google to much better achieve its goals.
we have Perl 5 already here for doing things, Perl 6 is going to innovate on everything, and spend it's time getting there, designing up-front.
I've seen this a lot with (in particular) JS language libraries. "The next major version solves this!" "Oh cool is it out?" "It was supposed to be out 2 years ago but well"
A very dangerous trap to fall into, IMO. But I get why
The lack of structured clear elements like classes and exceptions meant that on a language level, early Perl 5 struggled to scale well.
So things like MOOSE came around and tried to bake a frosting on, but it wasn't good enough vs something like Python or Ruby which had the capability built in.
For my part, I always found Perls community more welcoming and honest than Pythons.
Well, I'm sad that python won the war, I just think it had better marketing. If anyone has ever used Perl monks, they are the kindest community I've ever worked in. The python. People were kind of mean whenever I asked questions. Questions. Heaven forbid you used two spaces instead of four. So I would disagree with this article that it's not a siege mentality. If anything. Python was far more unfriendly than my experiences in perl
Perl is one of the languages I never actually learned, but my memory from back in 2000-ish/2001-sh, right or wrong, was that it was great for (aside from CGI): 1) text processing and in particular leveraging regular expressions 2) being a better bash when things get complex.
A language not mentioned in the article with the others that emerged/changed around that time... I remember at my work in 2000-ish/2001-ish, a .NET beta DVD arrived with C# on it as part of our MSDN subscription. I installed it and I remember C# having regex literals. I remember it was super easy for me to quickly learn C# and build basic web scrapers with regular expressions at my work. Maybe it didn't play much of a role compared to the languages mentioned in the blog post, no clue. But it comes to mind as something that would have reduced the bang/buck of Perl for me at the time for the work I was doing: the new expectation that general purpose languages should up their text-processing game and close the gap.
Re: the second point, I was able to get done the few admin things that I needed to with bash, and while I'm sure this would have been more pleasant if I had learned Perl, bash was good enough for my lightweight sysadmin needs at the time.
This would have left Perl as something I could learn for fun and for personal use, which I love doing, but it just never wound up being one of the languages I learned for fun out of sheer coincidence.
My only "experience" with Perl was back around 2012, so this was already pretty "late" in the story, when I was offered to take up maintenance of a particular little board game helper script written in Perl. I was a mathematics student with no real experience in programming or its culture, but somehow both the way the offer was made and the zeitgeist around Perl that I had picked up at the university made it clear to me that I should not accept, and that this was precisely because Perl would not be kind on me.
I have no idea where that idea had come from, but I think that back in 2012 the writing was already well on the wall. Perl may not disappear, but it did decline and the reason to me does seem like it indeed was cultural.
When I encountered perl, it was an entire application, not shell scripts. Not fun.
By the time I started working on servers, Python was already installed by default.
TLDR
reactionary community with conservative values
The article then goes on to mention the “conservative” values, such as "meritocracy", or “there's more than one way to do it” (TIMTOWTDI), because “elevating extreme flexibility like this is paradoxically also an engine of conservatism". Article also mentions that Perl comes from "BOFH culture", without much evidence, since after all, that was just an Internet meme at the time.
I never liked Perl, the language, and I ditched it for Python and Ruby as fast as I could, but I've been to 2 Perl conferences, and the people I've met were talented indeed, yet kind, and welcoming. At that time at least, Perl had an actual community, which was rare, and still is.
I've also read the HN comments. So, is anyone not going to comment on what steaming pile of garbage this article is?
So, is anyone not going to comment on what steaming pile of garbage this article is?
imho, when something very clearly presents itself as an opinion piece, it's in pretty poor form to do so, even if you think it. You're free to disagree with the opinions and interpretation, to offer a different one, or to point out any factual inaccuracy, but what's the value contributed by just calling someone's opinions garbage?
The value is in standing up for the many contributors that have, over the years, kept public projects in good shape and kept the Internet running, most of it in their free time, without much compensation.
It's baffling to me that I even have to spell it out.
The primary problem is that is not just an opinion piece. It’s making statements. Those statements are not even wrong. They’re basically nonsensical.
I don’t necessarily agree with the article but I don’t think your assessment is fair.
There certainly was a BOFH/Luser culture around the time of Perl. I don’t personally correlate Perl with BOFH but I can see how others might. So I don’t think it’s nonsensical, but I do think it’s incorrect.
FWIW, my opinion is that culture was at most a footnote. Instead, I think Perl was a local maxima. It solved many problems much better than what came before it, but much worse than what was to come. Everything else about Perl’s (relative) demise is just a footnote.
It's fair because I was an active member of perl5-porters, I've contributed code to the inside of Perl, I spoke at the first ever Perl Conference, I spent about a decade of my life programming in Perl, I was for a long time the only quoted person besides Larry Wall in the official Perl book, and I was there for the entire transition of the culture leading up to Perl 6.
The idea that "there was a culture of type X, and so therefore it's ok to say that the culture of Perl was of type X" is just vibe thinking, on the order of 'there is a cat, therefore all people are cats.' It's literally nonsensical and it's literally stupid to put that view out there. There's really no two ways about it.
You obviously had the 30000 foot view of what was happening. Why do you think Perl declined?
Perl declined because Larry got distracted with health issues and other concerns, and his lieutenants were heavily incentivized not to change anything because they were paid by O'Reilly to be book authors, and books don't like the language to change out from underneath them.
Around the same time, verbose and relatively powerless languages started to make ideological inroads, along with a booming culture of stack-overflow-copypasta programming which lent itself well to lower power languages. Perl was grown to be a power tool of great expressive power, and that put a lot of intellectual burden on newcomers. The language didn't stop people from reaching for the power too quickly, and so it gained a reputation as inscrutable and write-only. I probably did not help that by running the Obfuscated Perl Contest for several years; it gave detractors something to point to, as an example of how hard to understand Perl is.
In this thread, and in the HN thread, you can see this in effect; people complaining about the syntax and the sigils and so on because they're trying to get a low-power-language superficial understanding of a high-power language. Same as with Haskell, J, K, etc. Culturally there's lots of room for low power languages and they can be incredibly helpful and effective in certain corporate situations (e.g. Go), but it's annoying to see people whining that a high power language which requires study and engagement is bad because they can't superficially read it on the basis of their understanding of C, Java, Python, or Go.
There was not a weird mean defensive BOFH culture around Perl; it was one of the most welcoming and exploratorily liberal communities of its time, and that is saying a lot, because it was in the early days of the web based internet.
I think they mean lowercase-c conservative, ie. "resistance to change", not the political ideology.
I felt like this was less a statement about the humans involved and more one of the implicit tendencies of the group and the way the language presents itself. Where the necessary hoops and resulting in-jokes and habits created an online community for Perl that kind of got left behind as the Internet changed.
That matches my feeling on the language. Sort of culturally a relic from a different age, with different priorities and sensibilities. I'm sure they're all very nice though.
The downfall of Perl was that it became codified into O’Reilly books. The authors of those books were heavily incentivized to keep everything the same. This struggled hard against the natural essence of Perl, which was to be experimental, poetic, and fast moving. In the end, capitalism turns everything into bullshit.
Every other author was happy to write a new edition for each new version of MS Word, I don't believe this.
What you believe is not important - it was how it went down. I cannot begin to tell you how much history is not affected by your vibes.
I’m sorry but this also seems really far fetched to me, but I’m trying to remain open to the idea.
What language features were being suppressed? 5.8 alone introduced at least half a dozen significant new language features. Smart match made it in even though it should have been obvious that there would be no way to make it work without an actual type system in place.
Was this actively discussed—and therefore recorded—on p5p? Or was this all happening in the back rooms of Perl conferences?
Edit: looks like I had 5.8 mixed up with 5.10, a version that did take a long time to appear and which might support your thesis.
The thing I find difficult to understand about this take, is it essentially supports the conceit made in the blog post it takes issue with, rather than refutes it. They are literally describing a class struggle caused by an elite faction who were incentivized to maintain the socio-economic status quo vs an experimental creative fringe, leading to a paralysis, later a schism.
Supporting your interpretation is their other comment literally calling people stupid for saying Perl had any cultural issues.
Do you mean they were incentivized to not introduce breaking changes? That’s surely a normal part of not driving users away?
Yes. At the time, O'Reilly's economics were set up to sell the same book over and over. That's orthogonal to driving users away or not; some of the changes might have improved the onramp for new users, for example.
you can characterize HN comments in many ways but one universal constant is that it is always popular to dunk on people's lived experiences, let's not do that here
Have you read the article?
Because the author is not talking about his own “lived experience” for what he's saying. The only quote I can see:
"I did occasionally bounce off a couple of abrasive characters in the community, but mostly this just kept me loosely coupled, I enjoyed how the language solved the problems I needed solving quickly, I enjoyed the flexibility, and I also enjoyed the way that it made me feel smart, and en-route to my wizard’s robes and hat, when i used it to solve harder problems in creative ways, or designed ways around bugs and gremlins"
That's it. Maybe the author's lived experience is richer than this, but it's not in the article.
There are several other mentions, despite the author specifically stating they're trying to avoid wallowing in that aspect:
Perl had an, at best grudging, tolerance for 'difficult genius' types, alongside this baseline culture. Unfortunately, this kind of toxic personality tends to thrive in the type of culture I've described, and they do set to help the tone. I'm not here to call out people specifically, because I'm trying to make a point rather than feed a culture war, or dig up gossip, but there were several significant examples, you can probably find lore if you like. I think the kindest way I can describe the compounding effect of this is that there was a strong cultural norm along the lines of "It's OK to be rude, as long as it's for a good cause".
I remember this tension as always being tangibly there. Perl IRC and mailing lists were quite cliquey and full of venerated experts and in-jokes, rough on naivety, keen on robust, verbose debate, and a little suspicious of newcomers. And very cult-like.
I don't think the article is a steaming heap; I found it very interesting.
Now, I disagree with its reasoning. I didn't see a BOFH mentality in the Perl culture (and I've been programming Perl since 4.0 something or other) any more than in any other language of the time. Every language community had its rude uncles.
My personal thought is that programming ergonomics changed, and the tendency to write terse idiomatic code (which was a valuable skill in the days of more limited computers - I still write BASIC on the C64 this way) became seen as unnecessary and counterproductive for team programming. Perl in some conscious, deliberate ways is emblematic of this paradigm, and Python the opposite. Pendulums swing.
Mix in PHP eating Perl's lunch on the hosting side and Perl 6 splitting the community, and there you have it. My opinion is worth exactly as much as his is. :)
My comment is now flagged “4 unkind”. Which is fine, I really don't care, but I'd like to see that same energy go towards the article itself, and I'm not seeing it. Why is the article itself not labeled unkind?
To call a spade a spade, people are either thinking about their own grievances with Perl, ignoring TFA's actual content (reading is hard), or turning off their critical thinking whenever someone critiques “conservative values”, because it's on their side of the political spectrum.
The problem is you're seeing the article as "political" when politics isn't mentioned at all. I'd guess you've misunderstood the meaning of "conservative" in this context? The article is using it in the sense of "resistant to change".
To be fair I suppose, I did also think we were talking political spectrum when I first hit "conservative values" but after reading a bit further I realized we were talking about resistance to change as you mentioned. By the time I hit "liberal" in the article, it was very clear to me we were referring to openness to change, not political spectrum positioning. But I suppose some valid feedback to the author could be to have put (as in resistance to change, not political spectrum) after the very first mention of "conservative" to improve communication/reduce ambiguity.
(author) might it be a British idiomatic thing? Because our conservative political party is literally called "Conservatives", the word conservative uncapitalized tends to mean non-political use. Anyway, I think it's clear enough if you read the whole thing. I may add another footnote tomorrow when I'm at the computer, but they are annoying to do in my crappy homebrew blog software
Yeah, totes could be it. I'm in the US so maybe that's why I misunderstood for a split second. It's definitely clear to me the meaning reading the whole thing or even just reading a small part of it. Regardless, miscommunication seems to have occurred, though miscommunication is kind of inevitable if you're going to take the risk of communicating! (See: Wiio's Laws :P) So it doesn't necessarily mean you have to change it, per se.
I dig the color palette of your blog, btw!
My take: I worked on perl circa 2008 as part of the "interchange" framework. A framework that at one point was the Red Hat blessed ecommerce framework for perl. It powered some websites (backcountry.com, steapandcheap.com ) that made a few hundred million dollars/year in revenue.
In my opinion, basically ruby became the successor of perl 5. It added a friendlier syntax for objects while keeping many of the affordances of perl in text processing. Some examples of "perl in ruby".
%w{a b c} // it's qw
"string"=~/regex/ // good regex syntax
`ls` // easy integration with system commands.