Hacker News now runs on top of Common Lisp
37 points by giffengrabber
37 points by giffengrabber
Given that the same author of HN and Arc has written how lisp is a secret wepon so good that “blub” language developers don’t even see it… I must be a blub programmer.
HN used to have issues with “unknown or expired link” and couldn’t reliably show page 2, due to cleverly using closures for state management. Until this switch, the performance of Arc was so poor, the site couldn’t show more than a couple hundred comments at once.
I do think Lisp was a secret weapon back in the 90s-00s when pg was actively bucking the fashionable choice (even in the startup landscape) of writing web apps in Java. For context, when this article was written, Blub languages = {C++98, Java SE 1.4}, Rails didn’t exist, and the other options for scripting languages were Perl and Python (and I don’t think their web support was very good at the time). Mainstream programmers had never heard of lambdas, lazy streams, syntax-aware macros, or multiple dispatch.
If a developer has only worked in C++ or Java, even more modern versions, they do tend to have a stunted view of what programming languages are capable of (although the situation is much better now). These folks might be fine programmers and can still be successful. In fact, on large projects with big teams, code that is simple, procedural, and blub-like tends to scale better. But when racing against a clock, cranking out code as fast as possible, and working on a small team (or solo), most developers gravitate towards dynamic and flexible languages. Knowing a clever language can come in handy for prototyping and iterating quickly.
Lisp isn’t nearly as novel now as it once was, but many languages have adopted Lisp-like functional programming features and better REPLs (they still lag Lisp a bit, but the advantages of Lisp are much less striking now). The influence on Ruby especially is undeniable: Rails’ drop-in REPL when a request throws an exception, the pervasiveness of DSLs in many libraries, even naming mutating methods with a ! suffix. The fact that you can work through much of SICP in most modern scripting languages just shows how close they’ve become.
Agreed — early features like first-class functions + closures + GC were a pretty big advantage, and the C++98 / Java 1.4 type systems were brutally inexpressive (and yet not even nullsafe!) and thus pretty slow and painful to work in. IMO modern typed languages like TypeScript are much more productive than dynamically-typed languages once you have more than like… 15 lines of code, but dynamic types were a lot more productive than the common languages of that era at almost any size.
PG credits macros for Viaweb’s execution speed, and while I obviously wasn’t there, I think he was probably wrong about that: I’ve used languages-with-macros plenty, including various Lisp dialects, and while they’re nice I think they’re ultimately not a huge source of leverage compared to everything else.
To be honest I think REPLs are like that too: they’re nice, and sometimes useful, and languages that have them are eager to tout them, but I don’t really find them game-changing. They occupy this weird productivity space in between “insert print statements” and “break out a debugger,” and I think that space isn’t super large. YMMV.
PG credits macros for Viaweb’s execution speed, and while I obviously wasn’t there, I think he was probably wrong about that: I’ve used languages-with-macros plenty, including various Lisp dialects, and while they’re nice I think they’re ultimately not a huge source of leverage compared to everything else.
Yeah, I think you’re right – I went through a smug lisp weenie phase a while back and parroted some of the same arguments, but homoiconic syntax isn’t significantly better and macros usually just save you from typing a few extra characters. re PG’s perf argument: I guess you could get some small improvements by inlining some code blocks/dispatch decisions statically with a macro, but the effect has got to be really small with all the other dynamic language overhead.
Also, hygienic macros are cool, but they’re often abused to create DSLs which usually end up having a cutesy, non-composable syntax with incomprehensible error messages. It turns out implementing a good programming language (even a small domain-specific one) is hard :P
To be honest I think REPLs are like that too: they’re nice, and sometimes useful, and languages that have them are eager to tout them, but I don’t really find them game-changing.
Ehh, I think the prevalence of Jupyter notebooks speaks to the potential of REPLs. I hate the implementation of Jupyter notebooks and think something like Pluto.jl is far superior and should catch on, but the model of interacting with a live system is extremely useful for exploratory programming. REPL tech hasn’t advanced much in recent years, and that’s a shame.
In a similar vein, I’ve been trying to come up with a nice hot-reload setup for my recent gamedev projects (with complied languages like Zig/Odin). Being able to swap code out while maintaining state is a huge boon to productivity. I’m dreaming of a day when I can drop in one line of code into my game’s source to get a visual widget that helps me hot-patch a physics bug without having to restart the game. Hot-reload/REPLs don’t fit every problem, but they can be very comfortable for working with long-running systems. I want more language tooling to develop beyond batch processing :)
Also, hygienic macros are cool, but they’re often abused to create DSLs which usually end up having a cutesy, non-composable syntax with incomprehensible error messages.
While I hate myself for writing this…in the age of LLMs these terse DSLs become even more impractical as we now have a process that can generate the boiler plate for us and if we write clever code with macros the models will not be able to replicate it as proficiently.
If every user of an application needs to write the same code, your API is wrong. If you abstract away the boilerplate and reuse that abstraction, you’ve made the world better. If you create a tool that generates the boilerplate, you’ve made the world more fragile. @david_chisnall
I think the focus on “REPL” as a term leads to misunderstandings of what people actually like about programming in Lisp, tbh. What you want is a system designed for the level of interactivity that a good REPL requires, and to make the parts that make the REPL possible available to the programmer. Like in Common Lisp, you usually aren’t typing stuff into the REPL other than for testing, you write code in files and use a key combo to recompile and load some code. This uses the exact same infrastructure as a REPL, but is more convenient. You have a top level exception handler on every thread that pops up an interactive debugger in your editor, which allows you to run code and make change and tell the debugger to re-try the thing that failed (I made use of this when developing my Discord bot, whenever an event from Discord caused an issue, I could fix it and re-try handling of the event with no downtime). The class system is explicitly designed for interactive development, with considerations for handling classes being redefined.
Compare to Julia. Pluto notebooks are great, but not really suitable for application development. Classes just straight up can’t be redefined, so any changes to your data model requires a restart. And there’s no interactive debugging other than “change the code and let it auto-execute until it stops erroring”.
They occupy this weird productivity space in between “insert print statements” and “break out a debugger,” and I think that space isn’t super large.
In Common Lisp, the REPL and the debugger are the same thing.
I feel like Perl was a pretty big deal in the late 90s? I remember struggling to write CGI perl applications back in the day trying to figure out how to “use the internet”… but that might be more “mid 2000s”.
It is interesting to think about how huge of a dichotomy there is between writing C++98 and one of the scripting languages (I would put lisp in the “scripting language” bucket), just in terms of how little messing around you have to do with datatype declarations and the like in the scripting space.
People can talk about functions being data etc but even just being able to put stuff into containers without fuss feels so powerful. Just that, and even in a world where you don’t have proper closures, and you’re probably going to have a much chiller time scripting up a website
Perl had closures and higher-order functions back when the blub essay was written. I think PG even mentioned that he only really worried about competitors who used Perl?
even naming mutating methods with a ! suffix
This is a misconception.
The bang sign means “the bang version is more dangerous than its non bang counterpart; handle with care”” -Matz
It’s about “danger”, not about mutation.
Oh you’re right, even in the lisp community, it’s used to tag “side-effectful” and unsafe functions, not just mutation. But I’m still pretty sure the convention came from lisp, along with the convention of naming predicates with a “?” suffix.
From Scheme specifically, I think?
Yes. Common Lisp sadly doesn’t do this, rather postpends p for predicates and prepends n for destructive functions (i.e. which don’t just mutate but actively use that memory space for intermediary computations) and worse, inconsistently.
Makes me wonder. Would we dare to claim that the Blub of today is much more sophisticated than the Blub of 2001?
As someone who works quite a bit on a Java codebase from 2001, yes, definitely. Just having variants is a game-changer. Astonishing it took our field so long to realize that being able to return one of several possible things would be useful.
I worked in Java of that era, and C# now. While modern C# is still Blub, it is much, much more sophisticated than early Java.
I said this a few months ago and I’m surprised that in this (short blog post) format it’s gaining such traction. This makes me want to start spraying out trivia propaganda articles!
I think this may have a lot to do with the demise of a certain short-form blogging website. A significant number of people seem to have moved to posting short-form content to their blogs instead.
no assumptions after one short post please ;)
Didn’t Yegge describe Common Lisp as the Howl’s Moving Castle of Lisp? :)
It’s my favourite dialect, I’ve done a very small amount of commercial work in it (and loved the experience), and it has libraries for everything I want to do.
I know PG wants to invent a 100-year language with Arc, but CL has been around since (arguably) 1984. It’s getting towards half way there already …
I’m sure lessons could be learned from this entire journey. I doubt they will be.
I’m sure lessons could be learned from this entire journey. I doubt they will be.
I keep this quip from Keith Braithwaite around for such occasions:
It's a curious thing about our industry: not only do we not learn from
our mistakes, we also don't learn from our successes.
I’m not sure what (seemingly negative based on the tone of your comment) lessons would be learned from this switch, as notably HN still runs on Arc, they just ported the Arc backend from Racket to Common Lisp.
If anything, the lesson seems to be writing your own lisp dialect gives you greater flexibility to try out new language backends.
If anything, the lesson seems to be writing your own lisp dialect gives you greater flexibility to try out new language backends.
Definitely a high priority when writing forum software. /s
Eh, given that HN is meant to be running for a long time, that flexibilty strikes me as potentially quite useful.
Not really, they’ve rewritten their DSL 4 different times now. That’s what LISP’s flexibility is being used for in this case. The actual HN codebase probably saw fewer major changes to its implementation than Arc.
I used to have a soft spot for Common Lisp. That said, I’ve reached a point at which I don’t think I get anything out of new languages and I don’t think the choice of language makes much difference unless it’s highly optimized for the task at hand.
In part I think this is because all mainstream languages end up with a pretty similar set of features (can you tell I don’t work in highly constrained environments), modulo some features to statically reject certain programs.
At this point the killer feature of a language for me is the level of pay.
This is probably all related to the fact that I very rarely do side projects, because I rarely have a dissatisfaction I believe can be solved by information organization or computation. Which might all boil down to a certain lack of imagination.