Markdown Ate the World
68 points by emschwartz
68 points by emschwartz
I would love Djot to get some more traction: https://github.com/jgm/djot
From the README:
it fixes a few things that make commonmark's syntax complex and difficult to parse efficiently. It is also much fuller-featured than commonmark, with support for definition lists, footnotes, tables, several new kinds of inline formatting (insert, delete, highlight, superscript, subscript), math, smart punctuation, attributes that can be applied to any element, and generic containers for block-level, inline-level, and raw content.
The problem with Djot (and, I think, the problem that all Markdown-alternative formats tend to have) is this:
a sublist must always be preceded by a blank line
Yes, I get it, Markdown nested lists are ambiguous to parse. This is one of the first problems alternative formats try to solve. But I have muscle memory from a decade+ of writing Markdown, and I can't be the only one. Nested bulleted lists are my standard way of writing notes, whether it's in Markdown or plain text or anything else. And I always write them without extra newlines. If I used any format that didn't parse that correctly, it would constantly mangle my notes.
I stopped having problems with lists when I started writing them like this: https://jmmv.dev/2022/07/markdown-lists.html
What's even weirder is that the list parsing rule change isn't even strictly necessary. For example, when I wrote an Inko parser for a Markdown subset parsing lists was mostly fine. I did limit the list markers and only allowed inline values, but both those decisions were based on that being fine for 99% of the uses for lists anyway.
I'm also glad that markdown took off if the alternative was unconventioned plain text on the one hand, and horrors like docx on the other. But I do think we could have done, and yet can do, better.
13 years ago my notes were in in Rich Text, from there I moved to markdown, a few years ago I switched to Djot out of frustration that markdown's underspecified origins and overflavoured descent which make it so easy to write, make it extremely difficult to validate when you are doing anything custom. This year I migrated fully to Typst, including embracing its (experimental) HTML output as the rendering core of my own static site generator which previously used markdown, then Djot (example page generated by my SSG).
In each case I have had my own custom markup for things like galleries, embedding ePubs in documents, etc and in each case I have (programmatically) migrated all of my notes and digitised versions of my journals (now numbering almost a million words). Moving from markdown to Djot was painful because of how (imho) needlessly flexible markdown can be. Migrating from Djot to Typst this year was glorious, it felt as effortless as converting between JSON/YAML, including writing a script to convert all my custom "shortcodes" to Typst functions.
There was an interesting thread on here a while ago about how only XML and its descendants actually qualify as arbitrary markup languages. I can't find the thread at the moment on my phone but as I read it I found myself nodding along. Typst is not an arbitrary markup because every custom function must be defined somewhere, but for now at least this middle ground strikes a very pleasing note for me of flexibility (to add new functions) and strictness (anything unknown is an error).
If you never use anything but the subset of markdown which is consistent between almost all processors, I think it's a relatively fine "standard", but anything outside of that seems to fly in the face of the structure and specification that probably most of us here care a great deal about.
(To be clear, I'm not knocking anyone's use of markdown, I just think that better tools exist, and yet better tools can exist if we allow ourselves to conceive of things Beyond Markdown)
I am very excited for Typst to (eventually) replace LaTeX. I've run into so many issues writing LaTeX that all feel like they come back to the original problem of Not Enough RAM for multistage compilation.
I find djot to be exactly the wrong level of abstraction for anything I want, things I write come in, broadly, two categories:
Djot sits firmly between these two use cases.
I wish I didn't agree, because I'm very grateful to John for exploring what a better markdown could have been, and for my purposes Djot is strictly a better markdown. But as it turns out, I wanted more than a better markdown.
I'm inclined to think of Djot and markdown as the deconstructed components of Postel's law. Markdown is tolerant, but through the paradox of tolerance it has been mutated into something unwieldy. Djot is strict, and the move from tolerant to strict demands breakage, but Djot is ultimately unsatisfying as it breaks both too much and too little, landing in this marginal-improvement middle-ground a half step between markdown and a more ambitious breakage.
Typst feels like the right clean sheet approach, taking in all the decades of markup mutation and experimentation, and distilling it. By being multiply derivative (ie. of markdown and LaTeX), it does strike me as having charted a new ground, near to the simplicity of markdown, but without its footguns, while affording the expressivity of LaTeX.
After giving djot a honest try, I have to agree with you. The syntax is constantly getting in the way even after weeks of using it. Personally the asymmetric math syntax is what ruined it for me; I just could not get used to adding a dollar sign before backticks (that are awkward to type in my native keyboard layout already).
For my most recent book, I used something like SILE's syntax. It's basically cleaned-up LaTeX and is very easy to type.
Djot is truly great. I'm using it for everything now instead of Markdown. Finally a real standard with no ambiguity.
Djot is weirdly inconsistent
On balance I think it isn’t enough of an improvement over markdown to be worth using two lightweight markup languages instead of one.
Interesting, but I think these are minor setbacks that don't denigrate from the magic of having a real strict working standard that is easy to parse.
I really like markdown. I blog in it. I take notes with it. I have hundreds of recipes all neatly written out in markdown.
It lets me use version control. It means I can use super quick tools to search through files, without ever worrying about changing them by mistake.
I'm aware it isn't perfect but I'm glad it stuck.
Non-techie people I know have begun to use it and it has helped them see the benefits of a non-wysiwyg system. Achieving a similar level of consistency in styling can be really brittle in word processors, and quite time consuming to set up.
All I can say is that with the Advent of typora I haven't used word in over 3 years! Thank the universe for pandoc!
My current contract has a whitelist of software I'm allowed to use. I had no idea how much productivity emacs, pandoc and plantuml gives me before I lost it.
EDIT: the whitelist is so short everything could be installed on every machine at all times, this is of course not how it works.
I think the main reason why Markdown "won" was John Gruber.
When markdown was conceived it was one of many other text formatting "standards". Ward Cunningham had invented one for his Wiki. Wikipedia had one. Other wikis had their own. Textile was pretty big. I feel like there was another one that was pretty big, but I forgot.
What set markdown apart from all the others was not that it was better specified (it was not) or that it was exceptionally better than others.
What helped was that it was simple to implement and that it was well known thanks to Gruber and his friends being somewhat Internet famous. One of those friends was Jeff Atwood who later founded Stackoverflow and used Markdown as it's formatting mechanisms. And everyone else who was looking for a simple markup language was very likely also a reader of Gruber's blog at the time.
I feel like there was another one that was pretty big, but I forgot.
BBcode? I dunno what the blog engines used back then, just HTML?
Also, don’t forget Aaron Swartz and Reddit.
Or Aaron Swartz, who was friends with Gruber and then encouraged Reddit to use Markdown, which in turn inspired a lot of other uses.
I don't agree that Markdown "won" over doc(x). Perhaps in instant messaging and in the programming world...
In administrations, academics, etc., the word processor and its docx artifact is still the main way to produce and exchange documents.
My brother is a researcher in psychology and philosophy, and almost all journals require docx in their calls for papers.
It has been very frustrating to him because using LibreOffice in a professional capacity (typesetting entire books with indexes and bibliography) is a mediocre experience. I once had to develop a small tool to fix the docx contents because the index would keep getting corrupted on updates (duplicating each entry) for some reason. Another time, the entire file got corrupted and started adding a non-trivial amount of empty lines between each line, each time he was saving the document. It would even be transmitted by copying and pasting the content into a new document, so he ended up typing the whole article again.
For typesetting, LaTeX is much better than docx as you at least don't incur the risk of having your input corrupted. However, LaTeX penetration is virtually non-existent outside of maths, physics and computer science. As for Markdown, is it unsuitable for typesetting as it lacks too many functionalities.
As a Vim user, I struggle to figure out whether to insert hard line breaks within paragraphs in Markdown or not. I’ve been doing hard line breaks all my life but recently I’ve questioned it. Vim just doesn’t seem to have a good way to soft wrap at anything less than the width of the window. Am I missing something?
I always follow a one-sentence-per-line style for Markdown (and other markup languages). Vim doesn’t make it as pretty as a rendered version, but it makes diffs much easier to read because line diffs show only modified sentences and you never need to reflow unrelated things in the source as a result of a change.
I just recently installed markdown-lint as a precommit hook (and CI check) in all my repos to force line breaks in markdown, because the alternative is madness.
The vim shortcut to reformat your entire paragraph (gq{) is an important one to know.
I do one phrase per line: roughly, punctuation and "and"s etc each get a line break. Makes editing easier and diffs smaller. It's such a simple and easy method, I can only assume it's much older than me.
It’s the standard style in line-based markup languages such as roff, PUB, GCAL. See also the excessively prescriptive semantic line breaks.
The nicest part about markdown is that, even if you can't parse it, plaintext markdown reads exactly like you'd have written the same document on a typewriter (except for the 0/o thing)
I find it so depressing that Markdown won, because I personally prefer to write without markup of any kind, and to apply styles at output. So I use org, but org is locked out of a lot of the various tools that make working with other people easier. I've just been grinning and bearing it, but it's really a situation where, yes, it's better than XML or doc, but it's not good.
This reads like old news. Markdown has taken over the world over 10 years ago. Weird that this guy only realized it now.
This is what is called a retrospective. The article is tagged "history." Nobody is suggesting that this is news.