Email Simplicity vs Decentralized Chat
26 points by Riolku
26 points by Riolku
The recent discussions about the decentralization tag made me wonder: is email a successful decentralized system? From my perspective, email has its flaws (mostly no encryption, strapped-on security with SPF/DKIM, and... interesting HTML implementations), but is overall a great system from a decentralization perspective: it is (relatively) easy for me to run an email server and no one is the wiser.
So the main question of this post is: what makes this so much harder for chat? Is it a lack of standardization, encryption, history preservation? Or is email not as wonderful as it seems?
IRC, besides its lack of formatted messages, is also an interesting "decentralized" protocol to me: each user ends up running their own bouncer.
I know this post contains some innaccuracies, but I think it will generate good discussion :)
Take a look at these stats: Today in Email Hegemony · jwz
I've been running my own mail server for 25+ years, but it ain't for everybody, or even for most people.
I also run my own XMPP server. Talking to exactly 2 people.
I'm an awful person. When my first kid was born, I announced to my family that I am never going to send pictures of her through Big Tech's servers and that they have to install Conversations (XMPP client) to receive some. Now my close family use XMPP. Emotional blackmail FTW! :)
Before that, it turns out a few of my friends weren't opposed to stopping to feed Meta with their data and had adopted XMPP too. Don't give up. There are scandals about Meta about every week, sometimes it reaches non techies too.
I wonder if Movim.eu will be more suited for your use-case
Not necessarily movim.eu, but self-hosted movim, I did consider.
However, posts are either public or private (=all roster) ATM. That does not work well with my use case. Creating another account is cheap, but movim is a single account client, so it's a bit annoying.
I use an "antique" phone, PWAs are fine but native Conversations is much snappier.
Finally, as suboptimal as it is, my family members are used to mixing chat and pictures together in the same interface. They don't use Instagram or anything similar, so it was more natural to do things similarly.
Now, having said all that, I regularly re-give movim a try and maybe someday, I'll blackmail my family again xD.
73% market share from Gmail? Huge success for decentralization. It used to be over 80% from some studies years ago.
From a practical standpoint, all we need is for the likes of Google to be de-facto blocked from completely dropping the underlying protocols while still pretending to call itself email.
With over a quarter of users living in other mail servers the company is grudgingly kept in check.
That "5.8% other" metric could be hiding Gmail behind MX records for custom domains.
Custom domains are mobile. And we should safely assume this “other” includes protonmail, fastmail, zoho, etc.
Make no mistake, I’m saddened by the monopoly. But email is leagues better than, say, smartphones or browsers.
Chat is not as forgiving. Modern chat users expect real time updates and consistent state for group members. This is inherently harder to achieve in a decentralized system than email's optimistic message passing.
However it seems this has been solved by the recent MLS standard. We will probably see it widely adopted.
consistent state for group members
Is this a requirement? One of the worst parts of Matrix is the constant slowdown of syncing trying to make the state consistent. Can’t it be okay if 2 messages are received at a slightly different time/order as a tradeoff for performance & hardware requirements on both clients & servers?
It is not, neither WhatsApp or Slack fulfill this requirement and they are both widely used.
Yes. I was part of the group that tried out all the things before Rust went to Discord and it is amazing how many implementations struggle at getting a message from A to B in a reliable time frame and notify the user.
The first problem is that people are used to chat being instant. It's a question of trust. Even something as complex as SMS usually arrives within a few seconds.
The other experience is that a chat app that does not react instantly usually has the habit of hanging and often does not deliver at all without actively looking at it.
That is really annoying because that's fine most of the time, until - say at a conference - your chat platform becomes a dinner scheduling platform.
I still wish XMPP had gotten their client situation together, because it works really nicely on the server, but they never manged to get a consistently well-running client ecosystem.
Why is MLS better than the other protocols we have?
For encrypted groups - the simple way to do it (used by signal IIRC) is you encrypt each transmitted message once per recipient. When membership changes you simply switch which set of keys you encrypt for. In MLS you only have to send one ciphertext which is much more scalable.
The main difference in my opinion is network effects.
Everyone has email. Mail delivery is problematic, but if I set up an email server, I can send email to anyone. And everyone has to deal with the complexities of it.
(But what's the better alternative? Nothing exists, and I doubt anything can compete because of the network effects.)
Chat is all over the place and is even more controlled by large corporations, IMHO. If I set up Matrix, XMPP, Delta Chat or anything good, I'll be able to chat to a very specific 10 people. I don't know anyone in my country who does not have WhatsApp. As far as I know, nowhere the standard for instant messaging is not closed source crap.
One argument that's usually missing for me is the presentation and access to the history.
If I move from email hoster A to email hoster B I will grab my old folder structure and either migrate it, change it, or ignore it. The moment the switch is done no one can even tell I am now hosted somewhere else.
Compare this with Mastodon, where forwards are still kinda clunky (and came late), and because messages and message ids are (generally) public, I can't sort or move or remove them (emails also have unique ids, but except for syncing it's not the presentation as shown to the user, so no one cares - also let's ignore my kinda unfair comparison here because public ML archives exist, but they usually do provide proper views by thread, by month etc)
Not completely sure how it works for Matrix, I certainly have not found a way to organize my history here.
Mastodon IMHO is something completely unrelated.
History is different, yes, but I suspect only the 0.1% cares.
I recently migrated my mail archive from GMail to Migadu. It's painful even if you know what you're doing. (And GMail's uniqueness makes everything so much harder- that was actually one of my primary motivations to get off GMail!)
I think really an insignificant people are moving email addresses. Just owning your email address is still outside what's possible for most people, and that is the part I think we should focus on. (Next, email providers should provide easy migration services.)
Personally I think along two lines:
...
For chat... well, it's more difficult because with email mostly everything is compatible (except GMail's eccentricities), but... we still don't have a good open service usable by all, so migrating history is still very far off into the future.
We did have decentralized chat for a brief time in the early 2000s with XMPP. We could chat with people with accounts on Facebook, Gmail, Apple Messages, or wherever else you could get an account. It slowly got taken away in favor of walled gardens.
I don't believe that IRC is a decentralized system. It still requires a single server to hold all users and I don't think you can communicate cross-servers. For example, if you log into Libera Chat, you cannot join a channel not on Libera Chat's servers.
IRC is not decentralized, but in practice IMHO it doesn't matter. Most IRC clients handle multiple networks nearly transparently, and given that IRC is channel-centered, I think it's fine for "channels" purposes.
(Personally, as long as an instant messaging system has a good client API and allows third-party clients and preferably servers too, I don't care so much about decentralization.)
I think one could possibly make the case that IRC is federated, at least for some interpretation of the word. At least if I understood the server side correctly (I only only ever implementing client-side parts of it) you could have a bunch of people running their servers and connecting them together forming an IRC network.
This was how IRC used to work, it was one big federation with server linking. The problem was the protocol was not securely designed for this, and server linking was often abused, and the federation broke apart. Server linking is nowadays just used for load balancing, and the federation features are vestigal.
I think you have to take a very narrow definition of federation for this to apply. In a real federated network, different nodes can apply different governance policies, but I don't think this is possible with IRC; it's more like a set of mirrors to spread out load. There is still one canonical decision-maker for the network.
It isn’t? There isn’t one central irc network. Within an irc network, there can be more than one server, and that can be transparent to users. It meets my definition of decentralised.
That's more like high availability (and not a particularly great implementation).
It is not transparent to users at all when there's a problem either.
WeSome of us did have decentralized chat [...]. It slowly got taken away in favor ofwalled gardenssystems that work.
I don't want to say FTFY, but putting XMPP as even a proper contender at the time for the masses is disingenuous. It could have won the popular vote, but so many things should have gone differently
It worked for plain text, it usually worked for a certain subset of server configs and XEPs and it also sometimes delivered the message to the correct endpoint if you had several clients logged in. MUCs were terrible all around.
More like: it did exist and was somewhat usable, but no proper file sharing, no added benefits and so many problems, least of all no good mobile clients.
You might be thinking of the "finally does not drain your battery" time as being before big tech federated, but I would say by then most people had already moved on.
You should give XMPP a new try. I've been using to chat to non techies using my server, or quicksy (more WhatsApp-like, your username=your phone number), for years, with e2ee enabled. It just works. My mother uses it, and boy does she not like computers.
IMHO, unless you're going to count BBM (2005), until WhatsApp (2009) nothing in my environment did mobile properly, and much less multidevice. Maybe the best before WhatsApp was Google Talk... and that had XMPP support (until 2012?).
Note that the iPhone came out in 2007! So this was previous to the smartphone boom. When we had the smartphone boom, WhatsApp was what crushed the remains of XMPP- and it didn't have multidevice until 2015!
I may have the timeline wrong, but I think the timeline is more complex than what you are mentioning.
For me it's something like:
Telegram and Slack appeared in 2013, Signal in 2014, Discord in 2015... all long after WhatsApp had killed XMPP.
It is true that XMPP has never adapted well to the modern landscape- even when they've added proper mobile support, you need expertise to pick the proper components to use.
The other "open" alternatives are Matrix (2014) and chatmail/Delta Chat (2017).
Of course, the timeline varies according to geography; iMessage in the US means the US is quite different, and Asia was more advanced for a while, in my perception. However, it's just WhatsApp killed XMPP, like all technologies kill their ancestors.
It's impossible to answer this question without defining success.
Is email a successful system that happens to technically be decentralized? Sure. Is email decentralized in a meaningful way? Not even close--email continues to exist today only because a handful of unaccountable oligopolists have decided not to kill it yet.
Decentralized chat is harder because people working on it are actually interested in taking decentralization seriously.
I think the more important stuff that makes chat harder than e-mail is fact, that chat need to be on-line and immediate (so both parties are present at once, and we exchange messages live), while e-mail is off-line and delays are expected. These properties make e-mail much easier to host.
Disagree on a couple grounds; first of all, at this point now most people expect to be able to continue to receive messages while offline. In fact, I think from a governance/legal perspective, the email model is dramatically more difficult to implement, because if the provider has to store personal data beyond just username/password then their legal obligations are much, much higher than if they were just a router of messages from one socket to another. (I believe this is the main reason why libera.chat doesn't implement the ircv3 extensions that stores history of what happened while you were disconnected.)
I was talking purely from technical perspective, not socio-legal perspective. If we loon at that, then yeah, all distributed systems will have that problem.
I think email has been more accepted for a few reasons:
Chat lacked a standard at a time many many years ago, when it would have made a difference. @makeworld noted about MLS as a chat standard...but i think MLS only addresses the security layer around messaging systems, and not the foundational standards for chat systems...Now, i may be wrong, but i think that it doesn't matter! Even if there were some existing chat standards, far too many people just jump on whatever chat system their network already lives on.
Someone else here mentioned XMPP...It has standards, and was even adopted by Google's GChat, is the basis for WhatsApp, and so on and so forth...And yet, who here can say that XMPP is as well adopted - that is, adopted knowingly by its users - when one compares it to Email. I used to consider myself a fanboy of the matrix protocol...and while i still admire all the things and all the peoole behind the effort...its implementaitns have been far too heavy and resource intensive...so much so that I;m now looking back to XMPP...but, who the heck will i chat with? Sorry, felt like i was going on a rant there for a moment; apologies.
I purposefully did not address things like encryption, file attachments because extra features like would make my comments too cumbersome for me to start typing - without making this its own blog post. ;-)
Email is a federated system, which is not quite the same thing as "decentralized" (though it seems nobody can agree on what it really means).
One useful definition I've been noodling with lately:
Federated is where power structures can accumulate on a single provider and disadvantage others.
Decentralized is where the power structure is democratized/collectively owned and can't be concentrated to disadvantage others.
Simple : network effect.
AOL/compuserve wanted to make email private. Microsoft also tried but they never managed to do it.
https://ploum.net/winmail-dat-syndrome/index.html
XMPP is exactly that: decentralized chat. And it nearly took of. But Google managed to kill it. It worked. The rest is history.
https://ploum.net/2023-06-23-how-to-kill-decentralised-networks.html
This is not a technical problem, this is a purely political one. Email managed to exist before there were enough financial interests. No corporation want decentralization. Email is the only one they didn’t manage to prevent.
Same with Web/RSS: Google is successfully killing the decentralized aspect
IRC isn't really federated and I wish people would stop repeating that, it does have some degree of tree-like resiliency of the network, PER NETWORK, but that is about it sadly. The protocol is simply incompatible with a global federated network like email/SMTP.
That said, there is a standards based protocol just for that - XMPP, and the core protocol is codified by IETF RFCs while also having most extensions codified under the XMPP Standards Foundation (XSF) in a manner similar to IETF RFCs (they're called XEPs in XMPP-lingo).
Hell, even the address format closely resembles that of email - and it isn't uncommon to find email providers that automatically give you an XMPP account too.
That said, as others pointed out - email is merely decentralized on paper - the multitude of ad-hoc extensions done on top in the name of spam prevention and security (especially the requirements around PTR and rDNS) make it extremely difficult to not get blocked by one of the "Big Mailers".