Signal creator Moxie Marlinspike wants to do for AI what he did for messaging
38 points by snej
38 points by snej
This doesn't make me trust AI more, it makes me trust Moxie less.
I dunno. Regardless of my feelings about LLMs, if people are going to use them (which they do), I'd rather they have a way to do it that doesn't involve shoveling data into the surveillance machine.
If you sincerely object to the surveillance machine, don't consume its fruits. If you use locally-hostable LLMs you're still supporting the narrative that enables the non-locally-hostable LLMs to be created. The "privacy-preserving" version of this misbegotten technology is still built on the mass surveillance and exploitation of the commons, disrespecting the wishes and efforts of countless millions of unconsenting human contributors. Absolutely nobody needs LLMs, under any circumstances, even "as a treat".
Absolutely nobody needs LLMs, under any circumstances, even "as a treat".
That’s plainly false. LLMs will spend way more time with you, explaining that heart condition in ELI5 you’ve got, in a way you can understand and then you can discuss than any doctor / ask about those drugs the doctor gave you and even review the schedule, figure out ways to make sure you don’t forget to take them.
An LLM can help you build a car engine, because collecting all that sparse info about that 90s car engine online is very complicated.
Etc.
I asked how to tighten my accessory belt, it told me the wrong side of the engine, so I gave it the index of the service manual, it told me to wait 24 hours before I upload the section containing the information needed.
Its not going well so far.
I wouldn't want to be anywhere near that engine built on LLM-generated info when it gets started for the first time…
True.
But also, you do not want to be anywhere near the first start of any new-model-line engine!
I think you are mixing surveillance machine and plagiarism machine.
Largest LLMs can't source enough input data and so won't pass any surveillance data they can grab, hosted LLMs are parts of surveillance machine directly, but local-sized LLMs can be trained just fine on things written for an unbounded set of strangers to read. Putting the meatgrinder output produced out of such input into the commons is an issue that is not about privacy.
I think that this is a false dilemma given that llama.cpp exists; people can run LLMs locally on commodity hardware without any tokens leaving their LAN.
The general public is not going to run their own llama.cpp, but they might still want to interact with a Lemon.
I suppose the same argument could have been made for pgp or matrix servers.
Moxie isnt making this for tech people.
I think Corbin's argument isn't that end users can run the literal llama.cpp, but that it's possible for end users to run this sort of software on their devices in general. You could absolutely make a local LLM chatbot thing that doesn't target tech people, and that is instead supposed to be as simple as possible.
I get that, and I also don’t love all the decisions signal made, especially being tied to a phone number and defaulting to auto discovery.
Moxie is going to make moxie choices, and I probably won’t love them but generally his goal is ease of use.
LLMs are here to stay, high quality open source models will be important if we don't want to be forever beholden to large cloud providers
Posted before: The sound of inevitability
I don't believe in LLM inevitabilism.
There's no actual rejoinder in that article. There are many concrete reasons to view it as inevitable. People aren't going to deliberately forget a hugely helpful productivity technology, there will always be some jurisdictions where the training is either legal or bans are poorly enforced, the cost of compute marching lower means it will only be more practical more and more people to train them, and the worldwide research community isn't going to stop working on improvements either.
The irony is that “nothing is inevitable” is just as unresponsive to material reality as the view it takes itself to be responding to. Surely some things are, practically speaking, on short timescales, barring true global catastrophes, inevitable.
If we don't want to be beholden to providers, what we need is the tooling to do with truly locally models things impractical for hosted runs. (Not just running possibly open or possibly not model in a cloud)
I agree 100%. My personal LLM usage is negligible but I'm also happy something is being done in this space. LLM conversations encrypted via TPM TEE is quite an interesting way to do it, and I kind of see the vision here.
That said, trust in Moxie is dwindling, but I know he can deliver on the tech, as evidenced with Signal (excluding mobile coin drama, and excluding ridiculous delay on usernames).
Ridiculous delay on usernames shouldn't be overlooked though — it was a threat model issue with real-world consequences. With a user under duress, police (especially in countries where legal protections do not work) extracted identities of those Signal contacts that are within their reach — for free, identities of Telegram contacts with an overhead which meant they did not always bother to do it, and for non-phone-tied systems you need to do actual work. So you need to double-check what is the tech delivered even supposed to do…
The username change, which also ended the entire SMS-app integration aspect of Signal, put an end to the experiment of getting my parents on Signal.
People are free to think the username issue is important, it is for some segments of the population, but this ignores the original goal of the app.
The introduction of usernames and ending the SMS-app integration were separate issues i thought. SMS integration died long before usernames got added.
In my experience the introduction of usernames hasn't changed anything: If you have someone's number saved as a contact and they have Signal they will still show up in the app.
In settings there's a privacy toggle about users finding you by your phone number. But I thought it was on by default.
The username change, which also ended the entire SMS-app integration aspect of Signal, put an end to the experiment of getting my parents on Signal.
can you expound on that? I don't understand how introducing usernames made it harder to get parents on Signal.
Or is it the fact that they turned off SMS integration? (I found SMS-from-Signal really handy, though I also understand how it makes it too easy to mess up and send sms when you meant to send encrypted.)
It's the fact that they removed SMS integration. Previously you could give Signal to people and they could just use it for everything, as their primary messenger. Now it's yet another chat app, but this one only for talking to you.
Makes sense. I also notice some family members don't notice messages sent to them in Signal because they don't open the app, I would imagine they'd open it if it were their main messaging app.
OTOH, the SMS integration never worked on iOS (I think?).
I think you're right, which is why I had a hard time getting an iOS-using friend to use it. He agreed to use it, and installed it, but just never used it.
Ah, so they still failed to learn from Telegram and do better? Allowing username-only contacts does not in itself forbid having number-based contacts when both sides prefer that.
Introducing Confer, an end-to-end AI assistant that just works.
I assumed end-to-end [encryption] was a buzzword accidentally added into the copy by a redactor that didn't know better, because you're just connecting to a central server - but no,
Data and conversations originating from users and the resulting responses from the LLMs are encrypted in a trusted execution environment (TEE) that prevents even server administrators from peeking at or tampering with them.
That makes sense actually. Trusting hardware owned by others is definitely a reasonable thing to do, after all.
Conversations are stored by Confer in the same encrypted form, which uses a key that remains securely on users’ devices.
I assume the point is that they're stored so the LLMs can use them as context in future conversations? Storing that much user data feels weird, considering e.g. Signal didn't even support cloud backups until recently.
The article focuses on passkeys for some reason, but I'm curious how they do inference. I assume that just runs under TEE? I heard homomorphic encryption is still way too slow to be used for this sort of thing, but I don't really follow that space. Nor know much about this entire area.
I can't visit Confer's page because I'm not using "a device with platform authentication (Face ID, Touch ID, Windows Hello, etc.)", so I guess I won't get my answers there. Does anyone know how to install Face ID on Debian?
edit: My phone had all "these advanced features" required to be able to see the link to their blog, https://confer.to/blog/, which does actually work on my laptop. It does seem they run inference under TEE.
The article focuses on passkeys because that’s how Confer is able to generate and persist a secure private key on the client, and as a bonus sync it securely to the user's other devices. There’s a blog post about it.
That is a much better user experience than being shown 12 random words to write down and “store securely.”
I wish he explained how passkeys manage to avoid this. Sure, you can use some fancy hardware security features to lock the secrets behind a relatively weak secret such as a fingerprint or a face scan, but how do you back that up? How are the backups protected? Unless you have a second phone, you can't rely on these features anymore, right?
Also, this is the hardest case of TEE, as you need the protected domain to span CPU and GPU. I can even believe that Confer probably implement their side as correctly as possible. But will Nvidia?
Most LLM apis require the caller to provide the complete context in each request. Obviously in this case the context would still be plain text when it's processed by the model, but I can see a workflow where there's nothing persisted on the server. It would just stored in memory when it's actively being processed.
The private inference article states that the TEE hardware adds an attestation quote to the handshake. But how can the client be sure that this comes from a TEE?
Presumably, because some parts are signed by Intel keys and soem parts are signed by Nvidia keys.
It's depressing to see privacy advocates overwhelmingly pushing remote attestation technology while I am convinced it will lead to a future with no meaningful software freedom and hence no (or at least seriously limited) privacy options. Indeed I was prompted to write that by being locked out of a somewhat important app due to using a privacy-focussed OS.
why tf the literal landing page of the Confer project has an unskippable "sign up with google" modal. Like seriously, what is wrong with the designers.
I wonder how this server infrastructure compares to Private Cloud Compute
PCC is just the Apple name for TEE. Whereas TEE is just a slightly more generic term for Intel SGX.
None of then have been proven suffixes trustworthy imho.
What do you mean by suffixes trustworthy?
This misses so many things, and leads me to put less trust in signal than I had before. GPUs are not attested at all, Python dependencies are not properly pinned, etc.
I think https://github.com/edgelesssys/privatemode-public is how it should be done, although I’m a little biased here since I work on making that exact thing secure. ;-)