Researcher plants 2m canaries into the AI supply chain. A privacy disaster surfaced
85 points by itamarst
85 points by itamarst
What did people expect when using "brokers" that route your requests to the lowest bidder?! The entire premise is thoroughly incompatible with any notion of privacy!
That was also my reaction, initially. Then I thought it sounded like I was blaming the victims in some measure.
On one hand, it seems obvious that you should've known that these people who built their systems by using others' data without consent would also use your data without consent, even though they pinky swear that they'll keep it private. But on the other hand, there should be some mechanism for making people who claim to protect privacy won't just turn around and fling your data into other peoples' hands.
And I don't know how much awareness there is among general users that these operators don't respect their privacy.
It's not the same people though so I'm not sure that holds.
More like if you've been on the internet in the past 30 years probably you should have figured out that if you send data somewhere then the other party has it... But still the privacy policy should exist and say.
But on the other hand, there should be some mechanism for making people who claim to protect privacy won't just turn around and fling your data into other peoples' hands.
Facebook and Google are blue chip stocks.
I disagree that the premise is incompatible. I rely on Signal to broker my GIF requests to GIPHY and I've used many HTTP proxies to tunnel my HTTPS connections. What is lacking in the LLM routing department is end-to-end encryption and, frankly, character within the people running these services. As a counter example without end-to-end encryption, I've used IRC for 30 years without any of my DMs being leaked by an IRC admin.
The premise is incompatible with end-to-end encryption; and once there’s money in it and you’re dealing with lowest bidders, that seems pretty incompatible with character from the operators.
There was no money in IRC. It’s probably not that the admins are noble so much as that there’s not much advantage to exploiting their power.
… though IRC did come from a time when character was more valued.
The premise is incompatible with end-to-end encryption;
Hmm. Haven’t come across that approach before, sounds like it takes care of most concerns; but within the TEE the inference still occurs in cleartext, so vulnerabilities in that machine’s software or failures in the remote attestation (also I’m not familiar with the limitations of this stuff—can the server provider lie about your code running in a TEE?) would allow shenanigans, and I don’t get the impression the TEE protection applies to the GPU, so monitoring GPU memory could let you steal stuff too. So there are still vulnerabilities to the scheme; it’s not true end-to-end encryption, though calling it juiced-up transport encryption feels a little inadequate too. I’m gratified to find Proton acknowledging that their thing isn’t true E2EE; I’ve found them too often ingenuous in the past (and they still significantly oversell their email E2EE).
Mods removed the original referenced article ("This appears to be a marketing press release / product announcement"), so maybe this one will be an issue too. But this article does include additional information, like a list of at least some of the services that were involved in the leaks.
Ugh, this is one of the most alarming things I've read all week, because the extent of the data leak is just astounding. I'll be warning people at work about this on monday, and probably rotating creds for one person in my team :(
I've used the referenced https://canarytokens.org/ pre-LLM. It is very simple to use: you provide an email address for the alert, a memo (to remind you where you planted it), and you can create tokens for many things (AWS keys, DNS names, URLs, etc.). The server that notices when the tokens are used can also be (partially) self-hosted https://github.com/thinkst/canarytokens
I created some tokens for testing and they did get triggered as expected. None of the actual tokens that I've planted got triggerred though (although as I say that was pre-LLM)