You Bought Zuck’s Ray-Bans. Now Someone in Nairobi Is Watching You Poop
114 points by krinkle
114 points by krinkle
The only way this is ever going to stop is to make it completely illegal, with no exclusions, loopholes or contractual outs at all. It should not be possible to enrol an end user of a retail product in a program that shares personal data with third parties under any circumstances. End of story.
These companies have had decades to get this right, and they push the boundaries of language and the law every single time. Privacy law is a sham to let them write policies which explicitly reduce privacy protections.
These companies exist solely to transfer value from us to them, with nothing of value given to society in return. I am so sick of it.
For most of their products, nobody is using force or threats to get people on board. There's some value people (think they) get out of these products, even if it happens to be the same value as an addict gets out of a dose of heroin.
As a society, we need to figure out how to break that spell, or we'll end up in the same place through different means.
For example, even the bright line of sharing data with third parties is quite easy for a company like Facebook to solve, through acquiring the third parties. Now, it's a first party that's structured to operate independently within a conglomerate.
Yes individuals get perceived value but the net value to our community is negative.
You are right that it’s not just about third parties. It should just be illegal for companies to have access to private data under any circumstances. Maybe a very thin exception for data that’s been encrypted by secret keys held only by the user.
That is not what is happening though... Somebody bought these glasses and they're recording me in public, which in my country is forbidden without my consent. This is a tragedy of the commons. I never signed up to be filmed by these glasses.
This is like allowing people to drive dangerous cars on public roads because they made an individual choice, and you try to cope with it by saying "we have to teach people in making better choice." No! Their device is a danger to society and should be banned.
Agreed.
I think we should hold people accountable for their decisions.
Also, people vote with their money.
People are also happy not thinking about technology at all and just want to be consumers.
Its possible to not use these tools, or alternatively one can fund alternatives.
Thoughts?
"Voting with money" only works if we presuppose capitalism works. The idea is that "the market" is to conjure up "choice" to cover our tastes and preferences, but that doesn't happen. We've allowed market monopoly to the point where we have a handful of giant companies controlling everything.
It's not that it doesn't work, in fact I would argue it works perfectly. We must recognize that the objective laws of motion of capitalism trend always toward monopoly, because the only other alternative is an eventual death-spiral of falling profit rate.
It's also worth noting that in an environment where there is massive disparity in wealth, "voting with your wallet" is precisely the opposite of "one person one vote"; it's a voting system exclusively designed for the wealthy to control and everyone else is made powerless.
In political voting, there are no end of ways people can nullify or dilute the impact of your vote. In contrast, every single dollar gets counted, one way or another, and every one carries the same weight as every other one. It's important to distinguish between monetary decisions a consumer makes in their best interest and the massive corruption at scale among billionaires and politicians.
People also prioritise short-term benefits over long-term sustainability, specially if they are uninformed, or worse misinformed. Others might not have the means to "vote" how they would like to, even if they might want to prioritise long-term sustainability.
Do we know if those people have the information and resources to make an informed decision? Are we supporting an environment where we all have access to those resources, without risk?
"We judge our actions by our intents, and others' by the results". I don't remember how old I was when I first heard this, but it stuck with me since. We know the context that led us to make our decisions, and are thus kinder to ourselves, even when we fail. We lack that context for others, so we often infer our own context on their decisions.
We all make mistakes. While our actions have consequences, I think we should aim for a society that tries to "rehabilitate" rather than "punish".
Articles such as this one are a good step. Maybe it reaches someone who was not aware of the consequences of wearing these glasses.
A big thing in the original svd.se article is the bit about talking to salespeople at the stores that stock the Meta stalkerware. None of which could explain what data is collected and what is done with it, and whether it is possible to opt out.
And then also the exchange with Meta itself, wherein the as vague as possible terms of service could be interpreted to cover anything, ranging from gathering statistics on battery life, to live streaming someone's daily life without their knowledge.
to make it completely illegal, with no exclusions, loopholes or contractual outs at all
You will find that this thing you're wishing for doesn't exist. Many laws have been written like this, only to contain loopholes or be watered down over time. In fact, that is often the natural life cycle of laws.
Link to the full investigation referenced in this article: https://www.svd.se/a/K8nrV4/metas-ai-smart-glasses-and-data-privacy-concerns-workers-say-we-see-everything
This is so wrong in so many ways.
But as usual, we ignore the downstream ramifications of what we flush (in this case, almost a bit too literally?) down the toilet.
People should be made to feel embarrassed to wear these things in public. Any cafes, restaurants of bars that kick people out for wearing them will have my loyal patronage. These are unbelievably antisocial devices. Wearing these things is a marker of complete indifference to other people.
What specifically makes this wrong and bad? What ethical principles are violated, boundary crossed, harm done?
My first reaction was negative too, but I'm not really sure why exactly. Seems the case with other comments here as well. It could be interesting to try and understand why.
For me the worst bit is that someone else's decision to wear these affects my privacy if they happen to film me. And I probably don't even notice because they're relatively inconspicuous.
Makes sense. I guess here it falls under the somewhat established principle of when you may film people in public. The laws differ, but photographers generally try to not hide their camera and the fact of filming (that's why some cameras make loud clicks on purpose), and it's generally allowed to film things in public places as long as you're not overly focusing on someone. Like shooting a crowd is alright, but stalking one person without their consent is not. Or a more paradoxical example: you may shoot a a house standing on the street outside, but you may not shoot the street from inside the house (you're not visible).
This way, people are given a chance to notice someone who looks and acts like a photographer, and are not taken advantage of, while the photographer is given the opportunity to create art we all enjoy without unreasonable scrutiny.
By this metric, one could say Ray Bans are a device inherently made for a covert photography that does not follow established social boundaries. At the same time, these aren't really photos that go into someone's personal album or an exhibition, so you could argue it's not photography. Even when the guys in Nairobi process them, they're anonymized enough and not attached to a person they'll ever meet in their life, and enough measures are taken to make sure they don't remember or store them (i.e. phones are prohibited, and I'm sure Meta has enough lawyers to screw any petty employee who'd want to keep your wife's shower pic for their memories). For them, it's not different from any photo they may find with googling "naked person" or "poop" online. And for the person recorded, this is literally indistinguishable.
There's still the question of consent. Meta could argue that this has been in the service agreement all along, and we could argue we never read that because we don't have as many lawyers.
You are arguing from a legal perspective, if I understand you correctly. That has value in front of a court, but I think discussing how we want to shape our society and what should be (morally) permitted or not is better done outside of a legal framework. Especially if we want to argue about privacy, for example we also have to talk about how constant surveillance changes an individual's behavior. This is not covered by a legal framework.
I think legal is a slow, imperfect version of ethical. An I'm arguing about ethics here, i.e. what would be the reasonable ethical principles to break down this situation.
The gist of my point was that photography and data collection are valid reasons to film people, and to be conducted ethically they must follow some rules. In principle, a device like this could be ethical with both letting wearer take pics of others (say, let them make a shutter click noise when wearer saves a pic) and with anonymous data collection (make honest ToS, tell explicitly that it's wearer's responsibility to get consent from people around, and that Nairobian guys will be reviewing pics).
I agree with what commenters in the other branch said. It seems like Meta hasn't given the wearer a chance to make an informed choice, but tries to make it look like they did to then claim they're not the ones responsible. So to answer my own question from top comment: Ray Bands are bad because they're neglectful of privacy and deceitful about choices and responsibilities with with the wearer.
If you walk into a public restroom and you have these glasses on, recording, that would affect someones privacy and most likely be illegal. Is the person who walks around with these on going to make an active choice to put away the glasses whenever it becomes illegal once you go from public to private? Probably not.
It really depends on what side of the privacy fence you stand on.
For me, I would personally consider it a violation of my privacy if some complete stranger were filming me while I was doing whatever out in public, such as grocery shopping or getting gas. I do not wish to be filmed or appear on anyone's social media because I have a pretty strong stance about the dangers of living in a fishbowl, especially with fascism on the rise again where today's action may turn into tomorrow's violation, since one of their favorite tactics is moving goal posts about what is and is not acceptable under their authority. I say all this as a person who has been in a two decade fight against social media because I thought it was a bad idea from the start, at first for psychological well-being reasons, but my reasoning has grown since. Privacy is extremely important to me, not because I have anything to hide, but because I believe everyone has the right to have agency over their personally identifying data. I'm also old enough to remember an era of the nascent Internet where this was much easier to achieve and the data brokers at the time were mostly concerned with people's credit score and mailing address, as opposed to algorithmically mapped habits which is far, far more invasive feeling. Sure, there were street cams back then, but people were not walking around with cameras and audio recording devices in their pockets. Surveillance was a concern, but it a lot harder to achieve without people noticing.
However, not knowing you or your background, perhaps you are younger or have a different take on what privacy means. This is where the discussion really starts, with all of us sitting down at the same table and openly talking about our concerns and fears surrounding privacy, mass-surveillance and the ethics of data harvesting for the purposes of manipulating public opinion, consumer behavior or anything else we come up with during that discussion, including what benefits you see to the current path we're on.
Happy to have that discussion, if you are game, and anyone else, for that matter. My goal is not to convince anyone to take my side, only because I've wasted a lot of energy on that already and it really doesn't change anything. People need to come to their own conclusions, but when we come to different conclusions, open discussion promotes understanding, acceptance of differences and sometimes even agreement by compromise.
Thank you for the thoughtful response.
I feel the same as you about being filmed in public, but I'd balance that feeling against value of street photography. I think it's a public good to allow people to make art like that without having to go around and take signed letter of consent from everyone, and we're in a good vs. good situation here. As much as I'd like to have that privacy, I can't feel I'm entitled to it to such absolute degree. To me, the balance is in allowing public photos with some restrictions, that could more or less strict based on your culture's preferences.
Ray Bands are not doing photography though, it's more like data collection. It could be justified if the data is anonymized and can't be traced back to the person, but I really can't say what my stance is on this kind of data collection.
Well one of the worker explained that they were looking at someone's wife undress without the wife's knowledge. In case it needs to be spelled out, it is normal (in fact, it is a human right) to want privacy, and it hurts when that right is violated. In most places it is illegal to record someone where privacy is expected.
In the wife situation, I think it's the husband's responsibility to inform her and get her consent to use Ray Bands when she's undressing. If I go to rub my wife's back in the shower while running a live stream, it wouldn't be reasonable to blame the streaming service. It's not the first scandal caused by one person filming and posting another without the latter's consent, and we generally don't blame the platform for it.
The degree to which Meta made the husband aware of him having that responsiblity is unclear to me.
I agree with what you say about wanting things and being hurt in a situation like that. I just don't feel entitled to get something just because I want it, or call something wrong only because it hurts. Calling something unethical or wrong requires a more nuanced, comprehensive analysis.
It is unclear from the article whether the husband is
A) aware that the glasses are filming at all
B) aware that the glasses are sending the video feed off-site
In the wife situation, I think it's the husband's responsibility to inform her and get her consent to use Ray Bands when she's undressing.
I mean, in general, yes, absolutely, but this highlights just how irresponsible the regulatory logic behind modern data-sharing devices is. Or, rather, the wider public's logic about regulatory action, I guess? Have you noticed this bit of Meta's statement:
We take steps to filter this data to protect people’s privacy and to help prevent identifying information from being reviewed.
Speaking from unfortunate experience here: this is a standard bit that's not inserted just because it sounds fancier with extra words. It's a standard bit inserted by virtually everyone in the industry because it shifts legal responsibility.
If Meta said "We protect people's privacy and prevent identifying information from being reviewed", and it turns out, to the surprise of no one who's familiar with automatic II scrubbing, that that does not work, that would paint a big target on their backs.
But if the statement is, instead, that they take steps to do all that, and it turns out those steps are, unsurprisingly, pretty inefficient, that statement isn't as big a problem: it's factually accurate, and implies that, well, customers made an informed decision about the risks (with Meta's marketing and legal departments fighting them every step of the way to make sure it's uninformed, but that's not illegal).
Sharing someone else's pictures without consent is obviously illegal and wrong, and sure, they should have asked for consent (although, as someone else pointed out [1], it's unclear if they even knew they had to).
But the bigger problem is: if Meta can't efficiently protect people's privacy, why in the world should they be allowed to sell devices that break it?
None of us is morally obliged to just hang hang on and be supportive of Mark here until his nerds figure this out. If they can't solve this problem efficiently right now, well, that sucks -- recall the products, put them back out when it's solved. Maybe it's unsolvable and you can't make a privacy-preserving device like that? Aww, shucks, I guess it goes right up there on the shelf of things that would be cool if they could exist so you could sell them, right between the warp drive and the time machine.
In the wife situation, I think it's the husband's responsibility to inform her and get her consent to use Ray Bands when she's undressing. If I go to rub my wife's back in the shower while running a live stream, it wouldn't be reasonable to blame the streaming service. It's not the first scandal caused by one person filming and posting another without the latter's consent, and we generally don't blame the platform for it.
Partly yes, they are to blame, and fully so if it was intentional. But it seemed like an easy mistake to make, and it doesn't seem to be clear to people what data is being shared. After all, not even the vendors knew what kind of data got shared.
Generally when devices are recording, there are affordances made by the designers to indicate it. For photos, that is a sound (which is required by law in some places), or for recordings it tends to be a glowing light. I do think designers of these tools are responsible for making it clear when a device is recording or not. I've seen these glasses in the wild, and it was not clear to me at all that 1. they had cameras attached to them and 2. that they were recording.
I agree with what you say about wanting things and being hurt in a situation like that. I just don't feel entitled to get something just because I want it, or call something wrong only because it hurts. Calling something unethical or wrong requires a more nuanced, comprehensive analysis.
Right, which is also why I mentioned that there are established laws and rights around privacy which I think are very relevant here.
Let's consider another case of Meta "accidentally" causing harm:
Meta’s algorithms proactively amplified and promoted content which incited violence, hatred, and discrimination against the Rohingya – pouring fuel on the fire of long-standing discrimination and substantially increasing the risk of an outbreak of mass violence. The report concludes that Meta substantially contributed to adverse human rights impacts suffered by the Rohingya and has a responsibility to provide survivors with an effective remedy. (The Social Atrocity: Amnesty International)
Do you really think they deserve the benefit of the doubt here?
I do think designers of these tools are responsible for making it clear when a device is recording or not.
I think this is a very good technical take, but I also think this is a social and economic, not a technical problem, so it won't have a technical solution.
Designers who work for a company have very little professional autonomy. My friends who chose to go into medicine instead have that responsibility and the means to enforce it -- if their manager orders them to e.g. administer an illegal treatment, or one that they know would cause harm, they can refuse to administer it, and if they're fired for it they can absolutely take it to court and win. If Meta's VP of Engagement tells their hardware engineer to remove the recording LED and they refuse, they'll just get fired, there's no real legal recourse, and Meta will just churn through engineers until they find someone who'll remove the LED.
That's why I think the legal onus shouldn't be on designers, but on their employers. That's the party that vets these devices, drives every aspect of their design (including privacy issues), sells them and so on. It's also the party that gets the profits, so it stands to reason that they should bear the responsibility as well.
I seem to recall that phone manufacturers had to implement click sounds when a photo is taken because people were snapping upskirts behind women and girls on the escalator. Do these glasses make noise or even have a red "recording" LED?