Remote Attestation Should Not Be Tolerated
24 points by stilic
24 points by stilic
This then begs the question: Security for who? It's definitely not for the user, as they are locked out of their device. Their security and autonomy have been harmed significantly.
The article argues against using the label "security" for remote attestation. I think it's pretty much right in its criticisms, but it's not complete:
A horrifyingly common attack addressed by remote attestation is that an attacker calls the user and claims their device is already hacked, but the attacker will helpfully walk the user through sideloading an app to "fix" it. The malicious sideloaded app either breaks out of its app sandbox or uses disability assistance features to steal auth credentials to banking apps.
Another is stalkerware, where an attacker sideloads software to surveil their victim, often a family member or intimate partner.
Another is apps with intrusive permissions. Nowadays the review practices of the app gatekeepers do a reasonably good job of preventing apps from requesting overbroad permissions, like a flashlight app that accesses the user's contacts. These aren't all obvious BS: every banking app could truthfully argue that always-on precise location tracking would help them reduce fraud. (And of course give them a lucrative product line selling that data.)
A variation on the previous is coercion. Employers and governments force people to sideload similarly intrusive apps, especially marginalized people with reduced autonomy like the poor and those crossing borders. RA is a kind of commitment device.
Remote attestation does have real security value along with its problems because sometimes the user is part of subverting their own security. Like everything in security, it's a tradeoff. I think it'd be great to see an article that addressed why these protections aren't worth the downsides.
Author here! :3
I feel there is no alternative for education, the user has to be educated not to listen to these kinds of scams and to evaluate the permissions of apps. It's a bit of a bitter pill to swallow, as I know it's not an easy thing to do, but it is something we have to do. We can't base our security around someone that doesn't know how to use computers, or it'll be a race to the bottom, with other people controlling the devices we should be owning.
As for governments and employers forcing us to install things, this should be objected to and banned. That a device is mine should mean that I have full control over it, no exceptions.
I also believe sandboxing and shimming applications is a very good idea here, we can isolate everything into its own environment if we don't trust it, and either give it no permissions or give it fake permissions and feed it fake data. Remote attestation goes against this; I can't just run my bank app in a virtual environment or it will refuse to run.
To your point about education, you might be interested in the classic paper Human Error: models and management, maybe also Sidney Dekker's Safety Differently for a good longer read. Techniques like poka-yoke and the SHELL model have been (convergently) developed for designing systems that are safe for highly trained operators.
Fully isolating applications would trade off assistance features like zoom, font and button size, translation, screen reading, speech-to-text, eye tracking, etc. As well as core functionality like copy and paste, moving photos/data/files between apps, screenshots, screen share, shared login, password managers, search. There's some big bullets to bite.
I feel there is no alternative for education
We can't base our security around someone that doesn't know how to use computers, or it'll be a race to the bottom, with other people controlling the devices we should be owning.
When opening a new business bank account and wanting to be able to use it immediately to make a payment, I decided to transfer a sum of money which tripped some internal controls, locking the accounts. I had to make a thirty minute phone call to answer a very long list of questions intended to deter fraud, asking only whether I was tricked or coërced into making this payment. (None of the questions were about any other kind of auditing or control. It was only about whether I was being scammed by telephone or similar.)
Basically, it was half an hour of: “did someone ask you to make this transfer?”—“no, I'm transferring money from an account wholly under my control to a new account wholly under my control, so that I can use this money in a fashion that I can wholly control.”—“did someone suggest using this money to purchase gift cards?”—“no, I'm transferring money from an account wholly under my control to another account wholly under my control, so that I can use this money in a fashion that I wholly control.”
There was no way to skip any questions, no matter how absurd they were. I suspect I would have been subject to none of these controls had I used a paper cheque, but I would have had to wait for the cheque to clear (and the cheque could have been reversed, and this transfer presumably could not have been.)
I don't begrudge the bank for this process. After all, it was only a slight annoyance. But I wonder whether the purpose of these controls was security or avoiding liability. No matter how much documentation and explanation I could provide to demonstrate that the transfer was not the result of falling for some scam telephone call, they clearly wanted a legally legible record of someone answering these questions.
So I suppose that considering “alternative(s) for education” might be totally irrelevant here. These efforts are only peripherally interested in increasing your security; they are primarily about the institution increasing its security, meaning its liability when something goes wrong. (I might even go so far as suggest, as I think we see in discussions around passphrases, that your security is an easy cover for the people working on these projects to help them countenance what may prove to be an overall degradation in the customer experience. As long as you can pretend that's the point, you don't feel so bad about it.)
Similarly, I suspect that there are other, easier ways to perpetrate these scams is also irrelevant. As long as those ways do not introduce liability, then the institutions don't really care. It's not your security; it's their liability.
Then, it seems, if we suppose that the above isn't just cynicism or pessimism, does this provide additional means by which to engage these policies? For example, rather than protest device attestation through technical means, perhaps the better argument is to advocate for broadened liability irrespective of these technical approaches…? Similarly, should we demand additional liability for the other actors who make these scams possible? (Should gift cards even be allowed to exist? In fact, giving cash is fine in many other cultures…)
Additionally, this advocacy need not be adversarial. There are already products that address the need for additional scrutiny that can address the liability concern—e.g., conservator accounts for elderly relatives—but they're mostly about assuming outright control. I don't think there are any bank products that allow for elderly, legally competent (“sui juris”?) on fixed-income with predictable spending habits to allow more technically-competent relatives to provide additional controls to avoid scams. If there were, I'd definitely sign up for that (even though my elderly relatives have so far impressively savy about the bevy of scam calls they receive!)
Of course, I don't know whether similar analysis can be used to address the concern of lock-in. Maybe what's best in that case is to… completely disengage from using locked-in products, which is probably more realistic than disengaging from using modern banking…
...and of course the user can also make mistakes, like visit a typosquatted site or install a lookalike malicious app.
The more common scam is simply calling someone up and asking them to transfer money to their "son" or "mother" who urgently needs it for some reason.
Remote attestation doesn't solve the common case.
Having walked non-technical family through how to restore computers, the attacks via walking the user through replacing their OS with a backdoored one seem very implausible: they simply won't understand the instructions, and each attack will take hours of 1 on 1 technical hand-holding. This would include steps where they're rebooting the device and calling you back. The cost of the attack is simply too high compared to convincing them to use the well attested credentials on your behalf.
Attestation is weak defense against high cost attacks, but it does a good job of locking users into big tech.
There's one place where attestation helps: if you're a high value target, and don't trust the security of your own installed os (eg, you're in a journalist in jurisdiction that mandates encryption backdoors), you could ensure you are running an OS that hasn't gotten backdoored when, eg, you hand the phone over for inspection by the police. But in this scenario where you're a very high value target for a state actor, vendors are part of the threat model, so the remote attestation provided by current systems isn't enough; you need to be able to enroll your own keys to ensure that the OS you're running is the one you chose to install. None of the current remote attestation solutions allow users to ensure that they're running the right custom, third-party build, and that the vendor can't force an update.
I think it'd be great to see an article that addressed why these protections aren't worth the downsides.
I don't disagree with your analysis. But I don't it's just a risk/reward calculation, either, because there's a deeper value at stake. I think computers have practically and psychologically become "part of the self" in many ways. And just as you cannot, even if someone is willing, legally buy a slave, I don't think remote attestation should even be legal.
The case for "surveillance", broadly writ, is similar. There are good things that can come of it, like crime prevention. Still, most people have the (imo correct) instinct that something much more important, something incalculable, is being lost.
I worked in anti-fraud. I've heard this argument many times, and I'm sympathetic, but I wish people making it would engage with the counter-arguments, so I'll try restating them.
Since this is making a purely principled case, I'll start with a principled objection:
Our personal devices are in a very real sense extensions to our body, which means that bodily autonomy should in some way extend to them.
Yes, that's a reasonable analogy. And just like with our bodies, that autonomy extends to the freedom to make binding pledges to avoid certain kinds of behavior in our interactions with others, because there are desirable things which simply cannot happen in the absence of such a pledge. Many sports require you to agree to avoid the use of steroids, sometimes as confirmed by urine or blood tests, even though this is most certainly a restriction of your bodily autonomy; similarly many events are sober, where as a condition of entry you agree not to do drugs before or during. We can contract to do jobs (such as doctors, therapists, military planners, etc) where we are going to obtain information that should not be shared, and this works only because we can make binding pledges not to reveal that information later.
So, yes, you should be able to run whatever software you like on devices you own. But you should also be able to promise, and demonstrate, that you are not running certain software, in order to participate in certain shared activities (such as competitive games, banking, etc). Having the ability to make this pledge is not a restriction of your freedom any more than the existence of tests for steroids in competitive sports is a restriction of your freedom.
So that's the principled objection. But there's a practical one as well: attestation has significant concrete benefits for users, contra this post. Lots of people have lost their life savings because someone talked them into sideloading a malicious app and then drained their bank account. The only two kinds of popular competitive online games are those with attestation and those overrun by cheating. Attestation is why your bank is willing to let you put your card on your phone (for fraud/liability reasons they don't like to issue tap-to-pay cards that can be trivially cloned, which means they need your phone to verifiably keep the cards somewhere they can't easily be copied). If a website's login form can be brute-forced by anyone with a subscription to a residential proxy and the latest password dump, you'd best hope it doesn't have anything valuable behind it.
(Why do we still have login forms, then? As I said, I worked in this space, so I can tell you: it's captchas, behavioral analysis, and fingerprinting. That's it. Captchas were already a joke before LLMs trivialized them, behavioral analysis has either unacceptably high false positives or >20% false negatives, and everyone hates fingerprinting plus browsers are trying to kill it off, which leaves us with... well, either nothing or remote attestation.)
Of course attestation does not make any of these bad behaviors impossible. But there is still benefit in making them more expensive, which it does. And these benefits are very broadly popular, whereas the restriction to our autonomy is felt mostly by highly technical people. "I demand to be able to participate in all these activities without attestation; it's your job to figure out how to deal with fraud/etc without it" is a nicely principled stand but you have no power to enforce it and your bank/etc have little reason to allow this. If you want to convince the world to give up attestation, you're going to need to provide some alternative way of achieving its benefits.
Banking is not a shared activity. The bank is holding onto my money on my behalf, and it's their job to do what I ask with it.
When I can hold onto the bank's money and make arbitrary demands from them before they get it back, the same way they do with my money, then we can say it's a shared activity.
... Banking is definitely a shared activity. The bank is a different entity than you. You are doing this activity together. What?
They are my delegate when it comes to managing my money, not my peer -- a shared activity, to me, implies a roughly equal amount of agency and power on both sides.
Ah. Well, that's not how I intended it to be read. Read "multi-party" instead of "shared", then.
(This is a side point, but no, they're not your delegate. It's not even your money anymore; the bank is just your debtor. Banks have always placed restrictions on what you can do with your balance, including placing controls on withdrawals, many required by law. You can find entities for whom this is less true, but they're regulated differently and more expensive. That's not the relationship you signed up for.)
extends to the freedom to make binding pledges to avoid certain kinds of behavior in our interactions with others
Banking is not a shared activity.
Banking is a business activity, regulated as much as or more than any other, which means that the current apparatus should be free to exert even more control for the benefit of society.
It seems wholly reasonable to me to suggest that a bank that has been granted the ability to operate broadly should be held to certain requirements, such as… there should be enough branch locations with reasonable staffing and hours of operation so that people are not forced to use a device to perform basic, routine transactions.
If there were the case, perhaps we might be less anxious about the dubious benefits of attestation if all it meant was a minor inconvenience?
Instead, I think the worry about these kinds of technologies is that it is inevitably accompanied by an overall worsening of customer service and customer choice. It's not only that you are forced to use certain devices in a certain way, you are forced to only be able to operate through that means to survive the demands of modern life.
Setting aside the dubiousness of these security approaches and the outright carelessness of their implementation, I think they are often objectionable just in their outright indifference to providing good customer service…
A few points that deserve to be made in these discussions but rarely are:
Security teams at companies like $THAT_ONE_YOU_HATE are largely not in the business of twirling their mustaches and cackling about how this time they will finally destroy freedom forever. Mostly they're tired and sad, because they see people getting hurt and see the fallout from it, and because any time they suggest that maybe mass-market consumer computing should have some safeguards to at least reduce the amount of hurt, they're accused of being totalitarian megalomaniacal freedom-hating dictators who are waging a "war on general-purpose computing" (they aren't). This is intensely unproductive rhetoric.
"Remote attestation" is just a straightforward application of public-key cryptography. If your desire is to ban it, you likely will not be able to do so without also banning things you consider acceptable. For example: how do you distinguish, in your proposed law or rule, between an evil bank manager requiring attestation to use their mobile app, and a virtuous cyber-dissident requiring their comrades in the cyber-resistance to use encrypted/signed mail?
"I own my device!" OK. You own your device. But you don't own my device. So why do you believe I have an obligation to let your device(s) interact with mine? More importantly, why do you believe I have an obligation to do so on terms that are set unilaterally by you and to which I'm not allowed to object? Don't I own my devices? Aren't my rights over my property just as strong as your rights over yours? If so, why shouldn't I be allowed to set terms for whether and how I'll allow you to interact with the things I own?
"I own my device!" OK. You own your device. But you don't own my device. So why do you believe I have an obligation to let your device(s) interact with mine?
IMHO if you're a bank or a government, where I am forced by law or circumstance to transact with you, then the onus to enforce security without trampling on my freedom is on you, not me. If that requires remote attestation, so be it, but you will have to supply a compliant device, not force my device to comply. After you do supply such a device, it's "your device, your rules" and I have no problem respecting that. If you on the other hand want to save money and force my device into compliance/non-freedom, me not accepting this breach is your problem as a bank/gov, not mine.
This is not unprecedented; in fact banks used to supply electronic code generation devices in order to provide 2FA for their web banking, and for some reason I can't understand no bank uses standardized 2FA schemes like TOTP/HOTP in order to at least be able to use open source, standards-compliant authenticators. Now we're forced to use their crappy apps (that don't work on rooted devices, which is necessary to keep an old device alive) just to receive 6 digits.
Security for who?
That’s the right question. Remote attestation isn’t great when it’s used to limit what users can do with their devices but it can also be something that is used in favour of users.
An example (the only one I know of) of that is Apple’s Private Cloud Compute where users can know for sure what’s running on the server side.