An adversarial coding test
62 points by m_eiman
62 points by m_eiman
Today I read a blog post from Jamf Threat Labs about this, posted two days ago. Seems like this spread a little bit.
I have set modelines 0 in my .vimrc file. I do not have "." in my PATH. Why does Microsoft re-invent every terrible security flaw possible?
"We only ran it after you said OK when we asked you if it was trusted, so technically it's your fault and not ours!" is the vibe I get from vscode here.
To be quite honest I feel like VS Code really isn't in the wrong here. This stuff is useful and how many teams make sure that people have working setups and proper configuration, and the question is quite clear: do you trust the authors?
If anything they're being quite loud when other tools are quite likely to just let you go for it. Though perhaps something like "type the name of the folder" would be a good extra step here to really internalize what's going on.
The sort of counterpositive of not having this functionality... if you end up saying "do test a thing go do npm run test" and you put your payload in there the target're still owned.
(Then the question of course becomes why put it in the tasks at all, and there it feels obvious: people will hit "trust" because they want their LSP and the like to kick in to browse the code... so perhaps I'm being too glib)
If a completely uninformative popup opens for every single project you open, it ceases to be meaningful. (But I agree other editors that do these operations with no prompts are much, much worse, and would never even consider using such an editor.)
A better design would be to clearly identify what those unsafe operations are in the alert, and to encourage projects to use declarative configuration instead of arbitrary scripts. For example, "hey, this project is trying to run the shell script .vscode/init-project.sh every time you open it. Would you like to: [ignore (default)] [view the script] [run it anyway]". Honestly probably "run it anyway" should be greyed out until opening the script to audit it.
Running an arbitrary script upon opening your project in the editor should be treated as an extremely sketchy behavior, and well-behaved projects would not do that but use declarative configuration instead. The editor should only actually run code in response to user commands that showed clear intent to execute, like a "run tests" or "open repl" button. Ideally shaming projects like rust-analyzer would at least reduce the risk of other tools making the same mistakes.
But I guess it's too much to ask the company that owns npm (hello post-install scripts!) to take security seriously.
For example, "hey, this project is trying to run the shell script .vscode/init-project.sh every time you open it. Would you like to: [ignore (default)] [view the script] [run it anyway]". Honestly probably "run it anyway" should be greyed out until opening the script to audit it.
This seems like the equivalent of those programs that make you scroll through all the contents of their terms of service before you hit 'accept', notionally in order to force people to read them, and I suspect it would be about as effective. Any security model that relies on human vigilance is broken, and the article says as much:
It is my belief that given enough time and chances we will just click a button.
Phishing training and vigilance can only do so much. One day we will be distracted or tired and click the wrong button at the wrong time.
Not to mention, after you're done reading the source code for the tasks.json scripts, you'll have read the entire project's source code as well as the source code of all of its dependencies (because in this example this was a coding exercise, which means the author was going to have to actually execute the code they just downloaded).
If you want to develop safely, actual sandboxing so that malicious software can't do anything bad is the only real answer.
Wait, your argument is that saying "hey, this repo wants to autoexecute stuff, do you want to allow it" is less secure than vim having a single global toggle for modelines?
I think their argument is that text editors should not be running arbitrary code, in vim you can disable the misfeature globally, in vscode you have to constantly click no
I've never used modelines personally, but I've made pretty heavy use of direnv files in my projects (though direnv will hash the file and refuse to execute it if it's changed unless you re-allow, whereas IIRC vscode will still execute a file even it changed).
Yeah it didn’t even occur to me that another editor would have added yet another “run arbitrary instructions automatically” file - I had assumed the vscode docs were just windows, maybe build config, but in retrospect the many things it wants to do should have made me assume the worst.
Vim modelines are pretty limited, though (and having a global off switch is also a good thing)
No, just that Microsoft seems to think it's exceptional in some sense, and ignores other's past mistakes, that's all. Neither modelines nor the VScode thing is particularly secure, and neither should have existed. Like ActiveX, which seemed to ignore lessons from NeXTMail's active content.
This is why the "are you really sure" prompts do not work.
Sure, the text literally contains "automatically executes files"; but that's not how people work with computers. They don't read text. When they see a big blue button on a modal prompt, they click the big blue button. That's it.
Designing a user experience like this literally screams "I did not think about this at all" - or worse "I am putting the blame on the user, because I don't know either".
Oh look it is autorun.inf again
I remember years ago placing autorun.inf directory so viruses on someone else's computer would not infect my pendrive too well. It was surprisingly effective, because malware authors were as lazy as the rest of us.
There were a two more directories and probably three files I also placed there, but I don't exactly remember now, something with recycler probably.
At first I thought this would be about a new kind of competitive coding test, and was not sure how to feel about that. I did not realize "adversarial" meant "malicious" here, but that's horrifying.