Who does Anubis actually stop?
9 points by zk
9 points by zk
So who did we stop?
The exact adversary Anubis targets defeats it trivially.
I don't believe this is true. Anubis' targets are not LLM agents and vibecoded programs. To my understanding, Anubis main aim is to block web scrapers used by LLM companies. These are bots that just go out and slurp anything they can find. Sure, they could be powered by agents or LLMs themselves, and sure there can be work arounds. But the vast majority of these scrapers are "dumb" and their creators don't really care about bypassing the minority of sites that implement Anubis --- and that's kind of the goal here, it's kind of just annoying enough to where it doesn't really matter to the owners of the scraper that these sites can't be scraped.
Yeah, Anubis probably works fine to prevent your server to be hammered to death from residential proxy using scrapers. A request from an individual agent is not much different from its user browsing your site themselves.
However this also means, Anubis is overdesigned for what it does. Requiring a single button press and then issuing a cookie for access provides the same protection. I implemented that in Apache and that works just fine without any need for complicated proof of work.
Yep. I actually also do care about preventing RAG access to my personal site and software, too, though. I manage that with basic useragent blocking, and with basic testing (opening up chatgpt and asking it to access my site) that works fine. Super custom setups might fall through the cracks, but. my goal is to block the lazy, not the malicious.
I mean, if I understand this right, Anubis is more or less intended as an anti-DoS mechanism, rather than "blocking AI agents from accessing it" (which, let's be honest, it can't reasonably do)?
On the question of "who does Anubis actually stop", though, it includes me when on mobile devices and on sites I'm not familiar with (so I don't have JS on).
Anubis does not filter bots, it rate-limits clients.
For a scraping operation leveraging botnets of very low-end devices (i.e. smart TVs), it doesn't seem far-fetched that Anubis is an issue for them. https://lobste.rs/s/kpaxih/update_on_scraper_situation touches on that a little.
If I'm not mistaken SmartTVs are used for proxies and are not actually doing the PoW themselves.