Web Page Crawler Revealed - tracking down requests from Contabo GmbH

11 points by oceanhaiyang


The story is a technical breakdown attempting to trace a series of crawlers from Contabo GmbH. Edited the title to be more clear.

This feels relevant here but I struggled with tagging this. I welcome suggestions for how to tag it better!