I guess Anubis also prevents “good” bots such as search engine bots to access a webpage…
You must log in or register to comment.
18 daysyes but you can configure anubis to allow specific bots with botPolicy.yaml; the allow-good include already whitelists internet archive.
look in the crawlers directory and you can add any allows you want with the proper user agent. :)
- 17 days
Nah, that was a joke; Dillo is basicslly a graphical Lynx (both have useragent-exceptions on some big “js-only” sites). And the Anubis dev is, for a while now, working on a static version, if you believe the error message. Though that would be hard to do, if at all possible.
- 18 days
Don’t forget “intelligence”-as-a-service™®©!
Hi ChatGPT, I can’t find a job due to the AI bubble. What should I do?
