Hacker News
points by londons_explore 1 day ago | 0 comments
Behind nearly every AI bot is some human who will see real benefit from scraping my site.

I just make sure all my pages are super lightweight to cache and serve from cloudflare and let the bots do what they like.

If I put something on the internet for all to see, I want bots to be able to see it too.

I've been on the other side of bot blocks - when I ask my LLM to buy me X and it comes back and says the only place it could buy from is some Bulgarian seller because everywhere else has anti-bot scripts, I make sure to give that Bulgarian site my business.

[–] robinpie 1 day ago
Big agree.
Hacker News