Hacker News
points by sparkling 3 days ago | 0 comments
>absurd amount of effort every website is pouring into anti-scraping tech.

Maybe some people do not want to provide their entire website as free training material to [insert LLM lab here]?

[–] ethin 3 days ago
You... Do know that people scrape websites for non-AI reasons, right?
[–] anonreplier 3 days ago
Why should the site owners be interested in allowing them to do that?
[–] ethin 3 days ago
Because it's legal to scrape websites? And, oh, I don't know, maybe people like me might want to archive those websites for historical reasons?
[–] anonreplier 2 days ago
that's why you might want to do it, not why they would want to let you do it