Hacker News

points by mcmcmc 2 days ago | 0 comments

Ok but there’s still a high error rate, so why would you trust it with general queries? Especially if it’s something you aren’t already knowledgeable about

[–] fesoliveira 2 days ago

That is fair. I always ask for sources whenever it is something I am unfamiliar with instead of just trusting the response, that way I can go and double check and it ensures the agent at least read something on the topic

[–] InvertedRhodium 2 days ago

I've found the value of this approach has diminished over time - now I need to go and read the source and try to identify if it was generated by an LLM itself.

[–] tokioyoyo 2 days ago

Why do you think there’s a high error rate for general queries nowadays?

[–] mcmcmc 2 days ago

Because of the high error rate on queries where I know the topic well enough to tell when the answer is way off base

[–] tokioyoyo 2 days ago

Yeah fair. I guess I have a very different experience, as I mentioned above, I basically never use it for any niche/hard queries that depend on multiple sources or research. For all my common queries, it's reliable.

[–] mcmcmc 2 days ago

> For all my common queries, it's reliable.

Is it though? How often are you spot checking sources?

[–] tokioyoyo 2 days ago

Sorry, not going to take the bait here. I would argue in 2024/2025 it was dogshit, but it really isn’t nowadays. You can even see the sources for most of the summary queries, like literally on the same line as the bullet point.

My google queries are like different combos of weather, specific event questions, directly attributed to wikipedia data and etc. LLMs are way past the point of reliability when it comes to data extraction from static websites.

[–] jdiff 2 days ago

> LLMs are way past the point of reliability when it comes to data extraction from static websites.

Two very modern counterexamples:

https://app.opencve.io/cve/CVE-2026-83844

https://www.sentinelone.com/vulnerability-database/cve-2026-...

The information they feed from is simple, clear, and static. And yet the both of them are hallucinating mitigations, compromise indicators, and other details of the vector that flatly don't exist.

The question GP posed wasn't bait. It's still a blatant problem in 2026. It's not just shitty cheap models, either. Just this morning I caught 6.1 Sol (via Codex) incorrectly answering a question about the codebase it was actively working in.