points by krona 3 days ago | 0 comments
Just go ask some actual kernel maintainers who are being spammed by frontier models daily.
> These are relatively simple statistics.
According to Greg KH 50% of the vulns are flat out wrong, or the patches don't fix the problem, or the scenario is impossible in the real world, or just isn't a security issue.
[–] cogman10 2 days ago
[–] evan_a_a 2 days ago
All the marketing from OpenAI and Anthropic about "we used the model to find X vulnerabilities in Y" without any technical depth whatsoever, as is customary in the security space, basically boils down to "big number scary. model fix."
I said in another comment that I await the technical writeup of the so-called new techniques that they aren't sharing for "safety reasons" but if history is anything to go on, they will never publish a writeup. They set out to get their headline and that's where the work stops.
[–] spacebanana7 3 days ago
To be fair that sounds like similar false positive rate to human bug reports, if not better. Although the agentic scale of these reports must consume huge amounts of time.
I have to wonder how many of these 1000 paths are simply dead drivers with security vulnerabilities [1]. Impossible to exploit because nobody uses it, but easy enough to juice up the numbers of "found" vulnerabilities.
[1] https://lwn.net/Articles/1068928/