Hacker News

points by bananaquant 3 days ago | 0 comments

Good points. Unfortunately, they don't have the people nor the incentive to actually look into that.

Also, let's maybe not call the result impressive until we see what's under the hood.

>Unfortunately, they don't have the people nor the incentive to actually look into that.

These are relatively simple statistics. Some are even trivial. I don't see why it requires any people at all.

It wouldn't even have to be done by OpenAI. Given a list of vulnerabilities, it wouldn't be too hard for third parties to collect these statistics.

[–] pyrale 2 days ago

> These are relatively simple statistics. Some are even trivial. I don't see why it requires any people at all.

They're not here to help the community, they're here to get headlines, and, possibly, replace the community.

[–] krona 3 days ago

Just go ask some actual kernel maintainers who are being spammed by frontier models daily.

> These are relatively simple statistics.

According to Greg KH 50% of the vulns are flat out wrong, or the patches don't fix the problem, or the scenario is impossible in the real world, or just isn't a security issue.

[–] cogman10 2 days ago

The kernel has also been removing a bunch of old unmaintained drivers due to these reports.

I have to wonder how many of these 1000 paths are simply dead drivers with security vulnerabilities [1]. Impossible to exploit because nobody uses it, but easy enough to juice up the numbers of "found" vulnerabilities.

[1] https://lwn.net/Articles/1068928/

[–] evan_a_a 2 days ago

All the marketing from OpenAI and Anthropic about "we used the model to find X vulnerabilities in Y" without any technical depth whatsoever, as is customary in the security space, basically boils down to "big number scary. model fix."

I said in another comment that I await the technical writeup of the so-called new techniques that they aren't sharing for "safety reasons" but if history is anything to go on, they will never publish a writeup. They set out to get their headline and that's where the work stops.

[–] spacebanana7 3 days ago

To be fair that sounds like similar false positive rate to human bug reports, if not better. Although the agentic scale of these reports must consume huge amounts of time.

[–] embedding-shape 3 days ago

> don't see why it requires any people at all. It wouldn't even have to be done by OpenAI. Given a list of vulnerabilities

Who would put together that list unless individuals at OpenAI does it? It's not like we have machines that can think kind of like humans and do a bunch of stuff automagically.

[–] jon-wood 3 days ago

If only OpenAI had some sort of cluster of thinking machines they could apply to generating such a report.

[–] bananaquant 3 days ago

Indeed. Yet instead they generate a report that is 75% opinions on how everyone else should spend their money.