Hacker News
points by famouswaffles 1 day ago | 0 comments
OpenAI sat on GPT-4 for 8 months before actually releasing it and that was a major stride over 3. It's kind of silly not to realize they do indeed have access to models months ahead of the public at minimum. Nothing you have access to is the true frontier. It only takes some common sense to acknowledge that. It's not unbelieveable, you just don't want to believe it.
[–] bigstrat2003 1 day ago
No it wasn't. GPT-4 was indistinguishable from 3.5.
[–] troupo 1 day ago
> OpenAI sat on GPT-4 for 8 months before actually releasing it and that was a major stride over 3.

OpenAI said they sat on it for 8 months

> It's kind of silly not to realize they do indeed have access to models months ahead of the public at minimum

Yes, they do. Nothing in this sentence implies that what they have access to is "light years ahead of the public models".

[–] famouswaffles 1 day ago
>OpenAI said they sat on it for 8 months

They sat in it. We had concrete details ('rumors' that turned out surprisingly accurate) months before the public release.

>Yes, they do. Nothing in this sentence implies that what they have access to is "light years ahead of the public models".

"Light years" is subjective, but the accomplishments of those internal models we're priven to should be enough to give you some indication. You can fire up as many Astra agents as you like you're not going to get the performance of Open AI's internal model in math. Just as you can fire as many GPT-4 instances and you're not going to get anything resembling Opus 5.5.

[–] troupo 1 day ago
> They sat in it. We had concrete details ('rumors' that turned out surprisingly accurate) months before the public release.

So all you had was rumours then "confirmed" by OpenAI marketing

> "Light years" is subjective

Yes. And this subjective description invokes a certain image that I objected to.

> but the accomplishments of those internal models we're proven to should be enough to give you some indication.

Yes. The indication is: there are some improvements, with diminishing returns [1]. A lot of perceived improvements now come from tooling, harnesses etc.

And the people working at the labs have access to unrestricted models and unlimited compute that will not flag you for "bio research" and will not stop running after 5 hours, which adds to the marketing myths.

> You can fire up as many Astra agents as you like you're not going to get the performance of Open AI's internal model in math.

Of course I won't. I don't have the same access.

Doesn't mean any of their models are light years ahead. And as for maths, again, you literally have no idea what they were actually doing besides their marketing materials. But even their marketing materials let slip some details that amount to "commandeered a full cluster for 20 000 hours and let it rip".

Edit: Moreover. We know they are not light years ahead, or substantially better than what users want. While they let some of their researchers do cool shit with unlimited resources, you can see the rest of what these labs produce: from their websites, to their harnesses, to their apps. Buggy incoherent mess. Doesn't scream "light years ahead" to me.

[1] Subjectively the models haven't moved an inch since February this year, and regressed in quite a few areas (code duplication is worse, Claude-ish permeates everything, context compaction is even more random and training quickly overrides context etc.)