Hacker News

points by roncesvalles 2 days ago | 0 comments

Exactly. If you're an expert on the topic, framework, or library, even the best LLMs are still underwhelming. Even for 15 line snippets, if you're very familiar with the programming language, you'd have written it differently.

An LLM is like a low IQ person with vast knowledge.

[–] johnnudd 2 days ago

I have a PhD and Opus 5.5 came up with a novel solution to solve a problem in my area of research that I, and other experts in the field, had been unable to. This was from very general prompting it to find improvements over published methods.

[–] suddenlybananas 2 days ago

What was it?

[–] johnnudd 1 day ago

It came up with a way to efficiently evaluate the reliability of certain types of complex system using generating functions (https://en.wikipedia.org/wiki/Generating_function) to split the computation, which was far to large to solve with existing methods, into many embarrassingly parallel smaller problems and then reduce it back to a final result. The solution involved several distinct new ideas for it to work, really mind-blowing it was able to come up with it. The other cool thing was that, in the process and unprompted, it implemented code to validate and benchmark against existing approaches and known results just from the pdfs of the published papers (none of which gave reference implementations). In my research it would often take weeks just to replicate a single result from a paper.

[–] suddenlybananas 1 day ago

Sounds interesting but very vague!

[–] johnnudd 1 day ago

Perhaps but I have no incentive to write all the details here.

[–] aquentinn 1 day ago

I mean breaking a big problem into many small ones is hardly novel. It's how every business works.

[–] johnnudd 21 hours ago

By that logic modern cryptography is hardly novel because encryption is just mixing things up which is how every salad works.

[–] abc42 2 days ago

>If you're an expert on the topic, framework, or library, even the best LLMs are still underwhelming.

How many people on this planet are experts on every topic?

>An LLM is like a low IQ person with vast knowledge.

Are you aware that frontier models have scored in the 135-195 IQ range when they have been tested? How smart are you?

[–] AngryData 1 day ago

Humans test higher on IQ tests if they had previously taken IQ tests of similar types too tho. And LLMs are trained on IQ tests. IQ tests only really hold their value when the test is fairly novel to everyone taking it.

[–] abc42 1 day ago

Okay. So what does it mean when somebody says that LLMs are "low IQ"?

[–] AngryData 17 hours ago

That if you gave them a test far outside their trained materials that it will do terrible compared to a human.

[–] abc42 17 hours ago

How do humans usually score if you test them far outside their trained material?

[–] solyin 13 hours ago

Poorly, which is why IQ tests are sometimes regarded as racist. Say the test asks a question about a regatta, and your culture has never introduced you to that word, so you fail that question, dropping your score by 20 points. Is that then a fair assessment of your IQ? Another place we see this is math ability. Street kids with no formal training in math may not be able to do traditional word problems, because they've never ridden, eg a train, but when those word problems are phrased in terms they are forced to use to survive, eg the prices of the trinkets they're forced to sell, then they excel at multiplying numbers in their head.

[–] Personability 1 day ago

Said tests are often quite inflated, quite frankly. If you can give me a source that disproves this claim, I'll readily eat my words, but I readily believe that those tests are just as inflated as all other claims about AI.

[–] robrenaud 2 days ago

Does this gel with top mathematicans basically begging OpenAI to stop solving their open problems?

[–] lionkor 2 days ago

They're solving them in a way that isn't particularly interesting because it's near impossible to read and understand, and follow, what the LLM did. It's this way not because it's so smart, but because it's so stupid and still reached the goal by pure force and effort.

[–] QuesnayJr 2 days ago

I don't know how to communicate this, but what you are saying is almost entirely wrong.

People can follow what they did. The documents OpenAi produced in their big drop are hard to read because they are poorly written in weird ways, not because they are beyond human intelligence. They didn't brute force in the normal sense. AI doesn't get tired, so it can try 1000 promising ideas, but it's like 1000 Ph.D.-level mathematicians trying different ideas, not 1000 calculators trying all combinations.

The biggest results -- quasi-Riemann, no Siegel zeros, and Unique Games Conjecture -- have already been digested by experts. No Siegel zeros in particular already has like five different write-ups, and Tao has written a exposition on his blog.

[–] rsfern 1 day ago

So did we get any new mathematics out of this, or just solutions to problems without new techniques or insights? I haven’t been able to keep up with this or read the writeups you mention, genuinely wondering since this seems to be a central concern of the math community

[–] QuesnayJr 1 day ago

It's only been a month and people aren't that fast, but I think yes. Analytic number theory and complexity theory were stuck on the lack of progress on certain questions, that is now unstuck.

[–] JPLeRouzic 2 days ago

I am sorry but your argument looks to contradict itself: They solve hard mathematical problems because they are stupid and use brute force? Then why a human skilled in mathematics guided them for many turns in their conversation?

If they were really stupid, a Prolog program, a genetic algorithm or a SAT solver would be enough to reach the goal. Clearly it was not the case.

[–] suddenlybananas 2 days ago

Can't you kind of think of them as a genetic algorithm that has a very good hypothesis generator? Humans could, in principle, help guide that (already very impressive) hypothesis generator to make better guesses to find the proof more easily.

[–] solyin 13 hours ago

Conversations with an ChatGPT with Terence Tao at the helm are a thing of beauty. An advanced practitioner prompting things beyond my understanding. Jarred Sumner's paper on the Riemann Hypothesis, however, was not that. No slight against him, but Jarred was a non-mathematician, and his prompting to improve the minimum fraction of zeta zeros by Claude amounted to "keep going" and "believe in yourself". Shit, anyone coulda done that in his position.

https://www.anthropic.com/research/riemann-zeta

My point is that it does matter who's driving the AI but it also matters the route. Just like F1 champion Max Verstappen is a ridiculously better driver than me at driving an F1 car around a track, if we're just driving a friend to the airport, it doesn't really matter if it's Max or me driving, so long as we don't crash.

[–] YeGoblynQueenne 1 day ago

>> If they were really stupid, a Prolog program, a genetic algorithm or a SAT solver would be enough to reach the goal. Clearly it was not the case.

"Clearly" as in "somebody tried that and failed" or "clearly" as in nobody ever thought to try it and now nobody will because there's no point?

[–] ccppurcell 2 days ago

I think it does gel. You're right that the comment you're replying to is weakened by that fact.

It gels because they solved about 700 problems out of 4000 attempts. And they threw a ton of resources at each. So "per problem" it's less than 1 in 5, "per token" it's probably not a great record. I mean it's astonishing that it's possible at all of course. But on average, even an expert with a top model will probably have an underwhelming response to the vast majority of prompts.

[–] internet_points 2 days ago

and I'm guessing it was with "tool calling" ie. they can write a bunch of Lean proofs and hit compile and basically brute-force a whole bunch of hypotheses quickly?

Ie. the LLM itself isn't as "smart" as the combination of LLM, Lean, looping, search and vast gpu/cpu-time.

[–] ogogmad 2 days ago

In my experience they write an informal proof first, check it without Lean, and only then turn it into Lean code. That last step sometimes takes them much longer than checking their proof without Lean, which is a problem humans have as well. What they're truly bad at, however, is explaining their proofs. Until they improve at that, we'll have to keep resorting to Lean to check whether they solved something correctly.

I'm less sure about the following, however: I think that while current LLMs are bad at explaining their own proofs, some models are great for discussing older, human-written proofs. For this use case, from best to worst, Gemini > Claude > GPT-Astra >>>>>>>>>>>> GPT-Sol.

[–] ccppurcell 22 hours ago

I definitely don't know but I got the impression it's the opposite. Based on the observations about Navier Stokes and the natural language proof not matching up with the lean proof and other things. They generate lean and convert it into natural language at the end. But really I could be wrong just giving anyone interested a lead to check out.

[–] rowanG077 1 day ago

You cannot brute force lean proofs of any complexity. The space is infinitely large.

[–] seba_dos1 2 days ago

A relentless low IQ person with broad textbook knowledge and no need to sleep, I guess?

[–] Ar-Curunir 2 days ago

Look cmon, this is pure cope. The problems it has solved in TCS are far from trivial, with plenty of smart people making their careers in trying to solve them. The techniques these LLMs are using are non-trivial, and not just brute force.

[–] LtWorf 2 days ago

You mean techniques like using the prompt of mathematicians in the training set and brute force a solution before they publish?

[–] Ar-Curunir 1 day ago

No, I mean the techniques in the proofs produced by these LLMs.

[–] jasomill 1 day ago

I'm more interested in the techniques used to generate these proofs, and how they can be improved to be as reliable at solving a broader class of problem than already formalized mathematical conjectures.