Hacker News

points by OJFord 2 days ago | 0 comments

That's not really true any more, since given tool use and iterative 'chain of thought' reasoning, it's more like an answer refined by conversation with the asker or the second best answerer, from the best answerer, is the best that you can expect.

E.g. consider the best answer describes how to achieve something with some CLI, but the CLI's changed and the command is now broken. It can discover that, combine with the 'best answer' about what changed in the CLI, and fix it to work for the original problem even if that's completely novel and there were no extant answers about solving the problem with the new CLI form, or that it had broken that use case.

It doesn't change fundamentally how they work of course, but 'thinking', creating that feedback loop, made them significantly more useful IMO.

[–] vrighter 1 day ago

yeah chain of thought reasoning isnt really a think besides running the model for longer and hide most of the output (but still charging you for it). it literally does not work any differently than "non-reasoning" models (except for marketing)

[–] PaulRobinson 2 days ago

The “conversation” is just one between some redditors or book publishers.

It’s going to narrow in on a statistically better answer but it’s not going to create novel insight beyond the arguments made from compressed snapshots of existing knowledge.

There is no “thinking”. Just more attention, which is apparently, all you need.

[–] yorwba 2 days ago

All novel insights expressible as text can be assembled from snippets of existing text. The challenge is "only" that you need to find the right snippets. Hence being a stochastic parrot does not represent an upper bound on capabilities, but a suggestion that if you want your parrot to give you a specific answer, you should arrange for a context where it's statistically likely to be correct.

[–] discreteevent 2 days ago

Are you saying that a complex stateful system cannot generate a state that is not already recorded in text somewhere?

[–] yorwba 1 day ago

No. In a large enough state space, you can of course easily find a state that has never existed before. For example, if you pick a 1024-bit integer uniformly at random, it will certainly be different from any previous 1024-bit integer. However, it's much more difficult to find a new integer that cannot be assembled out of bits taken from previous integers. Composition from a handful of base atoms can cover very large state spaces, and unbounded composition can cover infinite state spaces. Which is why only being able to combine things that already exist is not much of a limitation.

To demonstrate:

"Are you saying that a" https://news.ycombinator.com/item?id=49642931

"complex stateful system" https://news.ycombinator.com/item?id=14881439

"cannot generate a" https://news.ycombinator.com/item?id=45956162

"state that is not already" https://news.ycombinator.com/item?id=43013637

"recorded in text" https://news.ycombinator.com/item?id=44731577

"somewhere?" https://news.ycombinator.com/item?id=50391679

A novel state, obtainable by combining six existing ones.

[–] serf 2 days ago

>It’s going to narrow in on a statistically better answer but it’s not going to create novel insight beyond the arguments made from compressed snapshots of existing knowledge.

ignoring the fundamentally dishonest representation of LLMs in this subthread..

if humanity was given a tool that could restructure already existing and known knowledge and facts into novel results with a statistically better answer automatically it wouldn't be as a minor a win as you make it sound; it would be tremendous.

[–] ogogmad 2 days ago

That's not what "attention" means.

> it’s not going to create novel insight beyond the arguments made from compressed snapshots of existing knowledge

So all of the maths breakthroughs were not novel?

[–] PaulRobinson 2 days ago

Firstly, many of them are still being checked, so hold your horses.

What I’ve found LLMs to be OK at when it comes to novel insight is being given a goal to work towards (“prove or disprove this”), with access to a large body of sources and it can then chase down 1000s of options no human would bother with.

This is why security and maths proofs seem to be an area where it can accelerate things. However I know from my own work in these areas it can still hallucinate and so the output needs careful checking.

Is that novel ideation? Or is it self-organising grid search? It’s more efficient than getting 1000 mathematicians to do the work, but it’s not doing something “new”, it’s building on existing work.

And yes, I know what “attention” means, I was making a pun. And it still stands up: they’re not in some deep thought about this and reflecting on new ideas, they’re just hammering away at everything in the literature.

And it still might be bullshit.