Hacker News

points by liuliu 10 hours ago | 0 comments

100 trillion parameters doesn't make any sense if your dataset is ~30 trillion tokens. My uneducated guess is 1/10 compression ratio is good and that point to ~300 to ~1000 trillion tokens, which again, doesn't make any practical sense.

[–] sbierwagen 10 hours ago

Depending on how you count, the human brain is between 100 trillion and 1 quadrillion parameters, which is one guess as to why we're so much more sample efficient at learning than LLMs are.

[–] jurgenburgen 7 hours ago

Synapses and LLM parameters are not equivalent.

[–] cite_please 6 hours ago

In Silicon Valley they are.

[–] mschuster91 9 hours ago

One key difference between humans and (current) LLMs is that we can actually judge the material we learn from at the time we're learning it, contextualize it and discard garbage when we spot it.

LLMs need an awful lot of handholding all the time, meanwhile humans get pretty good at it on their own.

[–] Frost1x 9 hours ago

Unless quantum effects do infact play a role in consciousness. It’s not clear they do by any means, but if it did that could shift the understanding of our own minds and consciousness drastically. Otherwise we are just a dense complex network we can reproduce with enough money.

I mean we literally do this every few seconds already with natural birth processes.

[–] qlte 9 hours ago

I don't think you even need to go to hypothesized quantum stuff. A single biological neuron in isolation is a surprisingly complex system that resembles a small artificial neural net more than an artificial neuron in many ways. On top of an entire feedback system from diffusion of neurotransmitters which doesn't cleanly map to ANNs.

Trying to estimate equivalent "parameter size" of the brain when we only superficially understand behavior of individual neurons let alone how intelligence emerges from the connectome seems too speculative to be a particularly useful metric.

[–] slashdave 7 hours ago

The extra parameters are useful for optimization. Deep-learning relies on highly singular solutions.