points by NaiveEmpiricist 1 day ago | 0 comments
Indeed. Who'd have thought that mere statistics could do that? Hat's off to them - they made the darn thing work.
There's pretty interesting (I think) research in how effectively it's possible to scale down LLMs. If you're interested, take a look at the BabyLM conferences:
(I have no connection to them.) The work there demonstrates that gap between what we can do now with human-relevant amounts of training text, and what remains to be discovered. Quite a bit, it looks like.
I mean shifting influence of gravity relative to an object with enough mass type ratios are everywhere
The language of math started as visual art to depict such ratios; in modern times this ground truth associations is frequently overlooked or never understood to exist in the first place without engagement
Differential diagnosis in medicine is exactly the same process; such processes and their successes are everywhere.
The issue we have in software is the outcome isn't as obvious as jabbing a blood draw needle in a patients eye; we spent a lot of time wasting time on illusory needs