Hacker News

points by mrandish 6 hours ago | 0 comments

> there's a straight highway ahead

Which many credible, AI researchers believe is already starting up an increasingly steep scaling wall of diminishing returns that eventually ends at a brick wall.

[–] falcor84 6 hours ago

I've been hearing a lot of "credible AI researchers" talk about these diminishing returns of transformer-based architectures since 2023 (at the least), and I'm still waiting to see this promised plateau materialize, because I definitely can't see it in https://metr.org/time-horizons/ or in my personal experience.

[–] mrandish 4 hours ago

Assuming super-linear growth will continue based on the first 3 years of a new technology? Almost every physical, computational and economic system in history, from Dennard scaling in silicon to transonic aviation speed looked super-linear until it hit the inflection point in its eventual S-curve.

A Copernican/Lindy baseline suggests the growth rate in LLM intelligence will peak in around another 2-4 years and I'm leaning on the low side because while the growth of LLM intelligence per dollar and per month are still growing in absolute terms, the growth rate of both seems to already be falling as power-law constraints in compute and data bottlenecks take hold.