points by jmull 10 hours ago | 0 comments
What makes sense is to do both.
Race down the transformer highway as fast as possible and explore other AI methods.
One need not preclude the other.
[–] knollimar 10 hours ago
[–] Fordec 7 hours ago
Also in an exponential competition environment, the difference between 100% commitment and 98% commitment is very possibly not even the difference between a win and second place, but outright lapping by the winner and humiliation at the scale of the gap for second place.
For an example of what this looks like at a more pronounced scale just look at the discourse around European models and how far behind Minstral are perceived to be.
[–] fallingbananna 10 hours ago
My understanding is that the resources are limited and even AI labs with all the funding in the world don't have enough hardware.
[–] caminante 9 hours ago
Except doing both..is..so..hard when the future payoff of one is more unclear and you are faced with quarterly earnings reports.
Look at what happened to Meta where they squandered a lead, had some mixed results, and reorged to more product certainty and away from fundamental research.
[–] joe_the_user 7 hours ago
Theoretically one need doesn't preclude the other. The OP gives a detailed argument and his own experience of one need precluding the other in practice.
Both of these approaches require quite a bit of institutional support notably.
[–] lanthissa 6 hours ago
it literally does when gpu's are scarce.
Going close to as fast as possible while saving a stream for the other approaches makes sense as an approach, but let's not ignore that there's a compromise/tradeoff here. Might not matter in orgs with near infinite resources