Hacker News

points by wtetzner 13 hours ago | 0 comments

Even if he was wrong about those statements, exploring other types of models isn't necessarily a bad thing. Especially if we can find ones that perform well with less demanding hardware requirements.

[–] dragonwriter 11 hours ago

Its not a bad thing, but exploratory research as opposed to reaearch that finetunes along a path already paved by exploratory research is something that doesn’t usually happen much in for-profit product-oriented firms when there is such a path available; it happens in academia and corporate labs pick it up once it reaches the point where the basics show commercial promise. CS is a little bit of an exception because the capital cost of basic research isn’t, proportionally, as high, and because the promise-to-product path is often short enough that having done the basic work in house can be a meaningful advantage in getting to market first (though, see Google on much of modern AI for a counterexample).

But even in computing, when there is a promising way forward that the basic reaearch has been done on, its hard to justify side quests.

[–] bigfatkitten 10 hours ago

Those exploratory "side quests" in industry are how we got the transistor, the liquid crystal display, Shannon's information theory, the laser and practical photovoltaic cells, among many other foundational technologies upon which we rely today.

[–] publlus_enigma 7 hours ago

Blue LEDs too, which have transformed lighting and displays.

[–] catlifeonmars 7 hours ago

Laser printers were a side project that almost got the inventor fired.

[–] bitwize 6 hours ago

And then they made back 200x the operating expenses of the lab (Xerox PARC) they came from over its entire lifetime.

[–] WalterBright 9 hours ago

Jet engines, too.

[–] RetroTechie 10 hours ago

> and corporate labs pick it up once it reaches the point where the basics show commercial promise.

How does that work when the research team moved elsewhere? Re-hire talent? That takes time - the 1 thing you don't have in a fast-moving field like AI. Post sums it up nicely:

"Do the math—how much is it worth to halve training and inference cost given that the industry has investments approaching a trillion dollars?"

How much is these researchers' salaries, compared to what MSFT spends on compute? Chances are it's peanuts in comparison.

[–] lovich 6 hours ago

Yes but you forget that spending on compute is spending on machines that the company owns while spending on researchers is spending on other apes that trigger the social hierarchies of executives brains and who have the dreaded key word of “rights”

Companies will try to ruthlessly rid themselves of labor as long as those members of labor have agency and rights.

If someone disagrees they can convince me by explaining why a company wouldn’t choose to spend X on a machine they wholly own vs X on an employee they don’t.

There’s some room for argument that employees can do more than machines can but in the current AI zeitgeist arguing that we have machines with equivalent to human intelligence I don’t see how you wouldn’t see this coming if you believed the zeitgeist.

[–] pyrale 10 hours ago

> CS is a little bit of an exception because the capital cost of basic research isn’t, proportionally, as high, and because the promise-to-product path is often short enough that having done the basic work in house can be a meaningful advantage in getting to market first (though, see Google on much of modern AI for a counterexample)

...And because they have monopoly money. Basically, research labs are usually vanity spending for corporations with too much money. Xerox parc was funded by a monopoly. Bell labs was funded by a monopoly. Google funds its moonshots with its monopoly.

[–] robotresearcher 8 hours ago

Bell and Xerox became powerful because they invented game-changing technologies. They were fundamentally research-based organizations from the start.

[–] vlovich123 9 hours ago

Remind me again where the transformer comes from?

[–] eli_gottlieb 7 hours ago

> it happens in academia and corporate labs pick it up once it reaches the point where the basics show commercial promise.

Ah, well, good thing we've almost entirely defunded academic science.

[–] dfdydx 11 hours ago

It’s a simple exploration/exploitation trade off. Always explore, never exploit is probably not the optimal strategy every time.

[–] bmitc 11 hours ago

It seems to me that everyone is in "always exploit" mode.

Seems weird when it's become apparent that LLMs aren't really special sauce outside of just having the compute.

[–] gscott 10 hours ago

The right idea usually comes from a wrong one.

[–] frays 12 hours ago

The issue isn't the exploration itself, it's the opportunity cost. When a figure of Yann LeCun's influence pushes a specific direction, it pulls key talent and resources away from mainstream LLM work. That kind of organizational distraction has a massive cost, and would've stopped Meta from delivering a polished product like Muse sooner.

[–] thatsabadlook 12 hours ago

The mainstream is wrong and in a lot of ways hazardous. He had every right to say he wasn't willing to sign is name on the wrong technology when he new better. When you hire smart people you listen to them. Elsewise you get what you deserve.

[–] frozenseven 11 hours ago

>when he new better

He was against LLMs, reinforcement learning, etc. Claiming that those are a dead-end. He was wrong. His own "JEPA" ideas have basically gone nowhere. Meta was right in moving on without him.

[–] ForHackernews 9 hours ago

I'm not sure he was wrong. No one has solved the fundamental problem with LLMs, which is that they cannot be trusted.

[–] jack_pp 3 hours ago

Most humans can not be trusted either, that's why you have processes in place to test and review work

[–] johnnyanmac 53 minutes ago

Sure. And we made efforts over millenia to ensure that.

Let me know when LLM's get those. Otherwise I trust it as much as a human under anarchic rule. Probably less because I know exactly who controls it.

[–] frozenseven 8 hours ago

Ok, so we're talking about applying connectionist systems/models to the messiness of the real world. And "trust" isn't an on-off switch. The goal is to build systems that are smarter and more general. It's a process. I don't think even Yann LeCun has made any strong claims beyond that. He just thinks that his way of doing deep learning is better.

[–] esseph 12 hours ago

It's not a distraction if the research pans out and then they become the market leader for the next several years or more.

[–] siva7 11 hours ago

The job of the CEO is to assess the research risk correctly. Zuck made the right bet here to replace him.

[–] hvb2 11 hours ago

The guy that named his company after his own diversion?

It's his call for sure, but rather funny as an argument.

[–] hikarudo 10 hours ago

The fact that someone made a mistake before doesn't mean that they will always make wrong calls.

[–] ImPostingOnHN 9 hours ago

not as a matter of certainty, but it certainly increases the bayesian prior probability of their calls being wrong

[–] thelastgallon 10 hours ago

Maybe he is keeping the name as a constant reminder of the mistaken. Like Berkshire Hathaway: Warren Buffett says Berkshire Hathaway is the “dumbest” stock he ever bought. https://www.cnbc.com/2010/10/18/warren-buffett-buying-berksh...

[–] shadowpho 12 hours ago

Ok yeah but it’s not a certain bet. Resources are limited and you gotta allocate them efficiently.

10% chance to be market leader doesn’t sound as good as 60% second place to them for example

[–] esseph 11 hours ago

This is a prime example of why large companies have a problem innovating.

Once you get close to the top, the risks don't look worth it.

[–] geertj 9 hours ago

> It's not a distraction if the research pans out

It is because even if it would pan out, it would mean temporarily losing AI leadership, reducing stock price, losing the ability to hire talent, and by virtue of that losing the opportunity permanently.

[–] gamblor956 7 hours ago

LeCun wasn't wrong. He simply had higher standards for results and accountability that LLMs are, by design, incapable of reaching.

Zuckerberg wanted to placate his shareholders with a shiny new thing now.

would've stopped Meta from delivering a polished product like Muse sooner.

Muse is not polished, unless usability and programming standards have fallen so far in the LLM vibe coding era that a pre-alpha state is now considered polished.

[–] teruakohatu 5 hours ago

You will find brilliant scientists in every discipline who are not the least bit interested in a particular branch of research because they/their lab didn’t invent it. That’s fine for public research but not helpful private research.

[–] solyin 6 hours ago

Compared to the user buying a Mac mini and following a tutorial off Twitter to install some random shit they found online?

Muse is imperfect, as is all software, but calling it pre-alpha is a bit much.