Making LeCun report to Wang was the most boneheaded move imaginable. But… I suppose Zuckerberg knows what he wants, which is AI slopware and not truly groundbreaking foundation models.
In industry research, someone in a chief position like LeCun should know how to balance long-term research with short-term projects. However, for whatever reason, he consistently shows hostility toward LLMs and engineering projects, even though Llama and PyTorch are two of the most influential projects from Meta AI. His attitude doesn’t really match what is expected from a Chief position at a product company like Facebook. When Llama 4 got criticized, he distanced himself from the project, stating that he only leads FAIR and that the project falls under a different organization. That kind of attitude doesn’t seem suitable for the face of AI at the company. It's not a surprise that Zuck tried to demote him.
These are the types that want academic freedom in a cut-throat industry setup and conversely never fit into academia because their profiles and growth ambitions far exceed what an academic research lab can afford (barring some marquee names). It's an unfortunate paradox.
The Bell Labs we look back on was only the result of government intervention in the telecom monopoly. The 1956 consent decree forced Bell to license thousands of its patents, royalty free, to anyone who wanted to use them. Any patent not listed in the consent decree was to be licensed at "reasonable and nondiscriminatory rates."
The US government basically forced AT&T to use revenue from its monopoly to do fundamental research for the public good. Could the government do the same thing to our modern megacorps? Absolutely! Will it? I doubt it.
Used to be a Google X. Not sure at what scale it was.
But if any state/central bank was clever they would subsidize this.
That's a better trickle down strategy.
Until we get to agi and all new discoveries are autonomously led by AI that is :p
> Google X is a complete failure
- Google Brain
- Google Watch/Wear OS
- Gcam/Pixel Camera
- Insight (indoor GMaps)
- Waymo
- Verily
It is a moonshot factory after all, not a "we're only going to do things that are likely to succeed" factory. It's an internal startup space, which comes with high failure rates. But these successes seem pretty successful. Even the failed Google Glass seems to have led to learning, though they probably should have kept the team going considering the success of Meta Raybands and with things like Snap's glasses.
Didn't the current LLMs stem from this...? Or it might be Google Brain instead. For Google X, there is Waymo? I know a lot of stuff didn't pan out. This is expected. These were 'moonshots'.
But the principle is there. I think that when a company sits on a load of cash, that's what they should do. Either that or become a kind of alternative investments allocator. These are risky bets. But they should be incentivized to take those risks. From a fiscal policy standpoint for instance.
Well it probably is the case already via lower taxation of capital gains and so on.
But there should probably exist a more streamlined framework to make sure incentives are aligned.
And/or assigned government projects?
Besides implementing their Cloud infrastructure that is...
It seems DeepMind is the closest thing to a well funded blue-sky AI research group, even despite the merger with Google Brain and now more of a product focus.
Google Deepmind is the closest lab to that idea because Google is the only entity that is big enough to get close to the scale of AT&T. I was skeptical that the Deepmind and Google Brain merge would be successful but it seems to have worked surprisingly well. They are killing it with LLMs and image editing models. They are also backing the fastest growing cloud business in the world and collecting Nobel prizes along the way.
I thought that was Google. Regulators pretend not to notice their monopoly, they probably get large government contracts for social engineering and surveillance laundered through advertising, and the “don’t be evil” part is they make some open source contributions
I'd argue SSI and Thinking Machines Lab seem to that environment you are thinking about. Industry labs that focuses on research without immediate product requirement.
I don't think that quite matches because those labs have very clear directions of research in LLMs. The theming is a bit more constrained and I don't know if a line of research as vague as what LeCun is pursuing would be funded by those labs.
> A pipe dream sustaining the biggest stock market bubble in history
This is why we're losing innovation.
Look at electric cars, batteries, solar panels, rare earths and many more. Bubble or struggle for survival? Right, because if US has no AI the world will have no AI? That's the real bubble - being stuck in an ancient world view.
Meta's stock has already tanked for "over" investing in AI. Bubble, where?
> 2 Trillion dollars in Capex to get code generators with hallucinations
You assume that's the only use of it.
And are people not using these code generators?
Is this an issue with a lost generation that forgot what Capex is? We've moved from Capex to Opex and now the notion is lost, is it? You can hire an army of software developers but can't build hardware.
Is it better when everyone buys DeepSeek or a non-US version? Well then you don't need to spend Capex but you won't have revenue either.
And that $2T you're referring to includes infrastructure like energy, data centers, servers and many things. DeepSeek rents from others. Someone is paying.
Man, why did no one tell the people who invented bronze that they weren’t allowed to do it until they had a correct definition for metals and understood how they worked? I guess the person saying something can’t be done should stay out of the way of the people doing it.
>> I guess the person saying something can’t be done should stay out of the way of the people doing it.
I'll happily step out of the way once someone simply tells me what it is you're trying to accomplish. Until you can actually define it, you can't do "it".
The big tech companies are trying to make machines that replace all human labor. They call it artificial intelligence. Feel free to argue about definitions.
I'm not sure what 'inventing bronze' is supposed to be. 'Inventing' AGI is pretty much equivalent to creating new life, from scratch. And we don't have an idea on how to do that either, or how life came to be.
Intelligence and human health can't be defined neatly. They are what we call suitcase words. If there exists a physiological tradeoff between medical research about whether to live till 500 years or to be able to lift 1000kg when a person is in youth, those are different dimensions / directions across we can make progress. Same happens for intelligence. I think we are on right track.
I don't think the bar exam is scientifically designed to measure intelligence so that was an odd example. Citing the bar exam is like saying it passes the "Game of thrones trivia" exam so it must be intelligent.
As for IQ tests and the like, to the extent they are "scientific" they are designed based on empirical observations of humans. It is not designed to measure the intelligence of a statistical system containing a compressed version of the internet.
Or does this just prove lawyers are artificially intelligent?
yes, a glib response, but think about it: we define an intelligence test for humans, which by definition is an artificial construct. If we then get a computer to do well on the test we haven't proved it's on par with human intelligence, just that both meet some of the markers that the test makers are using as rough proxies for human intelligence. Maybe this helps signal or judge if AI is a useful tool for specific problems, but it doesn't mean AGI
Hi there! :) Just wanted to gently flag that one of the terms (beginning with the letter "r") in your comment isn't really aligned with the kind of inclusive language we try to encourage across the community. Totally understand it was likely unintentional - happens to all of us! Going forward, it'd be great to keep things phrased in a way that ensures everyone feels welcome and respected. Thanks so much for taking the time to share your thoughts here!
I became interested in the matter reading this thread and vaguely remember reading a couple of the articles. Saved them all in NotebookLM to get an audio overview and to read later. Thanks!
I always take a bird's eye kind of view on things like that, because however close I get, it always loops around to make no sense.
> is massively monopolistic and have unbounded discretionary research budget
that is the case for most megacorps. if you look at all the financial instruments.
modern monopolies are not equal to single corporation domination. modern monopolies are portfolios who do business using the same methods and strategies.
the problem is that private interests strive mostly for control, not money or progress. if they have to spend a lot of money to stay in control of (their (share of the)) segments, they will do that, which is why stuff like the current graph of investments of, by and for AI companies and the industries works.
A modern equivalent and "breadth" of a Bell Labs (et. al) kind of R&D speed could not be controlled and would 100% result in actual Artificial Intelligence vs all those white labelababbebel (sry) AI toys we get now.
Post WW I and II "business psychology" have build a culture that cannot thrive in a free world (free as in undisturbed and left to all devices available) for a variety of reasons, but mostly because of elements with a medieval/dark-age kind of aggressive tendency to come to power and maintain it that way.
In other words: not having a Bell Labs kind of setup anymore ensures that the variety of approaches taken on large scales aka industry-wide or systemic, remains narrow enough.
More importantly even if you do want it, and there are business situations that support your ambitions. You still have to do get into the managerial powerplay, which quite honestly takes a separate kind of skill set, time and effort. Which Im guessing the academia oriented people aren't willing to do.
Its pretty much dog eat dog at top management positions.
Its not exactly a space for free thinking timelines.
It is not a free thinking paradise in academia either. Different groups fighting for hiring, promotions and influence exist there, too. And it tends to be more pronounced: it is much easier in industry to find a comparable job to escape a toxic environment, so a lot of problems in academia settings steam forever.
But the skill sets to avoid and survive personnel issues in academia is different from industry. My 2c.
> Its not exactly a space for free thinking timelines.
Same goes for academia. People's visions compete for other people's financial budgets, time and other resources. Some dogs get to eat, study, train at the frontier and with top tools in top environments while the others hope to find a good enough shelter.
as I understand, Bell Labs mandate was to improve the network, which had tons of great threads to pull on: plastics for handsets, transistors for amplification, information theory for capacity on fixed copper.
Google and Meta are ads businesses with a lot less surface area for such a mandate to have similar impact and, frankly, exciting projects people want to do.
Meanwhile they still have tons of cash so, why not, throw money at solving Atari or other shiny programs.
Also, for cultural reasons, there’s been a huge shift to expensive monolithic “moonshot programs” whose expenses need on-demand progress to justify and are simply slower and way less innovative.
3 passionate designers hiding deep inside Apple can side hustle up the key gestures that make multi touch baked enough to see a path to an iPhone - long before iPhone was any sort endgame direction they were being managed to.
Innovation thrives on lots of small teams mostly failing in the search for something worth doubling down on.
Googles et al have a new approach - aim for the moon, budget and staff for the moon, then burn cash while no one ever really polished up the fundamental enabling pieces in hindsight they needed to succeed
I would pose a question differently, under his leadership did Meta achieve good outcome?
If the answer is yes, then better to keep him, because he has already proved himself and you can win in the long-term. With Meta's pockets, you can always create a new department specifically for short-term projects.
If the answer is no, then nothing to discuss here.
Meta did exactly that, kept him but reduced his scope. Did the broader research community benefit from his research? Absolutely. But did Meta achieve a good outcome? Probably not.
If you follow LeCun on social media, you can see that the way FAIR’s results are assessed is very narrow-minded and still follows the academic mindset. He mentioned that his research is evaluated by: "Research evaluation is a difficult task because the product impact may occur years (sometimes decades) after the work. For that reason, evaluation must often rely on the collective opinion of the research community through proxies such as publications, citations, invited talks, awards, etc."
But as an industry researcher, he should know how his research fits with the company vision and be able to assess that easily. If the company's vision is to be the leader in AI, then as of now, he seems to have failed that objective, even though he has been at Meta for more than 10 years.
Also he always sounds like "I know this will not work". Dude are you a researcher? You're supposed to experiment and follow the results. That's what separates you from oracles and freaking philosophers or whatever.
If academia is in question, then so are their titles.
When I see "PhD", I read "we decided that he was at least good enough for the cause" PhD, or PhD (he fulfilled the criteria).
He's speaking to the entire feedforward Transformer-based paradigm. He sees little point in continuing to try to squeeze more blood out of that stone and instead move on to more appropriate ways to model ontologies per se rather than the crude-for-what-we-use-them-for embedding-based methods that are popular today.
I really resonate with his view due to my background in physics and information theory. I for one welcome his new experimentation in other realms while so many still hack away at their LLMs in pursuit of SOTA benchmarks.
If the LLM hype doesn't cool down fast, we're probably looking at another AI winter. Appears to me like he's just trying to ensure he'll have funding for chasing the global maximum going forward.
> If the LLM hype doesn't cool down fast, we're probably looking at another AI winter.
Is the real bubble ignorance? Maybe you'll cool down but the rest of the world? There will just be more DeepSeek and more advances until the US loses its standing.
Yeah that stuff generated embarrassingly wrong scientific 'facts' and citations.
That kind of hallucination is somewhat acceptable for something marketed as a chatbot, less so for an assistant helping you with scientific knowledge and research.
I thought it was weird at the time how much hate Galactica got for its hallucinations compared to hallucinations of competing models. I get your point and it partially explains things. But it's not a fully satisfying explanation.
Meta had a two prong AI approach - product-focused group working on LLMs, and blue-sky research (FAIR) working on alternate approaches, such as LeCun's JEPA.
It seems they've given up on the research and are now doubling down on LLMs.
None of Meta's revenue has anything to do with AI at all. (Other than GenAI slop in old people's feeds.) Meta is in the strange position of investing very heavily in multiple fields where they have no successful product: VR, hardware devices, and now AI. Ad revenue funds it all.
LeCun truly believes the future is in world models. He’s not alone. Good for him to now be in the position he’s always wanted and hopefully prove out what he constantly talks about.
He seems stuck in the GOFAI development philosophy where they just decide humans have something called a "world model" because they said so, and then decide that if they just develop some random thing and call it a "world model" it'll create intelligence because it has the same name as the thing they made up.
And of course it doesn't work. Humans don't have world models. There's no such thing as a world model!
I do agree humans don't have a world model. It is really more than that. We exist in the world. We don't need a world model because we exist in the world.
It is like saying a fish has a water model. It makes no sense when the fish existence is intertwined with water.
That is not to say that a computer that has a model of the world would not most likely be extremely useful vs something like the LLM that has none. The world model would be the best we could do to create a machine that simulates being in the world.
I don't think the focus is really on world models, rather than on animal intelligence based around predicting the real world, but to predict it you need to model it in some sense.
IMO the issue is that animals can't have a specific "world model" system, because if you create a model ahead of time you will mostly waste energy because most of the model is not used.
And animals' main concern is energy conservation, so they must be doing something else.
There are many factors playing into "survival of the fittest", and energy conservation is only one. Animals build mental models to predict the world because this superpower of seeing into the future is critical to survival - predict where the water is in a drought, where the food is, and how to catch it, etc, etc.
The animal learns as it encounters learning signals - prediction failure - which is the only way to do it. Of course you need to learn/remember something before you can use that in the future, so in that sense it's "ahead of time", but the reason it's done that way because evolution has found that learning patterns will ultimately prove beneficial.
Right - I've no idea how LeCun thinks about it, but I don't see that an animal needs or would have any more of a "world model" than something like an LLM. I'm sure all the research into rats in mazes etc has something to say about their representations of location/etc, but given a goal of prediction it seems that all is needed is a combination of pattern recognition and sequence prediction - not an actual explicit "declarative" model.
It seems that things like place cells and grandmother cells are a part of the pattern recognition component, but recognizing landmarks and other predictive-relevant information doesn't mean we have a complete coherent model of the environments we experience - perhaps more likely a fragmented one of task-relevant memories. It seems like our subjective experience of driving is informative - we don't have a mental road map but rather familiarity with specific routes and landmarks. We know to turn right at the gas station, etc.
LLM hostility was warrented. The overhype/downright charlartan nature of ai hype and marketing threatens another AI winter. It happened to cybernetics, it'll happen to us too. The finance folks will be fine, they'll move to the next big thing to overhype, it is the researchers who suffer the fall-out. I am considered anti LLM (transformers anyway) for this reason, i like the the architecture, it is cool amd rather capable at its problem set, which is a unique set, but, it isnt going to deliver any of what has been promised, any more than a plain DNN or a CNN will.
Meta is in last place among the big tech companies making an AI push because of lecun’s llm hostility. Refusing to properly invest in the biggest product breakthrough this century was not even a little bit warranted. He had more than enough resources available to do the research he wanted and create a fantastic open source llm.
Meta has made some fantastic llm's publically avliable many of which continue to outperform all but the qwen series in real world applications.
LLMs cannot do any of the major claims made for them, so competing at the current frontier is a massive resource waste.
Right now a locally running 8b model with large context window (10k tokens+) beat google/openAI models easily on any task you like.
why would anyone then pay for something that is possible to run on consumer hardware with higher token/second throughput and better performance? What exactly have the billions invested given google/oai in return? Nothing more than an existensial crisis I'd say.
Companies aren't trying to force AI costs into their subscription models in dishonest ways because they've got a winning product.
I dont really agree with your perception of current LLMs, but the point is it doesnt even matter. This is a pr war. Lecun lost it for meta. Meta needs to be thought of as an AI leader to gain traction in their metaverse stuff. They can live with everyone thinking theyre evil but if everyone thinks theyre lame has beens they are fucked.
are they thought of as lame has-beens? OR even on a trajectory for that to be thought of them? I don't think that's true, at least not in my circles. Like you said, evil, sure, but not has been.
This is the right take. He is obviously a pioneer and much more knowledgeable than Wang in the field, but if you don't have the product mind to serve company's business interest in short term and long term capacity anymore, you may as well stay in academia and be your own research director, let alone a chief executive in one of the largest public companies
It's very hard (and almost irreconcilable) to lead both Applied Research -- that optimizes for product/business outcomes -- and Fundamental Research -- that optimizes for novel ideas -- especially at the scale of Meta.
LeCun had chosen to focus on the latter. He can't be blamed for not having taken the second hat.
Yes he can. If he wanted to focus on fundamental research he shouldn’t have accepted a leadership position at a product company. He knew going in that releasing products was part of his job and largely blew it.
Yann was in charge of FAIR which has nothing to do with llama4 or the product focussed AI orgs. In general your comment is filled with misrepresentations. Sad.
tbf, transformers from more of a developmental perspective are hugely wasteful. they're long-range stable sure, but the whole training process requires so much power/data compared to even slightly simpler model designs I can see why people are drawn to alternative complex model designs down-playing the reliance on pure attention.
I totally agree. He appeared to act against his employer and actively undermined Meta's effort to attract talent by his behavior visible on X.
And I stopped reading him, since he - in my opinion - trashed on autopilot everything 99% did - and these 99% were already beyond the two standard deviation of greatness.
It is even more highly problematic if you have absolutely no results eg products to back your claims.
Yeah I think LeCun is underestimating the impact that LLM's and Diffusion models are going to have, even considering the huge impact they're already having. That's no problem as I'm sure whatever LeCun is working on is going to be amazing as well, but an enterprise like Facebook can't have their top researcher work on risky things when there's surefire paths to success still available.
I politely disagree - it is exactly an industry researcher's purpose to do the risky things that may not work, simply because the rest of the corporation cannot take such risks but must walk on more well-trodden paths.
Corporate R&D teams are there to absorb risk, innovate, disrupt, create new fields, not for doing small incremental improvements. "If we know it works, it's not research." (Albert Einstein)
I also agree with LeCun that LLMs in their current form - are a dead end. Note that this does not mean that I think we have already exploited LLMs to the limit, we are still at the beginning. We also need to create an ecosystem in which they can operate well: for instance, to combine LLMs with Web agents better we need a scalable "C2B2C" (customer delegated to business to business) micropayment infrastructure, because as these systems have already begun talking to each other, in the longer run nobody would offer their APIs for free.
I work on spatial/geographic models, inter alia, which by coincident is one of the direction mentioned in the LeCun article. I do not know what his reasoning is, but mine was/is: LMs are language models, and should (only) be used as such. We need other models - in particular a knowledge model (KM/KB) to cleanly separate knowledge from text generation - it looks to me right now that only that will solve hallucination.
Knowledge models, like ontologies, always seem suspect to me; like they promise a schema for crisp binary facts, when the world is full of probabilistic and fuzzy information loosely categorized by fallible humans based on an ever slowly shifting social consensus.
Everything from the sorites paradox to leaky abstractions; everything real defies precise definition when you look closely at it, and when you try to abstract over it, to chunk up, the details have an annoying way of making themselves visible again.
You can get purity in mathematical models, and in information systems, but those imperfectly model the world and continually need to be updated, refactored, and rewritten as they decay and diverge from reality.
These things are best used as tools by something similar to LLMs, models to be used, built and discarded as needed, but never a ground source of truth.
>Knowledge models, like ontologies, always seem suspect to me; like they promise a schema for crisp binary facts, when the world is full of probabilistic and fuzzy information loosely categorized by fallible humans based on an ever slowly shifting social consensus.
I don't disagree that the world is full of fuzziness. But the problem I have with this portrayal is that formal models are often normative rather than analytical. They create reality rather than being an interpretation or abstraction of reality.
People may well have a fuzzy idea of how their credit card works, but how it really works is formally defined by financial institutions. And this is not just true for software products. It's also largely true for manufactured products. Our world is very much shaped by artifacts and man-made rules.
Our probabilistic, fuzzy concepts are often simply a misconception. That doesn't mean it's not important of course. It is important for an AI to understand how people talk about things even if their idea of how these things work is flawed.
And then there is the sort of semi-formal language used in legal or scientific contexts that often has to be translated into formal models before it can become effective. Law makers almost never write algorithms (when they do, they are often buggy). But tax authorities and accounting software vendors do have to formally model the language in the law and then potentially change those formal definitions after court decisions.
My point is that the way in which the modeled, formal world interacts with probabilistic, fuzzy language and human actions is complex. In my opinion we will always need both. AIs ultimately need to understand both and be able to combine them just like (competent) humans do. AI "tool use" is a stop-gap. It's not a sufficient level of understanding.
> People may well have a fuzzy idea of how their credit card works, but how it really works is formally defined by financial institutions.
> Our probabilistic, fuzzy concepts are often simply a misconception.
How eg a credit card works today is defined by financial institutions. How it might work tomorrow is defined by politics, incentives, and human action. It's not clear how to model those with formal language.
I think most systems we interact with are fuzzy because they are in a continual state of change due to the aforementioned human society factors.
To some degree I think that our widely used formal languages may just be insufficient and could be improved to better describe change.
But ultimately I agree with you that this entire societal process is just categorically different. It's simply not a description or definition of something, and therefore the question of how formal it can be doesn't really make sense.
Formalisms are tools for a specific but limited purpose. I think we need those tools. Trying to replace them with something fuzzy makes no sense to me either.
I believe the formalisms can be constructed by something fuzzy. Humans are fuzzy; they create imperefect formalisms that work until they break, and then they're abandoned or adapted.
I don't see how LLMs are significantly different. I don't think the formalisms are an "other". I believe they could be tools, both leveraged and maintained by the LLM, in much the same way as most software engineers, when faced with a tricky problem that is amenable to brute force computation, will write up a quick script to answer it rather than try and work it out by hand.
I think AI could do this in principle but I haven't seen a convincing demonstration or argument that Transformer based LLMs can do it.
I believe what makes the current Transformer based systems different to humans is that they cannot reliably decide to simulate a deterministic machine while linking the individual steps and the outcomes of that application to the expectations and goals that live in the fuzzy parts of our cognitive system. They cannot think about why the outcome is undesirable and what the smallest possible change would be to make it work.
When we ask them to do things like that, they can do _something_, but it is clearly based on having learned how people talk about it rather than actually applying the formalism themselves. That's why their performance drops off a cliff as soon as the learned patterns get too sparse (I'm sure there's a better term for this that any LLM would be able to tell you :)
Before developing new formalisms you first have to be able to reason properly. Reasoning requires two things. Being able to learn a formalism without examples. And keeping track of the state of a handful of variables while deterministically applying transformation rules.
The fact that the reasoning performance of LLMs drops off a cliff after a number of steps tells me that they are not really reasoning. The 1000th rules based transformation only depending on the previous state of the system should not be more difficult or error prone than the first one, because every step _is_ the first one in a sense. There is no such cliff-edge for humans.
You're basically describing the knowledge problem vs model structure, how to even begin to design a system which self-updates/dynamically-learns vs being trained and deployed.
Cracking that is a huge step, pure multi-modal trained models will probably give us a hint, but I think we're some ways from seeing a pure multi-modal open model which can be pulled apart/modified. Even then they're still train and deploy not dynamically learning.
I worry we're just going to see LSTM design bolted onto deep LLM because we don't know where else to go and it will be fragile and take eons to train.
And less said about the crap of "but inference is doing some kind of minimization within the context window" the better, it's vacuous and not where great minds should be looking for a step forwards.
I have vague notions of there being an entire hidden philosophical/political battlefield (massacre?) behind the whole "are knowledge models/ontologies a realistic goal" debate.
Starting with the sophomoric questions of the optimist who mistakes the possible for the viable: how definite of a thing is "the world", how knowable is it, what is even knowledge... and then back through the more pragmatic: by whom is it knowable, to what degree, and by what means. The mystics: is "the world" the same thing as "the sum of information about the world"? The spooks: how does one study those fields of information which are already agentic and actively resist being studied by changing themselves, such as easily emerge anywhere more than n(D) people gather?
Plenty of food for thought from why ontologies are/aren't a thing. The classical example of how this plays out in the market being search engines winning over internet directories. But that's one turn of the wheel. Look at what search engines grew into quarter century later. What their outgrowths are doing to people's attitude towards knowledge. Different timescale, different picture.
Fundamentally, I don't think human language has sufficient resolution to model large spans of reality within the limited human attention span. The physical limits of human language as information processing device have been hit at some point in the XX century. Probably that 1970s divergence between productivity and wages.
So while LLMs are "computers speak language now" and it's amazing if sad that they cracked it by more data and not by more model, what's more amazing is how many people are continually ready to mistake language for thought. Are they all P-zombies or just obedience-conditioned into emulating ones?!?!?
Practically, what we lack is not the right architecture for "big knowing machine", but better tools for ad-hoc conceptual modeling of local situations. And, just like poetry that rhymes, this is exactly what nobody has a smidgen of interest to serve to consumers, thus someone will just build it in their basement in the hope of turning the tables on everyone. Probably with the help of LLMs as search engines and code generators. Yall better hurry. They're almost done.
Nice commentary and I enjoyed the poetic turn of phrase. I had to respond to it with my own thoughts if only to bookmark it for myself.
> how many people are continually ready to mistake language for thought
This is a fundamental illusion - where, rote memory and names and words get mistaken for understanding. This was wonderfully illustrated here [1]. Few really grok what understanding actually is. This is an unfortunate by-product of our education system.
> Are they all P-zombies or just obedience-conditioned into emulating ones?!?!?
Brilliant way to state the fundamental human condition. ie, we are all zombies conditioned to imitate rather than understand. Social media amplifies the zombification, and now LLMs do that too.
> Starting with the sophomoric questions of the optimist who mistakes the possible for the viable
This is the fundamental tension between operationalized meaning and imagination. A grokking soul gathers mists from the cosmic chaos and creates meaning and operationalizes it for its own benefit and then continually adapts it.
> it's amazing if sad that they cracked it by more data and not by more model
I was speaking to experts in the sciences (chemistry). They were shocked that the underlying architecture is brute force. They expected a compact information-compressed theory which is able to model independent of data. The problem with brute-force approaches are that they dont scale, and dont capture the essences which are embodied in theories.
> The physical limits of human language as information processing device have been hit at some point in the XX century
2000 years back when humans realized that formalism was needed to operationalize meaning, and natural language was too vague to capture and communicate it. Because the world model that natural language captures encompasses "everything" whereas for making it "useful" requires to limit it via formalism.
Is it that fuzzy though? If it was would language not adequately grasp and model our realities? And what about the physical world itself: animals are modeling the world adequately enough to navigate it. There's significant gains to make from modeling _enough_ of the world, without falling into hallucinations of purely statistical associations of an LLM.
World models are trivial. eg narratives are world models and they provide only pre frontal simulation, ie they are synthetically prey-predation.
No animal uses world models for survival and doubtful they exist (maps are not models), a world model doesn't conform to optic flow, ie instantaneous use and response. Anything like a world model isn't shallow, the basic premise of oscillatory command, it's needlessly deep, nothing like brains. This is just a frontier hail-mary to the current age.
> it is exactly a researcher's purpose to do the risky things that may not work
Maybe at university, but not at a trillion dollar company. That job as chief scientist is leading risky things that will work to please the shareholders.
They knew what Yann LeCun was when they hired him. If anything, those brilliant academics who have done what they're told and loyally pursued corporate objectives the way the corporation wanted (e.g. Karpathy when he was at Tesla) haven't had great success either.
>They knew what Yann LeCun was when they hired him.
Yes but he was hired in the ZIRP era where all SV companies were hiring every opinionated academic and giving them free reign and unlimited money to burn in the hopes that maybe they'll create the next big thing for them eventually.
These are very different economic times right now, after the FED infinite money glitch has been patched out, so now people do need to adjust to them and start actually making some products of value for their seven figure costs to their employers, or end up being shown the door.
I’ve yet to meet a single person who claims AGI will happen without recycling the same broken reasoning the peak-oil retards were peddling a decade ago.
Talking to these people is exhausting, so I cut straight to the chase: name the exact, unavoidable conditions that would prove AGI won’t happen.
Shockingly, nobody has an answer. They’ve never even considered it.
That’s because their whole belief is unfalsifiable.
LLMs and Diffusion solve a completely different problem than world models.
If you want to predict future text, you use an LLM. If you want to predict future frames in a video, you go with Diffusion. But what both of them lack is object permanence. If a car isn't visible in the input frame, it won't be visible in the output. But in the real world, there are A LOT of things that are invisible (image) or not mentioned but only implied (text) that still strongly affect the future. Every kid knows that when you roll a marble behind your hand, it'll come out on the other side. But LLMs and Diffusion models routinely fail to predict that, as for them the object disappears when it stops being visible.
Based on what I heard from others, world models are considered the missing ingredient for useful robots and self-driving cars. If that's halfway accurate, it would make sense to pour A LOT of money into world models, because they will unlock high-value products.
Sure, if you only consider the model they have no object permanence. However you can just put your model in a loop, and feed the previous frame into the next frame. This is what LLM agent engineers do with their context histories, and it's probably also what the diffusion engineers do with their video models.
Messing with the logic in the loop and combining models has an enormous potential, but it's more engineering than researching, and it's just not the sort of work that LeCun is interested in. I think the conflict lies there, that Facebook is an engineering company, and a possible future of AI lies in AI engineering rather than AI research.
This is something that was true last year, but hanging on by a thread this year. Genie shows this off really well, but it's also in the video models as well.[1]
I think World models is way to go for Super Intelligence. One of teh patent i saw already going in this direction for Autonomous mobility is https://patents.google.com/patent/EP4379577A1 where synthetic data generation (visualization) is missing step in terms of our human intelligence.
This is the first time I have heard of world models. Based on my brief reading it does look like this is the idea model for autonomous driving. I wonder if the self driving companies are already using this architecture or something close to it.
I thoroughly disagree, I believe world models will be critical in some aspect for text generation too. A predictive world model you can help to validate your token prediction. Take a look at the Code World Model for example.
> I think LeCun is underestimating the impact that LLM's and Diffusion models
No, I think hes suggesting that "world models" are more impactful. The issue for him inside meta is that there is already a research group looking at that, and are wildly more successful (in terms of getting research to product) and way fucking cheaper to run than FAIR.
Also LeCun is stuck weirdly in product land, rather than research (RL-R) which means he's not got the protection of Abrash to isolate him from the industrial stupidity that is the product council.
The last time LeCun disagreed with the AI mainstream was when he kept working on neural net when everyone thought it was a dead end. He might be entirely right in his LLM scepticism. It's hardly a surefire path. He didn't prevent Meta from working on LLM anyway.
The issue is more than his position is not compatible with short term investors expectations and that's fatal in a company like Meta at the position LeCun occupies.
> Facebook can't have their top researcher work on risky things when there's surefire paths to success still available.
How did you determine that "surefire paths to success still available"? Most academics agree that LLMs (or LLMs alone) are not going to lead us to AGI. How are you so certain?
I don't believe we need more academic research to achieve AGI. The sort of applications that are solving the recent AGI challenges are just severely resource constrained AGI. The only difference between those systems and human intelligence are resources and incentives.
Not that I believe AGI is the measure of success, there's probably much more efficient ways to achieve company goals than simulating humans.
How many decades did it take for neural nets to take off?
The reason we're even talking about LeCun today is because he was early in seeing the promise of neural nets and stuck with it through the whole AI winter when most people thought it was a waste of time.
But neural nets were always popular, they just went through phases of hype depending on the capacity of hardware at the time. The only limitation of neural nets at the time was computational power to scale up. AI winters came when other techniques became available that required less compute. Once GPGPU became available, all of that work became immediately viable.
No similar limitations exist today for JEPA, to my knowledge.
Depends on how far back you are going. There was the whole 1969 Minsky Perceptron flap where he said ANNs (i.e Perceptrons) were useless because they can't learn XOR (and no-one at the time knew how to train multi-layer ANNs), which stiffled ANN research and funding for a while. It would then be almost 20 years until the 1986 PDP handbook published LeCun and Hinton's rediscovery of backpropagation as a way to train multi-layer ANNs thereby making them practical.
The JEPA parallel is just that it's not a popular/mainstream approach (at least in terms of well funded research), but may eventually win out over LLMs in the long term. Modern GPUs provide plenty of power for almost any artifical brain type approach, but of course are expensive at scale, so lack of funding can be a barrier in of itself.
In the software development world yes, outside of that, virtually none. Yes, you can transcribe a video call in Office, yes, but that's not ground breaking. I dare you to list 10 impacts on different fields, excluding tech and including at least half blue collar fields and at least half white collar fields , at different levels from the lowest to the highest in the company hierarchy, that LLM/Diffusion models are having. Impact here specifically means a significant reduction of costs or a significant increase of revenue. Go on
I'm also not sure it even drives a ton of value in software engineering. It makes the easy part easier and the hard part harder. Typing out software in your mind was never the difficult part. Figuring out what to write, how to interpret specs in context, how to make your code work within the context of a broader whole, how to be extensible, maintainable, reliable, etc. That's hard, and LLMs really don't help.
Even when writing, it shifts the mental burden from an easy thing (writing code) to a very hard thing (reading that code, validating it's right, hallucination free, and then refactoring it to match your teams code style and patterns).
It's great for building a first-order approximation of a tech demo app that you then throw out and build from scratch, and auto-complete. In my experience, anyways. I'm sure others have had different experiences.
You already mentioned two fields they have a huge impact on, software development and NLP (this latter one the most impacted so far). Another field that comes to mind is academic research is getting an important boost as well, via semantic search or more advanced stuff like Google's biological cell model which already uncovered new treatments. I'm sure I'm missing a lot of other fields I'm less familiar with (legal, for example). But just these impacts I listed are all huge and they will indirectly have a huge impact on all other areas of human industry, it's just a matter of time. "Software will eat the world" and all that.
Personally, I find myself using LLMs more than Google now, even for non-development tasks. I think this shift is going to become the new normal (if it isn't already).
And what's the end result? All one can see is just bigger representation of those who confidently subscribe to false information and become arrogant when their validity is questioned, as the LLM writing style has convinced them it's some sort of authority. Even people on this website are so misinformed to believe that ChatGPT has developed its own reasoning, despite it being at the core an advanced learning algorithm trained on a enormous amount of human generated data.
And let's not speak about those so deep into sloth that put it into use to deteriorate, and not augment as they claim to do, humane creative recreational activities.
This seems a bit self-contradictory: you say LLMs mislead people and can't reason, then fault them for being good at helping people solve puzzles or win trivia games. You can't have it both ways.
Why would you postulate these two to be mutually exclusive?
> then fault them for being good at helping people solve puzzles or win trivia games
They only help them in the same sense that a calculator would 'help' win at a hypothetical mental math competition, that is the gist; robbing people of the creative and mentally stimulating processes that make the game(s) fun. But I've come to realize this is an unpopular opinion on this website where being fiercely competitive is the only remarkable personality trait, so I guess yeah it may be useful for this particular population.
While I agree with your point, “Superintelligence” is a far cry from what Meta will end up delivering with Wang in charge. I suppose that, at the end of the day, it’s all marketing. What else should we expect from an ads company :?
not sure I agree. AI seems to be following the same 3-stage path of many inventions: innovation > adoption > diffusion. LeCun and co focus on the first, and LLMs in their current form appear to be incremental at improvements; we're still using the same basis from more than ten years ago. FB and industry are signalling a focus on harvesting the innovation and that could last - but also take - many years or decades. Your fundamental researchers are not interested (or the right people) in that position.
Do you? Or is it possible to acknowledge a plateau in innovation without necessarily having an immediate solution cooked-up and ready to go?
Are all critiques of the obvious decline in physical durability of American-made products invalid unless they figure out a solution to the problem? Or may critics of a subject exist without necessarily being accredited engineers themselves?
LLM's are probably always going to be the fundamental interface, the problem they solved was related to the flexibility of human languages allowing us to have decent mimikry's.
And while we've been able to approximate the world behind the words, it's just full of hallucinations because the AI's lack axiomatic systems beyond much manually constructed machinery.
You can probably expand the capabilties by attaching to the front-end but I suspect that Yann is seeing limits to this and wants to go back and build up from the back-end of world reasoning and then _among other things_ attach LLM's at the front-end (but maybe on equal terms with vision models that allows for seamless integration of LLM interfacing _combined_ with vision for proper autonomous systems).
> because the AI's lack axiomatic systems beyond much manually constructed machinery.
Oh god, that is massively under-selling their learning ability. These models are able to extract and reply with why jokes are funny without even knowing basic vocab, yet there are pure-code models out there with lingual rules baked in from day one which still struggle with basic grammar.
The _point_ of LLMs arguably is there ability to learn any pattern thrown at it with enough compute.
With an exception to learning how logical processes work, and pure LLMs only see "time" in the sense of a paragraph begins and ends.
At the least they have taught computers, "how to language", which in regards to how to interact with a machine is a _huge_ step forward.
Unfortunately the financial incentives are split between agentic model usage (taking the idea of a computerised butler further), maximizing model memory and raw learning capacity (answering all problems at any time), and long-range consistency (longer ranges give better stable results due to a few reasons, but we're some way from seeing an LLM with a 128k experts and 10e18 active tokens).
I think in terms of building the perfect monkey butler we already have most or all of the parts. With regard to a model which can dynamically learn on the fly... LLMs are not the end of the story and we need something to allow the models to more closely tie their LS with the context. Frankly the fact that DeepSeek gave us an LLM with LS was a huge leap since previous model attempts had been overly complex and had failed in training.
>If you think LLMs are not the future then you need to come with something better
The problem isn't LLMS, the problem is that everyone is trying to build bigger/better llms or manually code agents around LLMs. Meanwhile, projects like Mu Zero are forgotten, despite being vastly more important for things like self driving.
I agree. I never understood LeCun's statement that we need to pivot toward the visual aspects of things because the bitrate of text is low while visual input through the eye is high.
Text and languages contain structured information and encode a lot of real-world complexity (or it's "modelling" that).
Not saying we won't pivot to visual data or world simulations, but he was clearly not the type of person to compete with other LLM research labs, nor did he propose any alternative that could be used to create something interesting for end-users.
Text and language contain only approximate information filtered through humans eyes and brains. Also animals don't have language and can show quite advanced capabilities compared to what we can currently do in robotics. And if you do enough mindfulness you can dissociate cognition/consciousness from language. I think we are lured because how important language is for us humans, but intuitively it's obvious to me language (and LLMs) are only a subcomponent, or even irrelevant for say self driving or robotics.
Seems like that "approximation" is perfectly sufficient for just about any task.
That whole take about the language being basically useless without a human mind to back it lost its legs in 2022.
In the meanwhile, what do those "world model" AIs do? Video generation? Meta didn't release anything like that. Robotics, self-driving? Also basically nothing from Meta there.
In the meanwhile, other companies are perfectly content with bolting multimodal transformers together for robotics tasks. Gemini Robotics being a research example - while modern Tesla FSD stack being a production grade one. Gemini even uses a language transformer as a key part of its stack.
The issue is context. trying to make an AI assistant with just text only inputs is doeable but limiting. You need to know the _context_ of all the data, and without visual input most of it is useful.
For example "Where is the other half of this" is almost impossible to solve unless you have an idea of what "this" is.
but to do that you need to have cameras, to use cameras you need to have position, object, and people tracking. And that is a hard problem thats not solved.
the hypothesis is that "world models" solve that with an implicit understanding of the worl and the objects in context
If LeCun's research has made Meta a powerhouse of video generation or general purpose robotics - the two promising directions that benefit from working with visual I/O and world modeling as LeCun sees it - it could have been a justified detour.
LLMs get results is quite the bold statement.
If they get results, they should be getting adopted, and they should be making money. This is all built on hazy promises.
If you had marketable results, you wouldn't have to hide 20+ billion dollars of debt financing into an obscure SPV.
LLMs are the most baffling piece of tech. They are incredible, and yet marred by their non-deterministic hallucinatory nature, and bound to fail in adoption unless you convince everyone that they don't need precision and accuracy, but they can do their business at 75% quality, just with less human overhead.
It's quite the thing to convince people of, and that's why it needs the spend it's needing. A lot of we-need-to-stay-in-the-loop CEOs and bigwigs got infatuated with the idea, and most probably they just had their companies get addicted to the tech equivalent of crack cocaine.
A reckoning is coming.
LLMs get results, yes. They are getting adopted, and they are making money.
Frontier models are all profitable. Inference is sold with a damn good margin, and the amounts of inference AI companies sell keeps rising. This necessitates putting more and more money into infrastructure. AI R&D is extremely expensive too, and this necessitates even more spending.
A mistake I see people make over and over again is keeping track of the spending but overlooking the revenue altogether. Which sure is weird: you don't get from $0B in revenue to $12B in revenue in a few years by not having a product anyone wants to buy.
And I find all the talk of "non-deterministic hallucinatory nature" to be overrated. Because humans suffer from all of that too, just less severely. On top of a number of other issues current AIs don't suffer from.
Nonetheless, we use human labor for things. All AI has to do is provide a "good enough" alternative, and it often does.
In this comment you proceeded to basically reinvent the meaning of "profitable company", but sure.
I won't even get into the point of comparing LLM to humans, because I choose not to engage with whoever doesn't have the human decency, humanistic compass, or basic phylosophical understanding of how putting LLMs and human labor on the same level to justify hallucinations and non-determinism is deranged and morally bankrupt.
You don't even need insider info - it lines up with external estimates.
We have estimates that range from 30% to 70% gross margin on API LLM inference prices at major labs, 50% middle road. 10% to 80% gross margin on user-facing subscription services, error bars inflated massively. We also have many reports that inference compute has come to outmatch training run compute for frontier models by a factor of x10 or more over the lifetime of a model.
The only source of uncertainty is: how much inference do the free tier users consume? Which is something that the AI companies themselves control: they are in charge of which models they make available to the free users, and what the exact usage caps for free users are.
Adding that up? Frontier models are profitable.
This goes against the popular opinion, which is where the disbelief is coming from.
Note that I'm talking LLMs rather than things like image or video generation models, which may have vastly different economics.
> We also have many reports that inference compute has come to outmatch training run compute for frontier models by a factor of x10 or more over the lifetime of a model.
Dario Amodei from Anthropic has made the claim that if you looked at each model as a separate business, it would be profitable [1], i.e. each model brings in more revenue over its lifetime than the total of training + inference costs. It's only because you're simultaneously training the next generation of models, which are larger and more expensive to train, but aren't generating revenue yet, that the company as a whole loses money in a given year.
Now, it's not like he opened up Anthropic's books for an audit, so you don't necessarily have to trust him. But you do need to believe that either (a) what he is saying is roughly true or (b) he is making the sort of fraudulent statements that could get you sent to prison.
He's speaking in a purely hypothetical sense. The title of the video even makes sure to note "in this example". If it turned this wasn't true of anthropic, it certainly wouldn't be fraud.
OpenAI and Anthropic are making north of 4B/year revenue so some companies have figured out the money making part. ChatGPT has some 800M users according to some calculations. Whether it's enough money today, enough money tomorrow, is of course a question but there is a lot of money. Users would not use them in a scale if they do not solve their problems.
People used to say this about Amazon all the time. Remember how Amazon basically didn’t turn any real profits for 2 decades? The joke was that Amazon was a charitable organisation being funded by Wall Street for the benefit of human kind.
That didn’t last. People in the know knew that once you have a billion users and insane revenue and market power and have basically bought or driven out of business most of your competitors (Diapers.com, Jet.com, etc) you can eventually slow down your physical expansion, tighten the screws on your suppliers, increase efficiencies, and start printing money.
The VCs who are funding these companies are hoping that they have found the next Amazon. Many will probably go out of business, but some might join the ranks of trillion dollar companies.
If you’ve got nearly a billion users, and are multiplying your revenue on an annual basis, then yes. You’re effectively showing that you’re in hyper growth trajectory.
Hyper growth is expensive because it’s usually capital intensive. The trick is, once that growth phase is over, can you then start milking your customers while keeping a lid on costs? Not everyone can, but Amazon did, and most investors think OpenAI and Anthropic can as well.
If you hire a house cleaner to clean your house, and the cleaner didn't do well, would you eject yourself out of the house? You would not. You would change to a new cleaner.
But if we hire someone to deal on R&D to automate fully the house cleaning process, we might not necessarily expect the office to be maintained in clean state by the researchers themselves any time we enter the room.
As far as an analogy, whatever happens, whoever messes up, Zuck, as the owner, of course, takes the final blame.
That's why he is changing the team.
Still, people bring up this weird point that would be equivalent to you giving up your house equity for free because you fucked up on hiring and managing your house.
I think he means Zuckerberg himself, the metaverse isn't exactly a major success, but this is a false equivalency the way he organized it only his vote matters he does what he wants
You joke, but the Star Wars games - especially the pinball one, for me at least - are some of the best experiences available on Quest headsets. I've been playing software pinball (as well as the real thing) since the 80s, and this is one of my favorite ways to do it now, which I will keep coming back to.
> But… I suppose Zuckerberg knows what he wants, which is AI slopware and not truly groundbreaking foundation models.
When did they make groundbreaking foundation models though? DeepMind and OpenAI have done plenty of revolutionary things, what did Meta AI do while being led by LeCun?
Musk only cares about AI as far as it can be used to replace all sources of information with versions that will say whatever he wants and spout his worldview of the day.
Musk cares about AI research as much as he cared about Path of Exile
I suppose they could solve superintelligence and cure cancer and build fusion reactors with it, but that's 100% outside their comfort zone - if they manage to build synthethic conversation partners and synthethic content generators as good or better than the real thing the value of having every other human on the planet registered to one of their social network goes to zero.
Which is impossible anyway - I facebook to maintain real human connections and keep up with people who I care about, not to consume infinite content.
At 1.6T market cap it's very hard to 10x or greater the company anymore doing what's in their comfort zone and they've got a lot of money to play with to find easier to grow opportunities. If Zuckerberg was convinced he could do that by selling toothpicks they'd have a go at the toothpick business. They went after the "metaverse" first, then AI. Both are just very fast growth options which happen to be tech focused because that's the only way you generate new comparable value as a company (unless you're sitting on a lot of state owned oil) in the current markets.
they are out for your clicks and attention minutes
if OpenAI can build a "social" network of completely generated content, that can kill Meta. Even today I venture to guess that most of the engagements in their platforms is not driven by real friends, so an AI driven platform won't be too different, or it might make content generation be so easy as to make your friends engage again.
Apart from it the ludicrous vision of the metaverse seems much more plausible with highly realistic world models
How do LLMs help with clicks and attention minutes? Why do they spend $100+B a year in AI capex, more than Google and Microsoft that actually rent AI compute to clients? What are they going to do with all that compute? It’s all so confusing
Browse TikTok and you already see AI generated videos popping up. Could well be that the platforms with the most captivating content will not be a "social" network but one consisting of some tailor made feed for you. That could undermine the business model of the existing social networks - unless they just fill it with AI generated content themselves. In other words: Facebook should really invest in good video generating models to keep their platforms ahead.
It might be just me, but in my opinion facebook platforms are way past the "content from your friends phase", but is full of cheap peddled viral content.
If that content becomes even cheaper, of higher quality and highly tailored to you, that is probably worth a lot of money, or at least worth not losing your entire company by a new competitor
But practically speaking, is Meta going to be generating text or video content itself? Are they going to offer some kind of creator tools so you can use it to create video as a user and they need the compute for that? Do they even have a video generation model?
The future is here folks, join us as we build this giant slop machine in order to sell new socks to boomers.
For all of your questions Meta would need a huge research/GPU investment, so that still holds.
In any case if I have to guess, we will see shallow things like the Sora app, a video generation tiktok social network and deeper integration like fake influencers, content generation that fits your preferences and ad publishers preferences
a more evil incarnation of this might be a social network where you aren't sure who is real and who isn't. This will probably be a natural evolution of the need to bootstrap a social network with people and replacing these with LLMs
Zuckerberg knows what he wants but he rarely knows how to get it. That's been his problem all along. Unlike others he isn't scared to throw ridiculous amounts of money at a problem though and buy companies who do things he can't get done himself.
There's also the aspect of control - because of how the shares and ownership are organized he answers essentially to no one. In other companies burning this much cash as was with VR or now AI without any sensible results would get him ejected a long time ago.
Zuck did this on purpose, humiliating LeCun so he would leave.
Despite LeCun being proved wrong on LLMs capabilities such as reasoning, he remained extremely negative, not exactly inspiring leadership to the Meta Ai team, he had to go.
But LLMs still can't reason... in a reasonable sense. No matter how you look at it, it is still a statistical model that guesses next word, it doesn't think/reason per se.
Reasoning is the act of figuring out how to solve a problem for which you have no previous training set. If an AI can reason, and you give it a standard task of "write me a python file that does x and y", it should be able to complete that task without ever being trained on python code. Or english in general.
The way it would solve that problem would look more like some combination of Hebbian Learning and Mu Zero, where it starts to explore the space around it interms of interactions, information gathering, information parsing, forming associations, to where it eventually understands that your task involves the action of writing bytes to a file in a certain structure that when executed produces certain output, and the rules around the structure that make it give that output.
And it will be able to do this through running as a model on your computer, or a robot that can type on a keyboard, all from the same code.
LLMs appear to "reason" because most people don't actually reason - a lot of people even in technical fields operate on a principle of information lookup. I.e they look at the things that they have been taught to do, figure out which problem fits closest, and repeat steps with a few modifications a long the way. LLMs pretty much do the same thing. If you operate like this, then sure LLMs, "reason". But there is a reason why LLMs are barely useful in actual technical work - under the hood, to make them do things autonomously, you basically have to specify wrapper code/prompts that take often as long to write and finetune as actual code itself.
It is insane to think this in 2025 unless you define "reasoning" as some mechanical information lookup. This thinking (ironically) degrades the meaning of reasoning and of intelligence in general.
Where is any proof that Yann LeCun is able to deliver that? He's had way more resources than any other lab during his tenure, and yet has nothing substantial to show for it.
No, it was because LeCun had no talent for running real life teams and was stuck in a weird place where he hated LLMs. He frankly was wasting Meta’s resources. And making him report to Wang was a way to force him out.
It wasn’t boneheaded. It was done to make Yann leave. Meta doesn’t want Yann for good reason.
Yann was largely wrong about AI. Yann coined the term stochastic parrot and derrided LLMs as a dead end. It’s now utterly clear the amount of utility LLMs have and that whatever these LLMs are doing it is much more than stochastic parroting.
I wouldn’t give money to Yann, the guy is a stubborn idiot and closed minded. Whatever he’s doing wont even touch LLM technology. He was so publicly deriding LLMs I see no way he will back pedal from that.
I dont think LLMs are the end of the story for agi. But I think they are a stepping stone. Whatever agi is in the end, LLMs or something close to it will be a modular component of aspect of the final product. For LeCunn to dismiss even the possibility of this is idiotic. Horrible investment move to give money to Yann to likely pursue Agi without even considering LLMs.
