> My point is it's not "gone very far" because he's taking it as far as he can, which isn't very far.
I wouldn't really agree - I'm no fan of LeCun, but the problem with JEPA isn't that it's a bad idea, or can't go very far, but just that it's not much of an idea in the first place!
It's no secret that our brain basically works by prediction, and what we're predicting is necessarily the external world as we perceive it though our own senses, aka latent representations, aka JEPA.
So, you COULD take this unoriginal smidgen of an idea and built it out to a full model of a human/animal brain, whether or not it's LeCun's intention to do so (he seems more interested in just the representational / world model aspect to it), but he certainly hasn't done so yet, nor created any research manifesto indicating that as his intent.
The fact that JEPA implementations to date are using pre-trained Transformers doesn't seem inherent to the approach - one could, with more effort, still predict latent representations (i.e. sensory feedback) but do so using a new real-time learning algorithm based on prediction failure.
LeCun seems more of an academic / research director than a builder, and I would never have put much stock in him being the one to build an animal brain.
Well I also don't think JEPA is a bad idea or anything. I don't think most of the alternative architectures or tweaks i've seen are bad ideas. On the contrary, some of them seem very cool and i'm all for interesting new ideas. I guess my opinion is more on the supposed necessity of it all.
I think the transformer is a powerful general learner. I think it's enough, and on the matter of intelligence, sometimes i think we make the mistake of zoning in too much on potentially spurious details. We still don't know much about Intelligence at the end of the day, what is and isn't really important, what is and isn't simply a detail of the environment and what kind of seemingly bizarre but surprisingly equivalent mechanisms can arise in the face of vastly different environments.
I wouldn't really agree - I'm no fan of LeCun, but the problem with JEPA isn't that it's a bad idea, or can't go very far, but just that it's not much of an idea in the first place!
It's no secret that our brain basically works by prediction, and what we're predicting is necessarily the external world as we perceive it though our own senses, aka latent representations, aka JEPA.
So, you COULD take this unoriginal smidgen of an idea and built it out to a full model of a human/animal brain, whether or not it's LeCun's intention to do so (he seems more interested in just the representational / world model aspect to it), but he certainly hasn't done so yet, nor created any research manifesto indicating that as his intent.
The fact that JEPA implementations to date are using pre-trained Transformers doesn't seem inherent to the approach - one could, with more effort, still predict latent representations (i.e. sensory feedback) but do so using a new real-time learning algorithm based on prediction failure.
LeCun seems more of an academic / research director than a builder, and I would never have put much stock in him being the one to build an animal brain.