"learn" is usually used in opposition to "taught", which refers to "expert systems"-type engineering; in other words, providing data and a success heuristic and asking it to devise its own optimal strategies vs. providing strategies hand-designed by humans.
Obviously Perceptrons came out well before 2003, but I don't think it's necessarily out of line to say that they had limited efficacy before then, both for theoretical and compute reasons. But maybe I'm misunderstanding your criticism?
"ILP" (as in Inductive Logic Programming not Integer Linear Programming) was first named in 1991 in a paper by Stephen Muggleton ("Inductive Logic Programming and Progol). The paper properly launched the field and generated a great deal of excitement at the time.
There were precursors. At least Ehud Shapiro's doctoral thesis ("Automated Debugging") in the 1980's and Gordon Plotkin's doctoral thesis in the 1970's ("Automated Methods of Inductive Inference"). Sorry for not giving the exact years off the top of my head but I think it was 1983 and 1976, respectively.
The point you are making is very right however because modern machine learning as a field started in the 1980's with the fall of expert systems, in fact it basically started as an effort to overcome one of the major limitations of expert systems, the so-called "knowledge acquisition bottleneck", which is to say, the difficulty of creating and maintaining huge databases of expert knowledge (in the form of production rules).
In any case the seminal textbook in the field for the first 20 years, Tom Mitchell's Machine Learning came out in 1997 (https://www.cse.iitb.ac.in/~cs725/notes/slides/tom_mitchell/...) and includes probabilistic, neural-net based and symbolic, logic-based approaches. So not only machines could "learn" way before 2003 but they could also learn in many different ways than what Ilya Sutskever means.
We can go further back, to Donald Michie's 1961 MENACE (the first Reinforcement Learning system, implemented on a computer made of matchboxes with coloured beads used to encode state) and Arthur Samuel's 1959 checkers player (a paper on which gave the name to the field of machine learning).
Lots of learning all over the place long, long before 2003.
Yes, so what does it say about our rigged system for a guy who neither invented the attention mechanism nor the concept of GPTs to be given that much credit for the current wave of AI? One lucky choice (betting on scale) backed by $100Ms of other people's money does not entitle one to genius-hood.
I mean, tbf, he hasn't won a Turing yet, right? So the academy hasn't fully embraced him as a genius. VCs/SV are more fickle, but even they aren't necessarily in love -- his Superintelligence startup raised a modest but far from unusual amount of cash, AFAIR
Obviously Perceptrons came out well before 2003, but I don't think it's necessarily out of line to say that they had limited efficacy before then, both for theoretical and compute reasons. But maybe I'm misunderstanding your criticism?