One of the big things that enabled extended context to work was training techniques for extending LLMs with reasoning, part of the thing he was saying wouldn't work.