I don't recall them saying that, but, I mean, is Gemini Ultra a "ton" better than GPT-4? It seemingly doesn't represent a radical change. I don't see any claim that it's using revolutionary new methods.
At best Gemini seems to be a significant incremental improvement. Which is welcome, and I'm glad for the competition, but to significantly increase the applicability of of these models to real problems I expect that we'll need new breakthrough techniques that allow better control over behavior, practically eliminate hallucinations, enable both short-term and long-term memory separate from the context window, allow adaptive "thinking" time per output token for hard problems, etc.
Current methods like CoT based around manipulating prompts are cool but I don't think that the long term future of these models is to do all of their internal thinking, memory, etc in the form of text.
At best Gemini seems to be a significant incremental improvement. Which is welcome, and I'm glad for the competition, but to significantly increase the applicability of of these models to real problems I expect that we'll need new breakthrough techniques that allow better control over behavior, practically eliminate hallucinations, enable both short-term and long-term memory separate from the context window, allow adaptive "thinking" time per output token for hard problems, etc.
Current methods like CoT based around manipulating prompts are cool but I don't think that the long term future of these models is to do all of their internal thinking, memory, etc in the form of text.