> Token efficiency is also an extremely relevant factor, but the industry unfortunately lacks reliable data here. Many people like to cite this chart from Artificial Analysis, but we do not believe the benchmark tasks in the AA Intelligence Index are at all representative of real work people do with LLMs.
Right, so they just ignore the fact that different models use very different number of tokens to achieve the same thing. Ignoring that means they can't say anything about "value". This is like comparing numbers with different units. They're Atokens and Otokens.
> Token efficiency is also an extremely relevant factor, but the industry unfortunately lacks reliable data here. Many people like to cite this chart from Artificial Analysis, but we do not believe the benchmark tasks in the AA Intelligence Index are at all representative of real work people do with LLMs.
Right, so they just ignore the fact that different models use very different number of tokens to achieve the same thing. Ignoring that means they can't say anything about "value". This is like comparing numbers with different units. They're Atokens and Otokens.