As context sizes grow, it's easier to add lots of information outside of training data via in-context learning, which should offset that issue quite a bit.