I'd be surprised if Google Translate isn't powered by a specialized, Transformer-powered model we would now recognize as a "small" language model heavily constrained to a specific task.

that already happened in like 2016