I think Jeff Dean is right in that we will see much more specialised silicon in the future.

If something more bio inspired ie. predictive coding and in-memory compute fundamentally makes continual learning and much lower energy consumption possible there will be specialised hardware for it at some point

FWIW I think the brain has multiple “learning rules” and operates at multiple timescales