You kind of summed it up right there...'it's just an arithmetic logic unit for tensors'...whether its a matrix multiply engine or implmenting sha256 still a custom rapidly iterated chip for narrow very specific use case. Google accomplishment here clearly the software, but doubt they only ones going to crack asic systolic arrays. At some point china inc figures this out in mass...maybe bitmain themselves.
Well... I'll concede that 2-instructions that aren't reading or writing memory is a pretty narrow set of operations. But it's also much more generally useful than computing one specific hash function.
Completely agree with you there. Guess the analogy I was making was more about the evolutionary speed we saw there. Google seems to be moving awful fast here.
Google is the leader on AI algorithms. Where GANs and Capsules and so many others came from for example. So things change and Google can implement in silicon far earlier than anyone else as needed.
But right now they have what they need.
BTW, I would be curious on the TPU 3 design considerations to better support dynamic routing with Capsule networks. I suspect creates different memory access patterns and where the big power savings are at today.
