It's an 8-bit int type though. Perhaps even that's sufficient for inference?

For a lot of network types that will do the job. 2x the performance is often better than 2x the fidelity. You can get higher accuracy with more nodes.