I wonder about that? In most cases once an algorithm is trained, running inference is just a function evaluation, which is usually computationally inexpensive.

(the training is the heavy compute part)