There was a post a few months ago about a developer who's job it was to rework ML stuff into actual efficient code. This reminds me of that post because it seems that lots of ML stuff is just plain inefficient...in that they use way too many resources given the problem.
And they're arguing about mmap, something that's been around forever.
It reminds me of that speedup in a package manager because they were reading uncached byte-at-a-time off of disk. You need to explicitly turn buffered reads off...but why would you do that in the first place? Unbuffered reads are almost never a good idea, ever.
It makes me wonder what other weird sub-optimal stuff is lying underneath the resource behemoth that is ML.
> it seems that lots of ML stuff is just plain inefficient
This is just like any other technology. Use it wrong, you will get burned. Doesn't matter how shiny the container is.
"ML stuff" also (mostly?) includes purely statistical methods from the 90s that are deterministic and arguably the best way to solve a large variety of non-generative problems.
In fact, unless generation of arbitrary output is a major objective, it's likely you can solve whatever ML task on a workstation from 2010 that uses intel integrated graphics.
And they're arguing about mmap, something that's been around forever.
It reminds me of that speedup in a package manager because they were reading uncached byte-at-a-time off of disk. You need to explicitly turn buffered reads off...but why would you do that in the first place? Unbuffered reads are almost never a good idea, ever.
It makes me wonder what other weird sub-optimal stuff is lying underneath the resource behemoth that is ML.