Article was touched by pr dept, but still has actually information.
longer tldr:
They did the same thing that has been done for thousands of years. Back then the hot area of research was how to stage advance food and resource caches along a route for long journeys. They came up with algorithms to optimize cache hits.
In this case, the problem is GPUs can be fast for ML, but usually only have 16GB ram when dataset can be terabytes.
Simple chunk processing would seem to solve the problem, but it’s turns out overhead of cpu/gpu transfers badly degraded performance.
Their claim here is they can on the fly determine how important different samples are, and make sure samples that yield better results are in the chance more often than those with less importance.
> Their claim here is they can on the fly determine how important different samples are, and make sure samples that yield better results are in the chance more often than those with less importance.
Isn't that the exact same idea as in active learning?
I don't think you need active learning to get results like this, just decent statistical analysis. There are parallels here with distributed query planning.
Could you please not post snarky dismissals of other people's work? I realize that PR-filtered bigco tech articles aren't the greatest medium. But when you hand-wave this back to "the same thing that has been done for thousands of years", that's the kind of cheap internet discourse that degrades and ultimately destroys a site like HN, which is trying for something at least a little better.
To be fair - most innovation boils down to this kind of incremental stuff. 99% of 'tech' is an amalgamation of more basic ideas, not 'magic leap' kind of innovation.
I mean, we all love the magic, but I think we're getting spoiled as of late with all the magic AI/Deep Learning stuff coming out.
I agree with your first statement to the point of disagreeing with your second. i.e. even the magic stuff is just incremental progress that people were not paying attention to. (Self driving cars have been wowing people since the 90s, object recognition just got incrementally better every year etc)
Oh certainly, I did not intend to be dismissive of their work.
My goal in a tldr is only to minimize the number of seconds it takes to digest some essential concept.
I wish for every article here someone would write up a 1 sentence tldr and a one paragraph tldr+, to help us track more happenings in our head at once and to help choose the ones we decide to spend our deep reading time on.
But of course your point is valid, shoulders of giants and what have you...
Article was touched by pr dept, but still has actually information.
longer tldr:
They did the same thing that has been done for thousands of years. Back then the hot area of research was how to stage advance food and resource caches along a route for long journeys. They came up with algorithms to optimize cache hits.
In this case, the problem is GPUs can be fast for ML, but usually only have 16GB ram when dataset can be terabytes.
Simple chunk processing would seem to solve the problem, but it’s turns out overhead of cpu/gpu transfers badly degraded performance.
Their claim here is they can on the fly determine how important different samples are, and make sure samples that yield better results are in the chance more often than those with less importance.