I don't understand why they felt necessary to rewrite a core component in another language. If you have a garbage collection problem my first intuition would be to produce less garbage!
For example better using the stack, or pulling out the big gun of manual memory management.
I'm sure they had reasons to choose Go when they first designed this project but they don't go into them at all.
Feels like they just wanted to play with a new toy.
> If you have a garbage collection problem my first intuition would be to produce less garbage!
FTA:
“These latency spikes definitely smelled like garbage collection performance impact, but we had written the Go code very efficiently and had very few allocations. We were not creating a lot of garbage.
[…]
the spikes were huge not because of a massive amount of ready-to-free memory, but because the garbage collector needed to scan the entire LRU cache in order to determine if the memory was truly free from references”
They explained in the post why this wasn't an issue: they were producing very little garbage, but there was a very large object graph.
> manual memory management
If you need to do manual memory management in a GC language with no builtin support for it, like Go, that's probably a sign that you should switch to a different language.
And importantly in a tracing GC the work load is a function of the non-garbage data not of the garbage.
Generational GC can avoid some of it by ignoring the old code entirely, but Go does not implement generations because its relatively strong ability to stack-allocate generally replaces the nursery, and for most work loads you don’t recoup the costs.
Problem with languages like Go is, because they have GC, their API is less tuned to the ability to prevent new allocations.
In C you see lots of functions whose first parameter is a buffer that will be re-used.
And even in Rust you don't see this often, because reusing a buffer means having to do &mut, which means having to re-structure your code. And it's really easy to do Vec::new().
One of big issue with GC is it need to scan every reachable objects to mark it is reachable. That mean even if your code produce no garbage but have a large amount of long lived object the GC still need to traverse all of those objects every cycle.
It's possible that median is a better measure of central tendency for those things. It also sounds like you're interpreting their use of 'average' as meaning median when you think that would make more sense, rather than asking them (and in fairness, I don't usually grill people on which measure they're thinking of when they say 'average' either). In my experience, people who actually understand the different measures of central tendency usually (on average, heheh) specify one explicitly. The rest of the population is probably only aware of arithmetic mean.
Largely agree, but it's hardly OP's fault. In my opinion this is one of the things we get by using 30 year old Operating System and there having been almost no innovation in that field for so long.
Much higher productivity demands for luxury products. If we want back to produce a variety of direct human edible vegetables the productivity would not be necessary at all. When you're raising corn, soybean and wheat for cattle then it's obviously a much bigger higher order problem.
we don't grow "for the sake of growing", there's plenty of poor people in the world that need that growth. Whether that growth reaches them is another story..
For example better using the stack, or pulling out the big gun of manual memory management.
I'm sure they had reasons to choose Go when they first designed this project but they don't go into them at all.
Feels like they just wanted to play with a new toy.