There are several reasons why Internet latency is inflated over direct-route speed-of-light latency. Some reasons may change others may not.
1) speed of light in fibre is 2/3c - unlikely to change soon
2) fibre is laid in the ground on a circuitous route - rarely changes but does happen
3) queueing delay - that's often the main culprit, changeable, but hard (see the buffer bloat debate)
The way I understand this proposal, the idea is to find at which request rate the latency of a storage configuration (eg, 2x Optane) explodes. If your request rate is above that rate, get more RAM. I'm not sure I understand how to figure out how much more RAM? Until the latency goes down I guess?
Also what about diurnal patterns? Can there be a way to dynamically react and save power..
Interesting reference to the morning paper. But don't get how that fits into the story?
This is well explored space in more tradtional storage land. Look in to cumulative distribution functions. This is going to show you your workload distribution and effective “working set” to optimize for in a time period. This will get you to a place where youre optimizing total cost vs performance as a function.
Youll also want to be cognizant of convex “shoulders” in the distribution that will trip up naive optimization algorithms. I do t have the link offhand, but search for hill climbing in relation to CDFs. Some related work might be in the relatively unexplored “cache insertion” problem area. Check out TinyLFU as an example of knowing what to cache being more beneficial than what to evict.
For more advanced techniques look in to some of the published work from places like Coho Data. They had a great paper back at usenix 2015ish around optimizing placement in dynamic workloads across different storage media.
And lastly experimentation is great to prove a hypotheis, but not the most effective discovery. Youll want to get representative workload traces and use those to replay/simulate against different constraints. Check out Fio and its IO trace capabilities for an example.
1) Super workload dependent unfortunately. Probably "easiest" way is to set up test systems with enough RAM and disk space. Then either slowly reduce the RAM or start from a low point and slowly increase until you hit your target; there's a command for memcached to live-adjust the RAM limit.
2) dunno! For RAM that's super hard. for disk I'm hoping tiering will do something. ie; for on-peak you can add extra NBD space then remove it off-peak.
3) referenced tail latency because most of the testing focuses on displaying latency outliers and how the system generally minimizes them.
I thought the RAM estimation was (1) benchmark your backend, (2) benchmark your memcache<->backend, (3) measure peak traffic performance requirements, (4) provide enough RAM that you reduce the latency to acceptable levels during peak traffic. If you don't have measurements of the first three you're shooting from the hip. It is workload dependent, but the more measurements you have, the better you can estimate. (Used to be mandatory so you could do trend analysis and purchase your new gear in time for inevitably outgrowing the old stuff)
Wouldn't the RAM + disk just be an extension of that method, though? It's like in the article, you test a subset of the system (RAM+disk) to find its limits, then you compare that to your other measurements and load requirements, and replace RAM with RAM+disk where it's cost and performance appropriate. But I might have missed part of the analysis
The article was significantly cut down for brevity, so any detail of how to find the right balance is beyond the scope :)
The original question was that it can be really hard to determine how much adding RAM will offload from the disk. IE: with extstore recent objects are served by RAM and never disk, so they don't count against your IO balance. If you're coming from a RAM backed system that has 500k requests per second, it's going to take some creative introspection to figure out how much RAM you'd need to keep the disk accesses below 400k, or whatever your target is.
Along those lines I just suggested a simple experiment. which is to just take an existing fully specced instance, add a disk, and test a few RAM settings to see how it looks. Most actual users of extstore have just done that since they're extending/replacing an existing cluster.
There're actually other ways you can simply read the numbers out of memcached but it just takes more effort or familiarity with its internals.
I believe the objective is to replace expensive RAM with less expensive SSD if your workload is suitable for it. It appears there are workloads for which that is an acceptable cost/latency tradeoff.
Can't you do that with DNS records where there are multiple IPs on a A record?
Basically, we're already doing this for fault tolerance and load balancing within a single CDN. Except that currently we randomize the IPs. To enforce priorities, you'd want the IPs in the A record at least partially ordered by provider.
Multiple IPs on an A record works to some extent, most (many?) browsers will silently retry another ip from the list if some of the IPs don't accept a connection; I don't know if they'll try another IP on timeout though.
But you can't actually expect any ordering to make it through to the client. Your authoritative server may reorder the records, their recursive server may reorder the records, and the client resolution library may also reorder the records. There's actually an RFC advocating reordering records in client libraries; it's fairly misguided, but it exists in the wild. Reordering is also likely to happen in OS dns caches where those are used.
For reference, RFC 3484 [1] is the misguided RFC that tells people they should sort their DNS responses to maximize the common prefix between the source and destination address. This is probably helpful when the common prefix is meaningful, but when numbering agencies give out neighboring /24's to unrelated networks, and related networks often have ips widely distributed across the overall address space, it's not actually useful.
This is a very hot topic recently. It seems to me that several communities (databases, CDNs, storage, CPU architecture, theory) do a lot of redundant work and don't communicate a lot with each other..
Yes, that paper LHD (from a colleague of mine) does use online stochastic modeling of the workload to make better decisions. So this definitely fits the bill
Author here: this is very much a research project with the purpose of prototyping the idea that caching can help tail latency. Previously, people didn't believe that caches can be used that way and it turns out you really have to rethink how you use the caches.
We build a testbed and replayed production workloads. This is not running in production yet. We're actively looking for new workloads and scenarios to test this at scale!
Sounds like a marketing blurb but from just a few days ago:
"Azure is breaking the speed barrier in cloud connectivity. ExpressRoute Direct provides 100G connectivity for customers with extreme bandwidth needs. This is 10x faster than other clouds."
1) speed of light in fibre is 2/3c - unlikely to change soon 2) fibre is laid in the ground on a circuitous route - rarely changes but does happen 3) queueing delay - that's often the main culprit, changeable, but hard (see the buffer bloat debate)