I've used Kubernetes for a few years now, and there was a description that really resonated with me: You can think of k8s as an operating system where we can deploy applications, especially those that run more than a handful of services.
Said another way, if Linux (or whatever) is the OS for your server / VM / host level / network device, k8s is the OS for your cloud application.
And, when k8s is implemented properly, it takes a lot of headaches that can come from dealing with the myriad problems that arise when your architecture goes beyond a basic handful of "tiers."
Do you know how many homes in your neighborhood have FiOS? I am wondering if it's a congestion issue - have you checked RTT in the day vs afternoon? What about looking at wireshark to see how many duplicate Acks/Re-transmits you're seeing?
As only the first-hop device appears to be different in the tracert, I doubt it's something upstream - if they were throttling, they'd probably do it at the network core - before it left the Verizon network for transit, which would apply to both circuits.
I'm on Verizon FIOS and since some time after the ruling Netflix has been taking 10+ seconds to buffer initial SD quality where before it was always less than a second. There are times where it takes 15 or 30 minutes to switch to HD and sometimes it never even does.
Meanwhile I can download games from Steam at 3+ MB/s, torrent at the same rate, and so on.
Maybe Netflix suddenly got a ton of demand shortly after the net neutrality ruling or changed their software, but frankly Verizon doesn't get the benefit of the doubt. They've done so many sketchy things that at this point if they aren't doing this on purpose they need to prove it.
I've updated the blog post to also include traceroutes from the 2 sites. I'm not a networking expert, but since the traffic is ultimately going through the same network - peering should be a non issue. Feel free to let me know if this is significant or not.
There are really too many variables to have any idea. For example, in both instances I see that your traceroute identifies the hop to your router, and you're using the 192.168.1.0 private network. Since your business example shows actual reverse dns lookups (from the looks of it), I'm guessing it uses a commercial switch.
Unfortunately, there's no way to determine the actual cause of the slow down.
For instance, you said that the slow down happens after 5pm, which quite literally is when 80+% of the working force gets off work. It could also be an artifact of AWS, and not Verizon FIOS.
Also, 2 instances does not a trend make. There's no way to know if you're one of those people who were sent a letter because they were using terabytes of data a month, and they're simply protecting the network traffic of 99% of their other users. Granted, I assume you aren't, but I can't be 100% sure.
For a truly independent test, you'd need the same modem/access device, without a local switch (i.e. a machine connected directly to the modem), and you would need VPN access to test both identifiable and non-identifiable traffic.
If you could verify that Netflix traffic through the VPN, on the same network, without any interference through the router, was delivering much more bandwidth than the same Netflix traffic, but in the open (not through the VPN), and could do so for a statistically significant portion of the Verizon FIOS user base, then you'd have reason to complain. Until then, I don't see your claim being taken very seriously.
> Unfortunately, there's no way to determine the actual cause of the slow down.
Is networking the most frustratingly complex field in technology? As someone who admittedly knows little about it it all seems like a pile of hacks stacked on top of one another. It's the only explanation I can come up for why it's near impossible to figure out why a network isn't behaving correctly.
We'd need more samples to make any serious conclusions, but it does look like you're taking the same route from VZ's gear to AWS. That would rule out any kind of peering spat or congested port theories. If the paths are indeed the same, it means that VZ is CoS'ing down residential traffic over business. You'll need to get more traceroutes during different times of day, too.
Said another way, if Linux (or whatever) is the OS for your server / VM / host level / network device, k8s is the OS for your cloud application.
And, when k8s is implemented properly, it takes a lot of headaches that can come from dealing with the myriad problems that arise when your architecture goes beyond a basic handful of "tiers."