Does anyone actually believe we will "run out of bandwidth" to the point where traffic shaping will become essential?
Traffic shaping of saturated links sounds like a waste of time and resources when it's so much easier to just upgrade the hardware. It's cheaper too! It also happens to be the fucking purpose of the ISP (to provide adequate bandwidth for their customers).
Think of how much it costs to pay people to manage an incredibly complicated traffic ruleset--not to mention the massively increased CPU overhead--versus a one-time hardware purchase.
What you're talking about is either an ISP that's dying and can't afford to upgrade their shit or a greedy evil monster that simply wants to extract more profit from existing infrastructure in the most obtuse and invasive way possible.
> Traffic shaping of saturated links sounds like a waste of time and resources when it's so much easier to just upgrade the hardware. It's cheaper too! It also happens to be the fucking purpose of the ISP (to provide adequate bandwidth for their customers).
This is a very naive comment. For anything other than a dedicated "business class" circuit with hard performance guarantees, speculation and shaping is involved in delivering the end product.
> For anything other than a dedicated "business class" circuit with hard performance guarantees, speculation and shaping is involved in delivering the end product.
The alternative to a connection with a SLA guarantee is a best effort connection, not discriminatory traffic shaping. Underprovisioning part of your network doesn't require you to show favoritism to certain traffic—the normal way to cope with larger than expected usage is to just start dropping packets where there's congestion. That doesn't imply or require traffic shaping.
> the normal way to cope with larger than expected usage is to just start dropping packets where there's congestion.
And lo and behold, that's what ISP's have always offered to consumers. There's a reason connections are always advertised as "up to 50Mbps" instead of "50 Mbps", hell, even my "business" connection with the local cable company has absolutely no guarantee beyond "we'll have someone out to repair your service within 4 hours".
ISPs will generally not sell you a plan that advertises throughput in excess of the achievable modem sync speed, unless your achievable speed is already below their lowest tier of service. AT&T won't let you sign up for 18Mbps DSL if you live 2km from a DSLAM that doesn't even have ADSL2+ equipment. The "up to" provision is because their backhaul capacity is less than the sum of the last-mile connection speeds.
My last mile is 1 GBps fiber. I have a choice of a hundred or so ISPs over that fiber. The cheapest ones drop to single-digit megabit speeds during peak hours because the ISPs have such oversubscribed interconnections.
> just start dropping packets where there's congestion
Aside from being normal, do you think this is preferable because it is fair?
I think of QoS as a way of allowing more types of SLA to exist. Maybe Youtube makes a deal with T-Mobile to keep videos playing longer (before packets start dropping) in exchange for money. This money might allow T-Mobile to invest in other parts of its infrastructure.
Isn't any backbone link in a sense a way of shaping traffic? The old pattern (from point A to B, say) may have been less efficient than the new pattern (using the new link).
The link makes economic sense to install simply because there is demand for bandwidth from A to B. If there weren't, a link would have been installed elsewhere instead. The price of the new link is passed along to downstream and upstream providers.
Assuming that each ISP has a shaping model in place that is generally fair, is there really any difference between these three things:
- the ISP shopping upstream providers based on SLA committments for traffic X
- the ISP imposing QoS that addresses the quality experienced by customers using traffic X
- the ISP investigating upgrading networking hardware to allow for superior handling of traffic X
I'd argue that in every case the ISP has a cost and a benefit. In the case of the QoS change, the cost is that some customers might be slightly worse off (those who used traffic Y but never used traffic X).
Maybe Youtube makes a deal with T-Mobile to keep videos playing longer (before packets start dropping) in exchange for money. This money might allow T-Mobile to invest in other parts of its infrastructure.
Another possibility: as a major ISP nears network capacity, they can implicitly (or explicitly, depending on law) solicit revenue streams from Youtube and many others for "priority access" to their somewhat- or completely-captive customer base. Eventually, secondary tiers are created to efficiently segment the content market -- a priority plan conveniently sized for every budget. Those who can't or won't pay fall into an ever decreasing performance category as the priority slices are snatched up.
The ISP then starts selling premium bundles on the customer side -- Platinum Service will include Youtube, Netflix, Amazon, Apple (and get our bonus in-house streaming service at no extra charge!). Platinum-Plus, for another $14.95 per month plus fees, adds priority streams from your choice of up to three major sports content sources. And Platinum-Pro adds non-degraded VPN so you can work from home. Choice!
The ISP is monetizing the fact that they're NOT adding capacity. Eventually they can choose to take the hit to their numbers and make that investment, but they'll do so within the now-normal tiered framework -- offering new capacity and premium deals at the top of the stack, rather than letting the bottom free-ride into better service.
You are describing a content lock-in dystopia. Yes, this would be anticompetitive (and appropriately addressed by the FTC or DOJ, not the FCC) and it is among the more unlikely scenarios.
The parent post is a little hyperbolic, but ISPs are certainly willing to artificially limit capacity upgrades as a way to extract tolls from content providers. Remember the spat between Verizon and Level3/Netflix?
Sure, I'm painting a picture to illustrate the point. But reductio ad absurdum can be a useful way to explore the problem. I don't think there's anything in my scenario that's beyond the pale. We could get there by degrees in an aggressively unfettered market.
>We could get there by degrees in an aggressively unfettered market.
Don't you mean the heavily regulated market we have now that prevents new ISPs from easily servicing customers? The entire reason ISPs can behave like this is because customers realistically have no alternatives.
Look at how Comcast behaved in markets where Google Fiber showed up. They often increased bandwidth and reduced prices effective immediately to try to keep customers. An 'unfettered market' would be one where an ISP doesn't have to spend years negotiating with city councils just to get the privilege to put fiber in utility tunnels where local cable companies already have it.
Your "unfettered market" would still leave incumbents with the massive advantages conferred by their natural monopoly; it just lessens the artificial barriers to competition on top of that. Your "unfettered market" would not be a healthy competitive market. Government intervention akin to local loop unbundling would be necessary to create a freely competitive market where one will not naturally develop.
Google Fiber gave up on silicon valley because of the regulatory burden of convincing local governments to let them install fiber. Let that sink in.
Google, who has enough cash to pay for nearly half of the entire Comcast company, was not able to rollout fiber to the area because of government regulations.
Why do you think they started in Kansas City of all places? It wasn't because they thought that was the strongest market for fiber in the country. It was because they actually convinced the government to let them bring fiber to peoples' homes.
>Government intervention akin to local loop unbundling would be necessary to create a freely competitive market where one will not naturally develop.
While this would make it even cheaper to get into the market, there is no proof that a market wouldn't develop if the government wasn't already intervening to make it so difficult to build networks.
> While this would make it even cheaper to get into the market, there is no proof that a market wouldn't develop if the government wasn't already intervening to make it so difficult to build networks.
Local governments aren't completely undemocratic. If the local government is hesitant to let corporations start digging up the streets and yards, it's likely that the residents are similarly hesitant to invite that sort of disruption. How many people are actually eager to have a second gigabit-capable cable buried through their yard? Whether or not the federal government intervenes, there will always be barriers to an upstart ISP. Yes, with all local restrictions usurped, a company as large as Google can certainly deploy new infrastructure to select markets where there's sufficient demand for a more reasonable alternative to the incumbent ISP. But that's still a far cry from a functioning competitive market. Who would fund construction of a third set of cables, or the sixth? Even with local restrictions removed, how much capital does it take to build a competitive ISP from scratch (keeping in mind that Google wasn't quite starting from scratch)?
In the most pedantic sense, there's obviously no proof that a functioning competitive marketplace would develop, but we are talking about a real natural monopoly, and that means something. Building last-mile infrastructure is very expensive. Each new competitor faces much worse prospects for recouping their initial investment, much more resentment from their prospective customers, and the incumbent(s) can always undercut you at the drop of a hat. Given the choice, the customers would vastly prefer a local loop unbundling solution—it doesn't have the high costs of building redundant infrastructure, it doesn't make a mess of the town, and it leads to the same end result of diverse choices for services.
>how much capital does it take to build a competitive ISP from scratch (keeping in mind that Google wasn't quite starting from scratch)?
It's significant, but it's often done in cities that don't have this regulation problem (see most high-density cities around the world with sane access to cabling tunnels). Or, if you take it to the extreme the other direction, look at all of the municipal fiber projects that end up working out. If a municipality can pay for it, certainly a company would be able to as well.
Laying fiber requires labor, but many of the service tunnels already exist so the only digging required is the last bit from the street to the customer's house.
There is an entire industry designed to help finance capital-intensive projects. Just because you can't do something with a text editor and a compiler in your basement doesn't mean it can't be done.
>but we are talking about a real natural monopoly
No we aren't. Trying to build up an ISP is 95% lobbying local governments. It's only 'natural' in the sense that it's a government created monopoly.
>much more resentment from their prospective customers
You wouldn't be pulling fiber to peoples' homes if they weren't already signing up (see the google fiber rollout mechanism).
>Given the choice, the customers would vastly prefer a local loop unbundling solution
Not if it means the best they can get is 5 mbps over cables that were laid to homes 20 years ago. If your city is filled with a bunch of NIMBYS, they are just as likely to vote down a bond or whatever it takes to raise money to lay in new fiber.
Backbone capacity is a separate issue from last-mile connectivity. Only one is truly a natural monopoly.
I don't care whether YouTube directly builds out backbone capacity or whether they outsource that to Level 3.
Incumbent last-mile providers enjoy a privileged position that gives them unfair bargaining power over end users. They should not be allowed to decide which content providers and applications get to have a successful business model. If anybody gets to pick winners and losers like that, it should be a democratically elected government, but it would be far better to just prohibit last-mile providers from imposing unfair artificial limits or costs.
If T-Mobile wants to make a deal with YouTube to provide a subsidized video streaming service with dedicated bandwidth to guarantee a certain level of video quality, then that service should be billed and provisioned separately from their common-carrier Internet and phone services. If they want this YouTube streaming service to be wireless, then they should have to acquire a separate spectrum license from the FCC and convince the FCC why a dedicated YouTube service deserves a slice of the public's limited RF spectrum when there are other video providers on the public Internet that would be disadvantaged.
> If anybody gets to pick winners and losers like that, it should be a democratically elected government, but it would be far better to just prohibit last-mile providers from imposing unfair artificial limits or costs.
Isn't it the proper authority of the FTC or DOJ to enforce anticompetitive behavior? Do we need the FCC also dipping in to impose penalties on firms who may have already spent a lot of money in the spectrum auction?
What privileges does winning the spectrum auction permit companies?
To your last point, what if T-Mobile showed that by following its preferred QoS scheme it actually transferred more information over its last mile link?
Economic incentives typically reward the highly efficient use of a resource. Being able to get an extra bushel of corn from an acre of land is one example, as is being able to get an extra Megabyte of data over a pipe each hour.
In the case of residential bandwidth packages, the metric for how effectively an ISP is maximizing the available resource should simply be its own competitive choices. Should T-Mobile sacrifice $1M in profit each month simply so that any customers who happen to be using an obscure protocol can get slightly better service?
It seems that Net Neutrality is akin to the government forcing corn farms to plant 25% of their fields with different crops, just in case anyone wants to buy them, and to force those farmers to offer them for sale even if they don't believe anyone wants to buy them.
I think if we want to go down the path of the fine-grained allocation of the spectrum as a public good, there would be large scale, sweeping changes that we'd make. Why should we waste spectrum broadcasting over-the-air TV in 1080i? Why should soap operas be broadcast in full quality or even broadcast at all? Why should anyone have LTE bandwidth when some people can't afford 2G?
> To your last point, what if T-Mobile showed that by following its preferred QoS scheme it actually transferred more information over its last mile link?
That is very obviously not even close to being the right metric. All you need to do is find the one customer with their laptop tethered and running Backblaze or bittorrent, and give him all the bandwidth. Don't bother slicing up the airtime to serve other customers, because that'll just reduce the total data rate.
> Should T-Mobile sacrifice $1M in profit each month simply so that any customers who happen to be using an obscure protocol can get slightly better service?
Yes, unless they're prepared to stop advertising it as an Internet connection. If they're all allowed to be an "Internet service provider" that is really just a HTTP(s) service provider, then we're shutting out any invention that isn't well-suited to what T-Mobile can double-charge for—any P2P application, incompatible but superior successors like QUIC, and more specialized protocols like SCTP. Technologies that are obscure today could be lucrative and more resource-efficient next year, if allowed to blossom.
Don't you think there's enough demand for halfway decent VPN service that a fair bit of market incentive exists for T-Mobile not to hinder opaque packets too much?
> Maybe Youtube makes a deal with T-Mobile to keep videos playing longer (before packets start dropping) in exchange for money. This money might allow T-Mobile to invest in other parts of its infrastructure.
That half of it isn't the problem, and I doubt anyone would argue against reasonable things that make it easier for a provider to increase their network capacity.
The problem is what it does to YouTube's competitors: they now suffer because YT can afford to ask T-Mobile to make their traffic more important.
> The problem is what it does to YouTube's competitors: they now suffer because YT can afford to ask T-Mobile to make their traffic more important.
But simply by virtue of YouTube's market share, an economic incentive exists to install backbone links that help route YouTube's traffic (as opposed to others'). In other words, if two backbone links might be installed, the one whose outcome makes access to Youtube more reliable is more likely to be installed than one that makes unknown startup's service more reliable.
In other words, network topology adapts to traffic demand (albeit more slowly than traffic shaping).
> In other words, network topology adapts to traffic demand (albeit more slowly than traffic shaping).
A fair traffic management policy would allow for YouTube users to consume bandwidth in exact proportion to their share of overall demand for bandwidth. Traffic shaping on top of that could only have the effect of meaning that a YouTube user will get more bandwidth than their Netflix-using neighbor even if they're both paying for the same service and both content providers have CDN nodes at the same local peering point. That's not adapting to demand, that's artificially distorting the market.
> A fair traffic management policy would allow for YouTube users to consume bandwidth in exact proportion to their share of overall demand for bandwidth.
I think this statement is in effect a straw man, since the relevant consideration for QoS is the timing of consuming that bandwidth.
If you mean instantaneous bandwidth, then you are saying that both Youtube customers and Netflix customers should be able to use a circuit that they primarily use for watching video to also be equally optimized for real time protocols.
If you argue that a VOIP customer should be able to place a call with the absolute minimal latency the network can provide, even if it means that video content will (on average) buffer for an extra second, that is not an example of a fair system, just one that is extremely friendly to VOIP.
Why shouldn't someone be able to start a Skype competitor that offers latency characteristics for audio calls that rivals POTS latency? As it stands, Skype unusable for low latency coms both because of protocol/codec design and typical network characteristics.
So, I think that unless you enthrone modern "typical" characteristics as ideal then it's hard to argue that imposing restrictions of what shaping approaches can and should be utilized is somehow qualitatively neutral, rather than simply benefitting incumbent interests.
> "If you mean instantaneous bandwidth, then you are saying that both Youtube customers and Netflix customers should be able to use a circuit that they primarily use for watching video to also be equally optimized for real time protocols.
> If you argue that a VOIP customer should be able to place a call with the absolute minimal latency the network can provide, even if it means that video content will (on average) buffer for an extra second, that is not an example of a fair system, just one that is extremely friendly to VOIP."
That's the real straw man. The current state of the art for QoS is active queue management that both increases effective link utilization and reduces latency to a point that VoIP is usable, as compared with naive FIFO queuing with tail drops when the queues are full. That's why I said QoS isn't a zero-sum game. In the real world, you don't actually have to choose between throughput and latency; both can be good enough for what people normally want to do. VoIP traffic doesn't cost enough bandwidth to have a significant impact on Netflix video quality, and firing up a Netflix stream doesn't have to produce queuing latencies that make VoIP unusable. And none of this requires any traffic shaping rules that deliberately identify what's video or VoIP—that's an objective criteria for "neutral" that doesn't force ISPs to use naive sub-optimal traffic management.
> And none of this requires any traffic shaping rules that deliberately identify what's video or VoIP
True, so why is net neutrality necessary then? Do you think ISPs are making this possible out of the goodness of their hearts?
I'd argue that their profit motive has made this happen because it's what consumers demand, and the internet (protocols and services) has changed rapidly enough that entrenched patterns have not really emerged.
And people are perfectly fine with that! Define a service level, any level, then you have a product to sell.
But the current situation, where they sell "up to" whatever the last mile will sustain and then don't ever upgrade backbones, don't add interconnects, that particular fraud can not go on. A market doesn't work when vendors can sell you a product, lock you in and face no repercussions when they don't deliver even 10%.
As long as a competitive market exists, I'm happy to let the market work out who has the better strategy: hardware oriented vs rulesets, etc. But where there is no competition (e.g. the last residential mile), I want regulations protecting me as a consumer with various accessibility, privacy, and fairness guarantees.
> Does anyone actually believe we will "run out of bandwidth" to the point where traffic shaping will become essential?
Traffic shaping has always been essential. Bandwidth has never been infinite, and there's always been latency vs. bandwidth tradeoffs that need to be managed.
> Bandwidth has never been infinite, and there's always been latency vs. bandwidth tradeoffs that need to be managed.
Dropping packets has always been necessary. Traffic shaping has never been necessary, and modern fair AQM algorithms are a net win to both throughput and latency relative to the most commonly deployed congestion management strategies, so that latency vs bandwidth tradeoff is far more illusory than you think.
no. there is a service level agreement so if they don't meet the promises there is repurcussione. it's also not an apt analogy considering one can easily switch to Azure or Google Cloud or Self hosting whereas most people are stuck with one ISP that has a monopoly in that area
Traffic shaping of saturated links sounds like a waste of time and resources when it's so much easier to just upgrade the hardware. It's cheaper too! It also happens to be the fucking purpose of the ISP (to provide adequate bandwidth for their customers).
Think of how much it costs to pay people to manage an incredibly complicated traffic ruleset--not to mention the massively increased CPU overhead--versus a one-time hardware purchase.
What you're talking about is either an ISP that's dying and can't afford to upgrade their shit or a greedy evil monster that simply wants to extract more profit from existing infrastructure in the most obtuse and invasive way possible.