Everyday Magic is a diary series that seeks to explain many of the computing concepts we use on a regular basis in a way that non-techies can understand, and introduce them to the lingo we techies use in the process so that people know what tech types are talking about as well as how technology changes in the news will affect you! For more information about The Technomancer or the Everyday Magic series, please take a moment to read the founding installment of the series.
Hello, Kossacks!
Sorry for not posting in a while. Work's been kicking my ass lately, and on top of that, I'm trying to bootstrap a gaming company, so I haven't had a lot of time to put one of these together.
However, in light of the recommended diary about Net Neutrality, I thought that I'd try to give an Everyday Magic explanation of the entire situation. This will likely be more dense and less humorous than my usual fare, but to really understand all the dynamics in play here, I encourage you to read it all and ask questions in the comments.
Follow me below the cloud for the nitty gritty on Net Neutrality, a view of arguments in play here for either side of the argument as it relates to Netflix...and why Net Neutrality is not actually what we should be fighting for, since it's a bandage for a much larger problem.
Yes, that's right -- Net Neutrality isn't the endgame here. In fact, while it dying will cause short term harm to both the consumers and the market, it will expose to everyone watching the larger problem that makes Net Neutrality even needed in the first place: Cable and wireless Internet need to be regulated as common carriers to promote competition from competing ISPs. And unfortunately, if we've learned anything from how our country runs nowadays, nothing changes if it's not causing a significant amount of pain.
Before we get into that, however, let's look at the issues in play here when it comes to the current Net Neutrality discussion, since it's possible for all parties arguing for/against it to be correct but still at odds. The Comcast/Netflix dispute is a great case study.
There are two main arguments in play when it comes to the Comcast/Netflix agreement that keeps getting brought up as an example of why Net Neutrality is needed. These are the consumer side of the argument and the peering (how the Internet is interconnected) side of the argument.
Finally, I'll address the regulatory side of the argument which nobody seems to be bringing up which is the central point to this installment of Everyday Magic.
The Consumer Argument
We're getting fucked here, plainly and simply, by a content provider being forced into a paid-peering agreement, because as far as ISPs at all tiers are concerned, Netflix is another customer, just like you or me...they're just buying upload-biased service as compared to the download-biased service that most home and non-tech business consumers use. Under the current paradigm of ISP connectivity, we're paying our ISPs for what should be a "dumb pipe" insofar as content is concerned. At the consumer level, we're buying throughput that can max out at a certain level (and may have an overall usage cap on the total data we can consume, especially when it comes to LTE connectivity) that is content blind. Netflix is paying their ISP for enough throughput to stream content to all of their customers that request it.
So for Comcast to both take our subscription fees as consumers and take money from Netflix (and drive up our subscription fees to that) rightly strikes many of us as a completely unfair arrangement, especially when Comcast is wildly profitable, and their markup on the service we're paying for can be measured by orders of magnitude.
See, at the scale Comcast and Netflix are at, it costs roughly a penny and a half to two pennies per gigabyte to transmit traffic within the US. This includes the cost of hardware, personnel, business operations, tech support, further capital investment to upgrade the network as bandwidth requirements increase...etc. And it only gets cheaper as our TCP congestion algorithms get better and backbone routers move from 40Gbps bonded channels to 100Gbps bonded channels.
Yeah, that's right, if you're paying $50/mo for a decent connection and have a 250GB cap (like Comcast customers do), you're paying $50 for around $5 worth of wholesale data transport.
It's extremely hard for a company like Comcast, which is making billions in revenue and profit, to make the argument that this is an undue burden on them -- as they need to increase their network speeds anyway to compete for business customers (as do the backbone providers), and due to how these networks are laid out, it's one of the few times where something built out for business actually does trickle down to the consumer, because the difference between small-business class service and your service is generally nothing more than they have a 4 hour or less SLA (service level agreement) in the case of an outage with perhaps a slightly larger upload throughput, while your consumer class connection has a 24 to 48 hour SLA in the case of an outage. Still uses the same infrastructure, from the last mile through to the data center.
Net Neutrality advocates are correct, due to these reasons, that Net Neutrality is absolutely needed to protect consumers and to prevent last-mile service provides like Comcast from turning the Internet into Cable TV 2.0 where you can get the basic Apple/Facebook/Google/Amazon package, or tack on the CNN/MSNBC/Fox news package, or the DailyKos/HuffPo/Gawker blog package, etc.
Hell, under that sort of paradigm, they might actually be able to get people to pay for porn again outside of cheap motel rooms.
That doesn't even get into the fact that's ass-backwards from the current cable model, where content bundlers (cable, satellite, FIOS/uVerse, etc) pay the content creators for the right to retransmit their content at a profit, which is worthy of an entire diary of its own, rather than getting paid at both ends like the Comcast/Netflix deal has done for Comcast.
The Technical Argument, or, wherein I cause myself pain by having to at least partially defend those fuckers at Comcast, or, yes, Netflix really does break a part of the Internet
The Internet depends on a certain level of agreement between all of the major throughput providers, from the Tier 1 "backbone" ISPs like Level 3 and Cogent Communications who shuttle everyone's traffic around, to the last mile ISPs like Comcast. These are known as peering agreements, which are based on the understanding that it benefits all involved to have strong interconnectivity between these various networks, and if they all ship each other's traffic around, we have a much better functioning Internet and less duplicated physical infrastructure.
The best analogy I can use to explain this is the sort of roaming agreements that the cell companies had back in the day when they were using the same type of radio communications and your AT&T phone would roam to a Verizon tower and stay connected. Except in AT&T and Verizon's case, they'd settle up accounts at some point along the way.
In the case of Internet peering, they don't settle up and assume that it's close enough for horseshoes and hand grenades at the end of the day, especially when it comes to the Tier 1 providers.
Now, Comcast is what's known as a Tier 2 network -- they're large enough to engage in peering agreements of their own and route traffic through their network for others (and vice versa), but to eventually connect their network to the Internet at large, they purchase IP transit from Tier 1 network providers. This is why the Tier 1 providers are known as backbone providers.
So how does this all relate to Netflix, you ask?
As far as a Tier 1 provider is concerned, Netflix is the same as you and me. They're purchasing the throughput to upload their content to consumers requesting it (or, more specifically, to content delivery networks, but that's more detail than what's needed) -- it's just that their connections have a much larger upload capability and much larger total throughput capability.
Comcast, on the other hand, being a Tier 2 provider, has to maintain good peering agreements with the backbones so that they don't start getting bills for not schlepping around as many packets for other providers as the Tier 1 providers are schlepping around for them. So when Comcast's consumers request streaming video like Netflix, the sheer size of that traffic dwarfs anything else other than spam email, which everyone suffers from equally.
Due to this, Comcast is in a position where they aren't able to hold up their end of the bargain on a non-paid peering agreement, so Tier 1 providers like Cogent and Level 3 start leaning on them to either peer more of their traffic or start settling up the transit cost differences between what Comcast transports for them and what they transport for Comcast.
Comcast, of course, rather than spending some of that massive profit to either pay up or expand their network to handle more peered traffic, simply starts throttling inbound traffic from the Tier 1 providers that Netflix pays for their outbound bandwidth to keep the amount of traffic peered for them somewhat close to what they peer for others.
So unlike popular belief that this has to do with the size of the traffic crushing Comcast's network, that's not the case at all.
This, of course, impacts Netflix, since consumers tend to blame the content provider rather than the transit layer or pipe provider. If it hadn't gotten out into the media, most people would still be saying "Fucking Netflix!" when House of Cards started buffering rather than "Fucking Comcast!"
The deal that Comcast struck with Netflix has Netflix edge servers (servers where content exits Netflix's network) interconnected directly into Comcast's network in at least one data center (Equinix in San Jose), and likely/will be more, if for no other reason than to remove the single point of failure for Netflix traffic into Comcast's network. Comcast still has to carry the same amount of traffic (more, actually) across its last mile infrastructure, but now it's not having Netflix's traffic crush its peering agreements with other transit providers nearly as bad.
Yes, Comcast is having Netflix pay for something that directly benefits Comcast and reduces their costs...and frankly, benefits the Internet as a whole since it greatly reduces the amount of traffic shipped across the backbones as the data streams that represent each show/movie being streamed on Netflix are duplicated inside Comcast's network, rather than being duplicated at a CDN (content delivery network, which is a type of company that will mirror your data across the world so you, as a content provider, only have to upload it once or twice to serve millions) then schlepped across the backbones into Comcast's network.
Plus, a lot of the bigger CDNs already have edge servers inside of larger ISP networks, because it makes them have faster delivery times as a selling point to content creators like Netflix.
So really, while in most cases an agreement like this would be bad for consumers because it creates a "fast lane" for services that can pay for it like Google and Netflix (Google keeps edge servers in many backbone and large ISP datacenters for this reason, and pays for that privilege), and despite how bad Comcast still looks in this situation, the sheer amount of traffic that Netflix pushes across the US makes this arrangement a benefit to the Internet as a whole, when it comes to the actual technology and agreements that allow us to have an Internet rather than a series of old-school-AOL-and-Prodigy-style walled-garden "splinternets".
And yes, it hurts me to say that, because I really hate Comcast. But that dispute, as a whole, is less about Net Neutrality as Comcast was throttling an entire set of networks that affected other clients of those networks like Amazon Web Services and anything hosted there, not just Netflix, and more about the intricate set of peering agreements that actually give us the Internet...because the Tier 1 providers like Cogent (which is a particularly dickish pack of assholes) were starting to be dicks about decades-old peering agreements.
Why (more) Common Carrier Regulation Is Needed
If you've made it this far, you're probably a bit confused. So is Net Neutrality a good thing or not?
Well, it is. By far. But only because it's a bandage that's covering the true reason why it's needed in the first place.
The only reason Net Neutrality is needed is because we, as consumers, generally have very limited choice when it comes to who we can use as an internet service provider if we want a download speed faster than 1.5mbps or 6mbps.
While some urban areas like San Francisco have a plethora of options for achieving wideband connections (urban Ethernet, Cable, VHDSL, fixed microwave + wired buildings, LTE), the vast majority of the country is generally left with the following options for wideband (>20mbps):
Cable internet, from one provider,
VHDSL/FIOS, from one provider,
Severely capped/expensive LTE wireless service.
Many areas don't have the VHDSL/FIOS option either, because VHDSL distance-limited technologies that require you to be within a certain distance of the nearest neighborhood node, and Verizon stopped expanding their FIOS offering in favor of expanding the far more lucrative LTE networks. And if you're in a rural area, you may not even have the LTE option.
Now, you could go for good old ADSL service that caps at 6mbps, and you have a plethora of competing ISPs there. Even VHDSL has competitors, because it's over phone lines and therefore subject to common carrier regulations that state the line owner must lease bandwidth/carry phone calls for competing local exchange carriers/CLECs -- this is why even if AT&T owns your land line if you still have one, you can get your actual service through Verizon, or Sprint, or one of the other boutique phone services. Sonic.net's Fusion offering is AT&T uVerse and only available where uVerse is, because common carrier transport regulations force AT&T to let Sonic.net resell their service and receive transit for that data at a fixed, fair rate.
No such regulations exist for cable and wireless.
Now, the argument against further regulation is that if we don't like it, we can start our own ISP. This, of course, has many, many flaws. Firstly, the capital investment required to build out a wholly redundant national (or even regional) network is staggering -- far too massive a risk compared to the other options that level of capital has for investment.
Secondly, no sane city is going to dig up the streets every time a new service provider wants to come into town, nor should they be expected to, and you can only put so many different radio towers up in one area that all do the same damn thing, just for different corporations. Finally, many areas have state, county, or local laws that prevent competitors from even trying to build out a network where one already exists...thanks lobbyists!
Since that puts us in a situation where we therefore have a de facto commons crisis, and none of us really want our streets to be in worse disrepair than may already are, the logical solution is to enforce common carrier regulations across all digital communications providers, not just ones utilizing the legacy telephone lines.
But how does that solve the Net Neutrality problem?
Because smaller ISP networks can split the peering of Netflix's massive traffic, and peering costs would need to be calculated into fixed rate Comcast can charge those ISPs for transporting their data around. It allows Comcast to even up their peering arrangements. It gives us consumers a real choice of service options. It forces the larger ISPs to continue to innovate lest they get outperformed by a hungry startup business that can do what they do cheaper now that they have access to the infrastructure, even when including their share of the buildout and maintenance costs of the lines.
And because of that competition, if one of them did decide to move to a non-neutral, cable TV style model, bringing competition against them now requires a much more reasonable capital investment, and that fixed cost per GB transported isn't going to go down for the non-neutral ISP.
Net Neutrality is good for consumers, to a point, since it stops them from being placed in a situation where a monopoly/duopoly leaves them with no choice to escape a service they don't want. But it does nothing to address that consumers are still stuck in a situation where their provider doesn't have to compete, and isn't a regulated regional monopoly utility like PG&E. We should support Net Neutrality, but at the end of the day, it's simply a bandage until we get real choice and competition for our connectivity dollar.
That's the endgame, not Net Neutrality. And if the bandage of Net Neutrality gets ripped off, we'll feel the pain first, but as the internet has graduated from luxury to utility, it will end up hurting Comcast and their ilk more in the long run, so long as we work to continue voting in people who believe in the power of sane regulation, not the crap we have now.