Dennis Forbes on Software and Technology   Subscribe to RSS


About the Author
Dennis Forbes Dennis Forbes is a Toronto-based software architect. While focused primarily on the .NET and SQL Server worlds, Dennis frequently ventures outside of this comfort zone into game development and image processing. He has been published in several industry magazines, has been quoted in the Wall Street Journal and has been interviewed by NPR.

He is a vice president and lead software architect at an innovative New York City hedge fund back-office services firm.

Dennis has been working on solutions for the financial, telecommunications, and power generation markets for over 15 years.




The Feed Bag
Feb 24 - TED

 
Friday, June 19 2009

It’s hard to believe that Slashdot has been around for over a decade.

Taking a couple of paragraphs to reminisce, I signed up for a Slashdot account not long after the site opened for business. Following my normal tradition at the time, I suffixed my newly created account with the two-digit year of creation, in that case 98. It was an easy way to choose account names that were less likely to be rejected as already in use.

I didn't imagine that I'd be using the same account over a decade later, so now I have a nick that makes me look like I’m pretending to be an eleven year old (if you ignore the 4-digit user ID, it is conceivable that an eleven year old might visit Slashdot).

I didn’t join the ranks of Slashdot to cheerlead Linux and open source, despite it being the overwhelming direction of the site at the time.

My intentions were much less idealistic.

DSC03277I actually signed up to troll a co-worker: a Linux-loving long-haired hippy named Warren Postma. The guy was always going on about the greatness of open source software and the related initiatives, about the evils of Microsoft and closed-source, and so on, and of course he was an early fan of Slashdot.

[Over time I came to appreciate that much of what he had to say was right on the money, and in reality he was one of the best developers I’ve had the pleasure of working with.

And he wasn’t really long-haired or a hippy, though maybe my perception imagined him that way at the time, what with all the free-software rhetoric]

That was an era when being up on the latest meme was a lot more difficult, so a site like Slashdot was a welcome source, allowing you to knowingly declare that you'd already seen the dancing baby when a less-connected co-worker gets it emailed to them and comes over acting all connected-like. Though to be fair Slashdot came a little late to give a heads up of the dancing baby craze, but it was there for All Your Base Natalie Portman hot grits, though along the way to enlightment came some less appealing memes involving tubs and goats.

Slashdot no longer fills the industry-dominating position that it once did (where being "Slashdotted" was a term used in the mainstream press) — I suspect it has more to do with its competitors pandering to the common, all degrading eventually to funny pictures, "facepalms" and non-stop political posturing and claims of oppression, with sites like Digg and later Reddit escaping from the technology ghetto, while Slashdot remained chained to its original niche — but it often still has interesting and elucidating discussions throughout its comments, even if the UI of the site continues to grow worse with every passing month. Seriously, guys, the endless page thing was demonstrastably lame on DZone, so why did you think it was a good idea?

So during a moment of mental relaxation yesterday I thought I’d see what was up in the world of /., where the top story at the time detailed a congressman’s bill to “Ban ISP caps”. I found the submission hard to believe, so I RTFA to find that it had been very poorly interpreted.

What the congressman really proposed is more of a fee oversight body similar to those that many utilities have to work within.

Here in Canada the monopoly phone companies, for instance, have to apply to the CRTC every time they want to change fees (whether to increase or decrease) or service levels, justifying why they need to make those changes. Of course they’ve always been grotesquely profitable, and seldom get denied their desires (except, humorously, when it came to reducing fees to compete better against new upstarts), but just pretend that such bureaucracies make them more competitive or accountable or economical or something.DSC03673

So I posted a comment saying that the summary was inaccurate. I also tossed in a little aside about utilities, drawing from the congressman and the submission’s analogy comparing ISPs with utilities, speculating that soon ISPs might just switch to a pure consumption model more analogous with the utilities that they are being compared to, saying "I don't get carte blanche from the electric company to use it all for free, complaining that "they provide 20A to the house so I should be able to use 20A around the clock for free!"."

I touched a sore spot, which I have to confess to doing intentionally. Sometimes I can’t help but troll a bit, and given that it’s a pretty common position on sites like Slashdot that it is a basic human right to torrent a pipe full around the clock, I thought I’d pick that scab a bit.

I got the typical “the bits are free and the man is just keeping us down.” variety of responses, including my favourite.

Anyone who is comparing ISPs and bandwidth to other utilities such as electricity, water, or anything tangible should simply stop posting… immediately. The Internet is not a series of tubes, your comparisons are invalid, and you are ignorant.

Please cease in the proliferation of these ludicrous analogies.

Ignoring that the utility analogy came directly from the congressman himself, this comment has made me personally invested in this spurious analogy, so now I must embrace it with vigour.

And, for that matter, the internet is effectively a series of tubes, as fun as it might have been to point and laugh at Mr. Stevens. Those tubes have a finite capacity, and if you want to push more through, you have to lay more or “wider” tubes.

Conceptually it isn’t all that different from a power or water network, where many of the same circuit principals apply.

And those “tubes” didn’t magically appear from nowhere. They aren’t public infrastructure. The internet isn’t free. Bandwidth isn’t free. Throughput isn’t free. All of it costs money to lay and service, and as demands grow, there is the capital required to increase capacity.

So let’s go back to the electricity analogy (even though I had no intention to draw it out so literally, I’m invested now).

Back to the responses again, a less stupid one said:

The total amount of power you use in a month directly affects the amount of fuel a power utility has to burn, or the amount of water you consumer affects how much water the utility has to treat. Bits on a connection aren't like that. If you don't use a bit on their fibre link to the backbone, that doesn't leave them with an extra bit, and if you use a bit, the next one is coming at the same time and same cost anyway.

Fair point. When you use electricity, the variable costs increase. When you use water they have to buy a little more chlorine and pay the electric company to run the pumps.

Yet here in Ontario most of my electricity comes from either hydro-electric (our power company was historically called “Ontario Hydro”, causing confusion for decades as children tried to understand why the power company was effectively named Ontario Water), or nuclear.

The variable costs with either are a small percentage of the total cost to operate those services, and instead the vast majority of the cost comes in the form of capital expenditure – what they had to build and maintain to handle the current load, and to handle tomorrow’s load.

Most of the “conservation” efforts – incremental and accelerated fees for heavy consumers, and rebates for power saving appliances and technologies — have been aimed at avoiding exceeding the gross capacity of the system right now, because the next step will cost billions, and it’s not to buy a few more fuel rods.

Nuclear power plants and giant dam projects don’t come cheap; nor does building a water purification system to service a city of 200,000. It’s one of those perverse situations where if demand dropped significantly, they’d actually have to increase costs.

To wring every last bit of ridiculousness out of this analogy, imagine that you live in a village of 100 people and @Tesla just twittered about a cool new tech called electricity.

Hyped about the latest fad, you get a bank loan and put up a wind generator, connecting it to anyone willing to pay an even split of the costs (a loan you amortized to get paid off just as the turbine needs replacement, plus basic maintenance and upkeep).

Your new turbine is spinning around the clock (it’s a very windy valley near the ocean), putting out a constant 100kW, and everyone is enjoying their 1kW space heater, all paying their 1/100th cost.

Every now and then someone turns on a hairdryer, but it balances out because other people sometimes turn off their space heater, and in the end the load can be handled due to its natural distribution.

But then someone decides that they’d also like to run their 5kW back massager around the clock – you did say that electricity could be used around the clock, and the wind is free, right? — so they crank it up and your system starts to suffer brownouts and everyone suffers.

You face some tough choices: You can either put a 1kW limit on everyone (“dedicated” capacity), which would be unfortunate given that most of your customers benefit from sporadically spiking their load while they use their motorized hair curler, or you can start charging or limiting only the outliers, maybe targeting the growing trend of back massagers specifically, putting limiters on that specific use of your power.

Or you can add capacity.

So you decide to do the latter in the most cost-effective way, arranging a power sharing agreement with a neighbouring village: When you have left over capacity you send it to them, and they do the same in return.

It all balances out, and everything is fine in the world again, and that one customer gets to enjoy his back massager, essentially freeloading on the standard customer load.

But then another user decides that he, too, might as well get a back massager. And then another. And then another.

The neighbouring village starts rethinking the power sharing agreement because it is no longer serving both partners equally. They demand either payment for their inconvenience, or they’re going to sever the line.

So now, despite the fact that there is “no cost” for each incremental kW, the village is faced with a massive capital (or loan servicing) cost of putting up another wind turbine.

This is grossly simplified, as analogies often are (and there are countless similar scenarios, from toll roads to bridges to movie rentals…this is like beginner economics, but so many are so blissfully ignorant of it), but there are a lot of parallels with internet backbones, where end-users are getting 10Mbps connections to a grossly oversold network at a heavily discounted price. Any delusion that there’s some 10Mbps dedicated channel available for every subscriber is absurd. Go and buy a commercial dedicated T1 at 1/6th the speed and see how the pricing stacks up.

Consider that my little suburban town has about 50,000 high speed customers, all with 10Mbps service. If we all used our throughput, this single town would be a 400Gbps load on the network. Try laying a 400Gbps city wide network, much less province wide, much less country wide, much less continent wide, much less…that is an enormously expensive proposition.

This is the reality of the internet today. Maybe the backbones should be a public bit of infrastructure, with massive, limitless pipes from coast to coast, but that isn’t currently the case. If people want to argue for that – I think a next generation, coast-to-coast Canada backbone would be a great project for some of those stimulus billions — then they should argue for that.

Things are getting better. There are billions of dollars constantly being spent improving the backbone of the net (YouTube and Hulu today alone today account for more traffic than the entire backbone could handle in 2000), and I’m sure in a few years we’ll all being using our 1Gbps connections to play a live-streamed 1080p never-obsolete video game system.

I have no desire to defend ISPs, or to support some of the various tactics they have pursued (traffic shaping, for instance, is not at all justifiable, nor should a carrier care or restrict which sites you visit, trying to monetize the fact that you visit Hulu, or trying to extort Google. Restrict or charge based on the throughput, not specific uses of it that should be none of your concern), but some of the rhetoric on this topic is just so absurdly ridiculous, and can only be drawn from the delusions of self-serving fantasy.

The denial of the reality of overselling and fair use is how we end up with traffic shaping, or stupid constraints like cellular carriers refusing to allow uses like tethering (which is because they rely upon most of their customers not using their "alloted" throughput, and tethering would completely throw off that assumption. I'd rather they base all of it around actual usage and cease trying to indirectly massage usage through unnecessary secondary restrictions and throttles.

The standard tripe about how one’s usage somehow only ever uses excess capacity (is there some “only if available” NoQuality-of-Service bit?), where people assume that their actions somehow exist in a vacuum of inconsequence, unbound by the limits or constraints of anyone else, offends common sense.

  internet   networking   morons 

Reader Comments

It is nice to hear someone talking about this in a reasonable way. Perhaps I figured this out when my first DSL circuit (jump.net in Austin in the late 90s) was metered. I certainly did by the time I had worked for nearly a decade at a mom-and-pop ISP.

Cheap, fast, reliable: pick 2.

I squirm every time I hear about 100+ mbit unlimited service for 100USD/mo. There's a part of me that wants it to be true. There's a part of me that rationalizes it by saying it's hard to saturate that fat a pipe (even with p2p). Akami &c help some, as do wireless laptop's lower throughput caps... But mostly I just assume the users of the service won't be getting what they think they are in terms of performance & customer service.
Matt Sayler @ 6/20/2009 1:48:28 AM
I'd like to see what sort of costs ISPs have to bear to hook up to the backbones. Or what the bigger backbones have to pay other backbones when their traffic isn't symmetrical, which it isn't for all but a couple of the hugest providers.

In the early web, a lot of ISPs talked about local repositories of common files, and you could access these relatively local resources in a truly unlimited fashion. It would be nice if more of the infrastructure could be distributed in such a way. Sort of a super-Akamai.
John S @ 6/20/2009 8:15:28 AM
Great post. Thanks.
Fan @ 6/20/2009 9:45:05 AM
"In the early web, a lot of ISPs talked about local repositories of common files, and you could access these relatively local resources in a truly unlimited fashion. It would be nice if more of the infrastructure could be distributed in such a way. Sort of a super-Akamai."

Can you say anycast bittorrent? I knew you could!
Matt Sayler @ 6/20/2009 12:07:49 PM
Thanks for the great article, Dennis. You are a rare voice of reason.
George Taka @ 6/20/2009 3:43:07 PM

Add Comment

Name *:

Email Address:

(your email address is not displayed)
Website:

Comment *:


Dennis Forbes