I was leafing through yesterday's dead-tree print edition of the National Post when an article afront the "FP Working" section caught my eye: It looked to be a piece on immigration, hinted at by the accompanying graphic of a shipping box containing several goofy-looking "techie" caricatures. The box was drawn with the label "URGENT, SHIP OVERNIGHT TO CANADA", and was emblazoned with the flag of India.

Given that Canada has the highest per-capita immigration rate in the world, articles on the topic are often thought-provoking and informative, and usually serve as worthwhile fodder for debate, so I gave it a closer look.

Disappointingly, it turned out instead to be a hilarious piece about offshoring[*1], written with such wide-eyed naivety that I had to check the date on the paper to see if I accidentally pulled out one from half a decade ago, when this sort of "win/win!" nonsense was even remotely believable.

Without tearing apart the numerous fundamental errors in this terrible article[*2] (oh, but everyone is doing it, the article claims, so surely it must be true?), I'd rather simply point you to Paul Graham's excellent essay on covert public relations via submarine PR.

If the author (and/or the paper) didn't get a handsome kickback, they were robbed. I was surprized (and dismayed) that there weren't "advertorial!" disclaimers atop it.

*1 - And I'm anything but an isolationist, and entirely believe in globalization. Apart from minor transition hiccups, the end result is the enrichment of everyone.

*2 - In the case of offshoring, the early "cheapness" quickly faded as knowledge workers in India and China decided that their lot in life wasn't just to be producers for the West, but rather that they too wanted a chance to "consume". Wages have been increasing at a torrential pace, and getting anyone anywhere above terribly incompetent has been described as close to impossible (while there are huge numbers of great talents, there are also huge numbers of potential employers. Bob's Oil Cleaning is going to get the absolute dregs of the dregs of the dregs working on their app - and that's ignoring the ridiculous contention that having the app developed during the twilight hours is somehow advantageous for an industry that hasn't even come to grips with telecommuting - leading to quality problems that are becoming a serious concern in the industry)

Some time back I wrote a brief entry regarding the adoption of products. In it, I made the blaringly obvious observation that many products that seem to be revolutionary, and that have taken the market by storm, really just made existing products or technologies slightly easier to use, or slightly more useful (as amazing and technologically remarkable an iPod is, for most users it's functionally equivalent to a 1980 Walkman).

It's a better mousetrap model that has driven business, and consumerism, for decades.

In reverse, making something a little less convenient, and a little less accessible, works effectively at avoiding undesired behaviour in a target audience.

Anti-piracy efforts, for instance, have never been pursued with expectations of absolute success, and it really hasn't broken their model when someone sits in IRC #warez channels all day, and then puts their PC at risk of spyware/trojans/viruses with cracks and serial gens. It's the other 99% of the population that's the target of low-barrier anti-piracy technologies. Those are the people who would rather just pay $49.95 at the computer store than waste the time or take the risks.

Authorization, serial numbers, machine keying -- all of these are intended to make it just a little more of a hassle to use unauthorized copies, decreasing the casual piracy of normal people. Of course sometimes it backfires, and the anti-piracy techniques are more of a pain than the alternative, but that's another story.

Manipulating "ease of use" can work for self-control as well. A common bit of wisdom for those looking to pursue healthy eating is "avoid it once at the store, or avoid it countless times at home" -- If you can stop yourself from buying a bag of cookies or box of ding dongs at the grocery store, the adage goes, that one exercise of self-control will save from having to use restraint countless times as said treats sit on your shelf, begging to be consumed. Sure, you could just hop in your car and go buy a box of ding dongs when the munchies hit, but for many people the desire is low enough that it isn't worth the trouble, and you either go without or choose something healthier.

This sort of "front-end self control" came to mind today as I analyzed the things that work, and the things that don't, in my weekly online adventures: Being in software development of course means consuming the news and information in the industry, and conversing (and hopefully debating) with informed, interesting people who have an enlightened point of view. The hope is to consume valuable, worthwhile information, and to engage in conversations that leave me feeling a little more knowledgeable.

On my web adventures, the things that work are those that move me towards goals, help my understanding of industry technologies and trends, or even just entertain me (all work is a recipe for trouble, and a funny YouTube video or The Onion article every now and then is very beneficial for productivity).

The things that don't are basically everything else, which is a set usually comprised of sites that I visit almost reflexively: Sites that once had merit for me personally, but no longer do (perhaps "we've grown apart", and they're at a technology level or scope that I'm not really interested in at this point, or perhaps their content has gone from quality to garbage), but I still find them sitting in my bookmarks listing, usually with shortcut keys.

I habitually find myself typing their URL without even really thinking about it. I'm human, and thus a creature of habit. Once there I'm invariably sucked into unfulfilling content, or annoying, unfulfilling debates.

Yet, while these sites have limited utility for me now, their "ease of use" is extraordinarily high simply as a function of acclimation and habit.

So, much like avoiding the bag of cookies at the grocery store, I've enacted some simple controls to make it just a little more of a hassle to visit them.

• I've "banned" them in Adblock. This basically means that visiting them in Firefox -- my browser of choice -- requires reconfiguring the Adblock rules, marginally decreasing the ease of use. Sure, I could just jump over to another browser, but losing the enormous functionality and benefits of Firefox with my select set of Extensions dramatically lowers the "ease of use" factor, eliminating any marginal interest in visiting these sites.
• I've "banned" them in my router. Most routers nowadays include L5-L7 filtering, allowing it to inspect and block requests for certain domains/URLs. I go in this section so infrequently, and the interface is so terrible, that it smashes ease of use to the ground, despite acclimitization and habit.

Of course there are many ways that I can circumvent this, most directly by just turning off the self-imposed "restrictions", but that's missing the point - that's like hopping in my car and driving to the grocery store because I feel like a cookie. It isn't going to happen simply because the functionality provided is far too low to offset the nuisance of getting there.

I've never been a fan of The Daily WTF. 

While I can appreciate that it might elicit an occasional chuckle, and may even serve as a "here be dragons!" warning of the dangers of bad code or UIs, something about it just rubs me the wrong way.

The root of my dislike, I think, is my feeling that there's a bit too much of a schadenfreude thing going on much of the time. While I'm sure that the site is run and visited by a lot of great people, many of the regulars seem a little too eager to bask in the foibles of others, presumably imagining themselves righteous fonts of perfection and clarity.

Even the most benign and negligible choice of bitmasks is apparently worthy of mockery.

That would be fine and good, and it would just be a site I don't visit, however a bit of a meme lately (given that there's a current SuperStar! Developer! thing going on) is something along the lines of "Do you want a great programmer, or someone whose code appears on TheDailyWTF?" This is a repeating theme: On the one side are the great programmers, and on the other are the people endlessly bound to give TheDailyWTF source material.

Do people really think such a schism exists? Is the impression that great developers are infallible, never creating any bad code at all, ever? Are bad programmers just stumbling from one WTF to another?

Of course not.

I fear the output of any developer who claimed that they've never written bad code. I would fear them because they're either bald-faced liars -- believing that simply saying it repeatedly will somehow convince others into this fiction -- or they're completely blind to their own weaknesses.

Every developer in the real world has had bad days, brain faults, or bad interpretations of new languages, environments or libraries. It's simply a given of the profession.

Building a myth of perfection fools no one.

Joel the Troll?

Joel Spolsky, the well-known blogger and ISV owner, kicked up quite a storm recently with his piece entitled Language Wars [for those following the `debate', yes, I'm late to the party on this. I make it a general standard to avoid responding to blogs on here -- the whole blog thing is entirely too recursive -- but some recent reactions to his piece pushed me to post].

The article leads off with some pragmatic wisdom, advising enterprise-y, low-risk type shops to use well-known and well-proven technology stacks -- solid advice that's hard to argue with -- yet he then ends the piece with a comment about an in-house, next-generation, super-duper language being used to develop FogCreek's premiere product, FogBugz.

The discord was so great that most readers presumed that the Wasabi thing was a joke, or alternately that the rest of the article was the joke (which would have been an awesome revelation). Much confusion ensued, to the point that Joel had to put up a post clarifying that he was actually serious about the Wasabi thing

Like Sharks, only with Ruby LASERs On Their Heads!

Aside from the seeming hypocrisy, what really instantiated some JoelCritic<T> instances (via the BlogCriticFactory) were Joel's comments about Ruby, where he seemingly indicated that it wasn't ready for prime time.

...but for Serious Business Stuff you really must recognize that there just isn't a lot of experience in the world building big mission critical web systems in Ruby on Rails, and I'm really not sure that you won't hit scaling problems, or problems interfacing with some old legacy thingamabob, or problems finding programmers who can understand the code, or whatnot...

...I for one am scared of Ruby because (1) it displays a stunning antipathy towards Unicode and (2) it's known to be slow, so if you become The Next MySpace, you'll be buying 5 times as many boxes as the .NET guy down the hall.

I'm sure Joel anticipated the backlash. Perhaps it was even the motivation behind the posting: The resulting torrent of discussion brought quite a few visitors to his blog, and earned him a lot of inbound links, both of which have definitely helped with his new business ventures. No publicity is bad publicity, they say, especially if it's timed to coincide with the launch of a new job board (as an aside, Ruby, Wikipedia, OSX, Python, Lisp, and ERLang are all terrible! People with the letters J or P in their names are jerks!).

Ruby is still new enough, and with a small enough community, that many of its users double as evangelists -- think of the Amiga computer, the BeOS operating system, or any other contextually-superior alternative embraced by a small enough group that many feel an ego-intersection with the technology, motivated to defend and advocate it when the opportunity arises. Linux once had such an attack-dog core of rabid enthusiasts, though as the user base has grown, and it has become more pedestrian, you really have to target a Linux-niche (such as a little used distro) if you're aiming to stir up a hornet's nest.

That entire lead-up was just some context for the actual topic of this entry: So-called premature optimization.

On Premature Optimization

A common response to Joel's complaint that Ruby is slow or resource inefficient is the frequently incanted declaration that such complaints are nothing but "premature optimization!"

I've seen the same deflection shield used to defend abhorrent database designs, convoluted, overly-abstracted class designs or message patterns, and virtually anything else where a realist might proactively ponder "but won't performance be a problem doing it like this?", only to yield the response "You know, premature optimization is a classic beginners mistake!"

If you don't want to be lumped in with beginners, the lesson goes, it's best to pretend that performance simply doesn't matter. We'll cross that bridge when we get to it.

Premature optimization is the root of all evil (or at least most of it) in programming.

Donald Knuth

I remember the early days: I once spent about 16 work hours optimizing a date munging function, increasing its performance from something like 2 million iterations per second to 4 million iterations. In the grand scheme of things, the performance difference was completely negligible, but from the perspective of artificial benchmarks it seemed like tremendous progress was being made.

That was premature optimization.

Indeed, anyone who's done time in the software development industry can identify with what Mr. Knuth was saying, probably having been involved with (or responsible for) project plans gone awry when efforts focused on highly-complex caching infrastructures, or ultra-optimizing some seldom used edge function.

Yet what is arguable, and situation specific, is deciding what qualifies as premature, versus what is simply proactive, predictive, professional performance prognostications.

NOT ALL PERFORMANCE CONSIDERATIONS ARE PREMATURE OPTIMIZATION!

While there is no doubt that there is such a thing as premature optimization -- it is an evil distraction that sidetracks many projects -- there are critical decisions made early in a project that can cripple the performance potential (both resource efficiency, and resource maximum), making later optimizations enormously expensive, if not impossible without an entire rewrite.

Whether it's heavily normalizing the database (or its nefarious doppelgänger, the classic database-within-the-database: "This single table can handle anything! Just put a comma separated array of serialized objects in each of the 256 varbinary(max) columns! Look at the flexibility! Query it? Don't you bother me with your premature optimizations!"), creating an application design that's incongruent with caching, or choosing an inefficient platform.

There are credible performance considerations that need to be addressed at the outset, and revisited as development proceeds. It is absolute insanity, and entirely irresponsible professionally, to simply stick one's head in the sand and hope that some magical virtual machine improvements or subcolumn indexing decomposition and querying technology will occur before deployment, or before the economics of scaling come into play.

And speaking of scaling, the canard that the horizontal-scalabilty intrinsic with most web apps (unless you really screwed up the design -- as many people do -- and made horizontal scalability impossible) makes the problem a nonissue is absurd: Perhaps if your project has a high transaction value then you have the luxury of adding more servers to serve a small number of clients, yet for most real-world projects adding resources is a big, big deal. And it isn't simply the cost of a low-end Dell 1850: Whether you're colocating or hosting in an expensively rigged corporate server room, the cost of each server is substantial.

You end up in the dilemma that you're financially (or physically) limited to a set quantity of resources, having to limit or scale-back the functionality provided to each user due to the inefficiencies caused by early decisions. "Sorry we can't implement that cool AJAX type-ahead lookups because the callbacks would kill our servers - we're already saturating them with our stack of inefficiency, so there's no overhead left."

I think the lackadaisical attitude towards efficiency is a result of experience derived from countless unvisited or seldom used web apps deployed across millions of PCs, colocated with equally as spartanly used peers. When a site sees a dozen visitors in a day, it's easy to declare that performance is a seeming nonissue nowadays - that it's only a concern for game programmers and nuclear modelling engineers. Then one day the page gets mentioned on Digg or Reddit or Slashdot or BobOnHardware and in that potential moment of glory the app falls over and dies, again and again.

None of this really has anything to do with Ruby. Personally I haven't used it beyond the tutorials, though I do know that it does very, very poorly on the standardized benchmarks. However it is distressing seeing so many people dismiss Joel's comments (or comments about Python, or ERlang, or XML, or any other technology) as premature optimization.

Why is it that "90% done" (and its partner in crime - the ubiquitous "almost done!") is the progress report for virtually any project, over virtually all of its life-cycle?

Why has 90% become the fictional number of choice? Why not the more conservative 80%, or the bolder 95%? Given that it usually has little correlation with reality, they're just as real.

Projects should be reported as 87% done. Even when there's the ominous "we'll solve that problem when we get to it" task maliciously eyeballing you from later in the project plan, or the "it doesn't work and we have no clue why?" runtme reality, still say 87% with confidence and pride.

I started blogging on September 4th of last year.

I had an internet presence prior to that (content that received several Slashdot mentions, along with a half-decent number of inbound links), but I didn't put up content with the regularity of a blog -- largely as a function of the hassle involved -- and I didn't have RSS, Atom, or any other feed technology (and thus wasn't aggregated into other feeds).

It was just a random hodge-podge of random pages prior to putting it into the structured form you see today.

All-in-all the past year has been a very, very rewarding experience: A very credible number of people visit, and from a search-engine perspective the results have been extraordinarily successful. Strange seeing several dozen people a day from my hometown coming by just because I happened to mention it in a blog entry.

To quote from a September 4th entry-

The question I am pondering, then, is whether the only way one can remain internet credible (in search engine terms) is to integrate heavily within the blogging community, quid-pro-quoing endless links and trackbacks, ingratiating oneself with other bloggers, posting meaningless comments about every posting every other blogger makes (which they will of course do in turn). It's a sort of super-pyramid scheme, but with no bottom level.

Thankfully I've never had to quid-pro-quo or ingratiate to maintain PageRank. In fact I think I've maintained a fairly antagonistic approach to many of the popular blogs and bloggers, and I've seldom resorted to inventing "material" out of mentioning other blogs.

Which brings up an interesting topic - I was chatting with a peer about blogging and the effort/reward ratio, and they asked if I felt that I had "succeeded" in this venture: Sure, they pondered, I'd gotten a lot of mentions, along with a couple of heavily visited pages, but overall I still sit quite low on the list of bloggers. My Alexa rankings stink (though I should mention that Alexa rankings are laughably useless outside of the top internet sites. Alexa ratings are culled from users utilizing the Alexa or A9 toolbars, which is a vanishingly small number of users, clustered into certain demographics. Just a couple of users occasionally visiting with the toolbar has an absurdly large impact, so if I wanted to shoot up in the rankings, I'd just recommend the toolbar every month. As a case-in-point, at one point I noticed that my Alexa ranking had jumped considerably, but became suspicious that a disproportionate number of visitors visited the webstats page...which of course only I visit. I realized it was me that was inadvertently impacting the rankings when I had installed the A9 toolbar, so I removed it), and I'm not even among the top 1000 bloggers (by one metric I'm #5,269).

I have something like 118 bloglines subscribers, versus say 21,000 for someone like JoelOnSoftware (bloglines is only one of many aggregators, and Joel has far more subscribers overall, but it's a metric that is meaningful in a relative sense).

Yet I am thankful for every single reader, and the success of this blog is worlds beyond what I imagined. More important than quantity is quality, and some of the feedback leads me to believe that a great group of people have decided to drop by every now and then (even though many don't use feed readers, and just added it to their bookmarks for a once-a-month browse. That's the same technique I use for most blogs). Sure, complimenting your readers is a suspect activity, and is often driven by egotism above all else, but I really mean it: I couldn't have asked for a better readership.

And perhaps this will come off as cheap or like sour-grapes (which it most certainly isn't -- I set out expecting an occasionally accidental search visitor, and never anticipated the success this has seen), but there are some pretty easy ways I could have modified the message a bit to build and maintain a much greater blog presence, but that wasn't my goal.

I could...

• Blog more frequently. As it is, with two pre-school children, it's ridiculously difficult finding time to get posts in, but sometimes I just have to get a thought out there so it's a great exercise.
  
• Blog a more consistent message. No thanks. One of the things I love about this is that I can blog about a .NET video codec one day, Wikimedia on Windows the day after, HTML compression the next, SSIS packages the next, and my opinion about work environments after that.
  
  Occasionally I hestitate, wondering "will the people who subscribed after {X} hit the front-page on Reddit really care for this?" -- for instance when putting up navel-gazing entries like this -- but then I realize that's getting caught in the classic trap of limiting oneself to a narrow range of topics. With readers and browsers, people can just hop past things they aren't interested in, and while I'm sad for anyone who unsubscribes or /dev/nulls this blog, that's preferrable to diluting it to a serve-everyone-but-really-noone message.
  
• Blog a more generalist message. I certainly don't mean to be elitist with some of the entries, but it's the nature of the beast that some of them aren't going to entertain or cater to generalists or technical tourists. Compare this to almost all of the top blogs in this space -- apart from a couple of very rare exceptions, most seldom venture outside of the realm of easy, accessible observations and pondering. The "colour of the shed" sort of entries that everyone can add their voice to the chorus with their opinion.
  
• Blog towards retention. This is really achieved by keeping a general, accessible, non-threatening topic going, consistently pounding the same theme, or by creating controversy and debate where none really exist, but it's also built by minimizing the number of outbound links, and maximizing the number of internal links. This has never motivated the way I author posts, and if I lose a small percentage of users with each outbound link as they go off exploring Wikipedia or Seth Godin or anything else, that's something I'm very happy about. Having said that, the number of internal links seems to be increasing on here as of late -- as I've built a larger and larger volume of content, it just seems like I'm a pretty good resource to reference!
  

None of these techniques are secrets, but they're only acceptable modus operandi if your primary goal is, well, blogging. That isn't my primary goal by a long shot, and I have no ambitions of becoming a professional blogger. Instead I'm motived to talk to, and hopefully influence -- and maybe even impress -- intelligent and influential people.

In the coming few months (or more correctly weeks) I have several very, very exciting things that are going to come out, including the most exciting and innovative web idea I've ever had. It's only going to get better.

But I'll never compromise the message, and I'll never let metrics and stats give me misdirected motivation.

The latest Community Technology preview of Visual Studio "Orcas" has been released (see Rob Caron's entry), coming in the form of a configured and ready to run virtual machine (while normally VMWare Player and VMWare Virtual Server can import VirtualPC machines, that doesn't seem to be the case for this release. I'll probe further on that, though at least VirtualPC is free).

Note: you also need the base virtual machine disk, which you can find here. Together it's some 4.6GB of downloads -- I wonder what Microsoft's bandwidth bills are like -- though apparently they'll be using diffs from here on in.

Not overly noteworthy in itself, given that this is a product that is so far out that it's of little practical interest for professional developers (outside of saying "neat!", or if you're in the Visual Studio add-in market and you need to develop for it concurrently), however I mention it because I wrote an entry about this form of delivery a while back, and I think it's a neat development.

Obviously Microsoft has the luxury of releasing their own products on their own virtualized platform without all of the legal issues that an external company would have doing the same. Perhaps at some point Microsoft will release a special appliance version of their server products, allowing 3rd parties to release products on a decked-out virtual appliance running, for instance, Windows 2003, and to do so legally and cost-effectively (they sort of do this with the Storage editions).

This segues to another topic topic that I recently touched upon, which is Joel and his Wasabi language, which is a really high level language (so high that it generates code for really high level languages) that he created to generate builds targeting multiple platforms (e.g. PHP on Apache on Linux, VBScript on IIS on Windows, etc). Joel might be better served simply setting up the platform he needs, and then using all of the functionality and capabilities of the chosen technology stack, and then delivering a ready to run virtual machine to his prospective clients.