The well-known Hanlon's razor states-
Never attribute to malice that which can be adequately explained by stupidity.
While it's a seemingly pessimistic perspective on the capacity of one's fellow human, it is an undeniable truth that we often mistake carelessness, thoughtlessness, or outright ignorance as malicious intent.
Yet it's a more serene existence -- to the benefit of one's lifespan -- to simply assume that the person who dangerously cut you off on the road, for instance, is just a moron deserving a bit of sympathy, rather than considering him or her a roadway foe challenging you to a deadly battle of wills.
From a software development perspective, however, I think an inverted variation would serve the industry well.
Never blame others until adequately considering the possibility of your own (negligence | carelessness | stupidity).
As a general rule, the denizens of the software development profession -- it certainly isn't limited to this profession, but given that it's the general focus of this blog it's the one I comment upon -- have a tremendous capacity for assuming the worst of others, far before considering the unsavoury prospect that maybe -- just maybe -- it's actually their own mistake or lack of knowledge that's the cause of the issues they face.
It is far too common to cast a wide net of blame, declaring that Microsoft's products are screwing up, the documentation is all wrong, the server is malfunctioning (maybe because of cosmic ray particles toggling memory bits), the installation tool is a dud, and one's coworkers are surely idiots insidiously and maliciously changing code just to make one's brilliant code poetry fail to achieve its momentous glory.
After such hand-waving, blame-weaving dramatics, in most cases the developer realizes that they skipped an obvious step in the instructions, or they forgot to get latest of the entire branch, or they were copying the wrong file or looking at the wrong folder or running the wrong executable, or they were using the class entirely wrong, or they completely misunderstood how the operating system security system works, or they set a global setting a week back that completely changed how the application functions, or they ignored the email and documentation and group meeting detailing system changes, and so on.
They quietly retreat -- don't expect a retraction -- until they repeat the same mistake the next time something doesn't go exactly as imagined.
I've met these people in the industry. I've worked with these people. I've been one of these people.
I think we can all relate to situations where we've railed against a company, a product or a person, only to have the embarrassing realization that we were simply doing something dumb.
And it's not even that doing something dumb is noteworthy: We're humans, and we're bound to make mistakes. The problem is that we often don't even give a moment of time to even the possibility that we could be at fault, instead just assuming the worst of others.
It's far more beneficial to both productivity and team morale to have a little bit of self-doubt in these situations: Assume the worst of yourself before assuming the worst in others.
After hashing out this entry, I wondered why it wasn't appearing on the public blog. After berating various products and services, I remembered that I recently outsourced my DNS (for the reasons described here, using the service recommended by a reader), and forgot to add an entry for the FTP server. Whoops.
As a completely offtopic aside, one of the reasons I switched DNS providers was to have support for a domain SPF record. While it does nothing to stop the tide of pump-and-dump investment scam spams, at least it allows those recipients utilizing the service to immediately dump-bin those that claim to come from yafla (I get about 100 bounces a day, and who knows how more actually get through), knowing that the from: address was forged.
We -- specifically the humans among us -- are a very uncreative bunch.
While we excel at refining existing ideas and solutions, we're extraordinarily poor at creative, original thought.
And I'm not just talking about business, scientific or engineering creativity: Even in the realm of the creative arts (writing, art, music), many practitioners are simply absorbing (being "inspired by") what others are doing and then making it their own.
In a way we're all unintentional, subconscious plagiarists, and while we often don't even realize it, many of our "original" ideas are nothing more than refinements (or degradations) of those that we've come across elsewhere at some point in our time on this Earth.
Other times it's completely accidental.
Many of the great inventions in science, mathematics, medicine, and healthcare, for instance, weren't scientifically pursued with an engineering discipline, but were rather accidental discoveries (usually stumbled upon while searching for a slightly better way of building a different mousetrap).
And sometimes confused communications yields innovation.
Many of my best ideas weren't the result of quiet time in deep contemplation -- in fact I've found intentional, focused thought to be terribly unproductive -- but rather were the wonderful benefit of the bad habit of not paying complete attention to presentations or media.
Quite a few times I've glanced over an article or casually listened to a presentation while distracted; drawing inferences and conclusions based upon what I think is being presented to me. As my mind casts it into a workable solution, I've often found that what I visualized bears little correlation with what was actually described. A few inputs, and the presumption that it's a logical solution, fooled my brain into creating new solutions out of nowhere, in a way that I could never have done otherwise.
This, I suppose, is the foundation for brainstorming sessions. It's the reason why it's often productive to randomize all sorts of product or business ideas and them merge them into seemingly bizarre combinations (I created this "How Will You Get Rich?" tool as a joke, but was surprized to find me actually stopping on some of the supposedly joke suggestions, imaginary scenarios where they would work).
How do you
innovate? How do you engage the lateral thinking part of your
brain, thinking up creative and novel solutions to
problems?
The latest Community Technology preview of Visual Studio "Orcas" has been released (see Rob Caron's entry), coming in the form of a configured and ready to run virtual machine (while normally VMWare Player and VMWare Virtual Server can import VirtualPC machines, that doesn't seem to be the case for this release. I'll probe further on that, though at least VirtualPC is free).
Note: you also need the base virtual machine disk, which you can find here. Together it's some 4.6GB of downloads -- I wonder what Microsoft's bandwidth bills are like -- though apparently they'll be using diffs from here on in.
Not overly noteworthy in itself, given that this is a product that is so far out that it's of little practical interest for professional developers (outside of saying "neat!", or if you're in the Visual Studio add-in market and you need to develop for it concurrently), however I mention it because I wrote an entry about this form of delivery a while back, and I think it's a neat development.
Obviously Microsoft has the luxury of releasing their own products on their own virtualized platform without all of the legal issues that an external company would have doing the same. Perhaps at some point Microsoft will release a special appliance version of their server products, allowing 3rd parties to release products on a decked-out virtual appliance running, for instance, Windows 2003, and to do so legally and cost-effectively (they sort of do this with the Storage editions).
This segues to another topic topic that I recently touched upon, which is Joel and his Wasabi language, which is a really high level language (so high that it generates code for really high level languages) that he created to generate builds targeting multiple platforms (e.g. PHP on Apache on Linux, VBScript on IIS on Windows, etc). Joel might be better served simply setting up the platform he needs, and then using all of the functionality and capabilities of the chosen technology stack, and then delivering a ready to run virtual machine to his prospective clients.
I started blogging on September 4th of last year.
I had an internet
presence prior to that (content that received several Slashdot
mentions, along with a half-decent number of inbound links),
but I didn't put up content with the regularity of a blog -- largely as a
function of the hassle involved -- and I didn't
have RSS, Atom, or any other feed technology (and thus wasn't
aggregated into other feeds).
It was just a random hodge-podge of random pages prior to putting it into the structured form you see today.
All-in-all the past year has been a very, very rewarding experience: A very credible number of people visit, and from a search-engine perspective the results have been extraordinarily successful. Strange seeing several dozen people a day from my hometown coming by just because I happened to mention it in a blog entry.
To quote from a September 4th entry-
The question I am pondering, then, is whether the only way one can remain internet credible (in search engine terms) is to integrate heavily within the blogging community, quid-pro-quoing endless links and trackbacks, ingratiating oneself with other bloggers, posting meaningless comments about every posting every other blogger makes (which they will of course do in turn). It's a sort of super-pyramid scheme, but with no bottom level.
Thankfully I've never had to quid-pro-quo or ingratiate to maintain PageRank. In fact I think I've maintained a fairly antagonistic approach to many of the popular blogs and bloggers, and I've seldom resorted to inventing "material" out of mentioning other blogs.
Which brings up an interesting topic - I was chatting with a
peer about blogging and the effort/reward ratio, and they asked if
I felt that I had "succeeded" in this venture: Sure, they
pondered, I'd gotten a lot of mentions, along with a couple of
heavily visited
pages, but overall I still sit quite low on the list 
of bloggers. My Alexa rankings
stink (though I should mention that Alexa rankings are
laughably useless outside of the top internet sites.
Alexa ratings are culled from users utilizing the Alexa or A9
toolbars, which is a vanishingly small number of users, clustered
into certain demographics. Just a couple of users occasionally
visiting with the toolbar has an absurdly large impact, so if I
wanted to shoot up in the rankings, I'd just recommend the toolbar
every month. As a case-in-point, at one point I noticed that my
Alexa ranking had jumped considerably, but became suspicious that a
disproportionate number of visitors visited the webstats
page...which of course only I visit. I realized it was me that was
inadvertently impacting the rankings when I had installed the A9
toolbar, so I removed it), and I'm not even among the top 1000
bloggers (by one metric I'm #5,269).
I have something like 118 bloglines subscribers, versus say 21,000 for someone like JoelOnSoftware (bloglines is only one of many aggregators, and Joel has far more subscribers overall, but it's a metric that is meaningful in a relative sense).
Yet I am thankful for every single reader, and the success of this blog is worlds beyond what I imagined. More important than quantity is quality, and some of the feedback leads me to believe that a great group of people have decided to drop by every now and then (even though many don't use feed readers, and just added it to their bookmarks for a once-a-month browse. That's the same technique I use for most blogs). Sure, complimenting your readers is a suspect activity, and is often driven by egotism above all else, but I really mean it: I couldn't have asked for a better readership.
And perhaps this will come off as cheap or like sour-grapes (which it most certainly isn't -- I set out expecting an occasionally accidental search visitor, and never anticipated the success this has seen), but there are some pretty easy ways I could have modified the message a bit to build and maintain a much greater blog presence, but that wasn't my goal.
I could...
None of these techniques are secrets, but they're only acceptable modus operandi if your primary goal is, well, blogging. That isn't my primary goal by a long shot, and I have no ambitions of becoming a professional blogger. Instead I'm motived to talk to, and hopefully influence -- and maybe even impress -- intelligent and influential people.
In the coming few months (or more correctly weeks) I have several very, very exciting things that are going to come out, including the most exciting and innovative web idea I've ever had. It's only going to get better.
But I'll never compromise the message, and I'll never let metrics and stats give me misdirected motivation.
While I've used Linux for years in the virtual machine space, primarily as a "native" location for many of the UNIX-style command line tools (even if I'm using them against an SMB share on the network, they're still extremely helpful), it has never been a primary operating system on any of my PCs outside of novelty "let's see what stage they're at" usage. Where I did use a Unix variant, it was always FreeBSD.
I've decided to give it another go, this time in an "Internet Appliance" of sorts.
This opportunity arose when a hard drive in an old eMachines Athlon 2400+ 512MB WXGA laptop I had kicking around died. It was running warmer and warmer until finally it stopped running altogether. Now it's just lounging on a park bench.
Given that the laptop saw
limited use -- mostly for my daughter to play PeepAndTheBigWideWorld
games online -- this didn't cause too many tears, and it gives me
an opportunity to pursue the internet
appliance desire (perhaps in the kitchen. Something that is
zero maintenance or security worry, hopefully low power, and that
allows for rich web browsing for the family and guests) that I've
been thinking about for some time. What I'm considering is running
the laptop diskless (meaning no hard drive or floppy drive, though
there will be an optical drive -- unfortunately this particular
laptop - an M5312 - can't boot off of the USB key, though I'm
keeping my eye out for possible bios options. Note that I do
not want a hard drive, so I'm not looking for replacement
options for it. This is a fun effort, and as I already have another
working modern laptop, I just think a diskless device, minus the
noise, heat and power consumption of the hard drive, would be
pretty cool).
After some futile efforts with Knoppix, my initiatives thwarted by the fact that it refused to work with the Broadcom wireless (either with the native driver or with the ndiswrapper driver), I switched over to Ubuntu's "LiveCD" desktop version and it works amazingly (though the wireless worked much better with ndiswrapper and the Windows Broadcom bcm43xx driver). With little hassle the machine is booted from the CD, the wireless is operating (with WPA and everything), Firefox is updated with Shockwave and various updates, and it's brilliantly usable with complete mobility throughout the home, low power, low noise, and limited heat. I could duct-tape it to a wall if I wanted to.
The only problem is that it's a temporal state, and once I shut down (or there's a power outage), I need to start from the delta of the CD image once again. The latest release of Ubuntu does have a rather sketchy persistent option where, with a bootup option (which is lame), it basically loads everything from the CD, overlays the contents of a USB key image, so you're still starting from the base and then consuming some of the ghost filesystem with the delta. This is on top of the fact that the persistent functionality uses the USB key almost like a R/W filesystem, purportedly constantly writing changes, which would lead to a very short key lifespan. Also I really don't want to save continually changing state, but rather want to choose a perfectly configured point and solidfy it, with each reboot starting exactly there.
What I'd really like to do is to configure the machine and somehow persist that onto a new boot CD, such that the starting state is exactly what I want (realizing that in the future, I may need to make newly updated CDs). Indeed, it is possible to make your own Ubuntu bootup CDs, and I've successfully done so to some success. However I need to get the changes from this live, no-persistent storage laptop onto the image for burning onto CD (many of the configuration steps are making changes unknown to me, so I can't just modify a couple of /etc .conf files). So I've got a Ubuntu virtual session running*, and I've extracted the boot-image filesystem, but it's the process of getting the laptop image over to the image for burning that I'm unsure of. I've tried rsync to dismal failure.
If anyone has any ideas, I'd greatly welcome them. The laptop has a 1GB USB key which it can access, and working wired and wireless connectivity. On the other end is a Ubuntu session with plenty of RAM and storage, ready with a decompressed image to be prepared for burning into the ultimate, personalized LiveCD.
* - the process described to make a custom Ubuntu boot CD -- such as creating a filesystem within a file, and then copying over from the compressed filesystem, completely borked VirtualPC...multiple times. Not to mention that performance in general was atrocious. I then tried under VMWare Server (the free product) and it worked absolutely perfectly, and the performance was enormously faster.
Normally I'm a fan of VirtualPC, but from here on in I'll be doing all Linux work under VMware.
Why is it that "90% done" (and
its partner in crime - the ubiquitous "almost done!") is
the progress report for virtually any project, over virtually all
of its life-cycle?
Why has 90% become the fictional number of choice? Why not the more conservative 80%, or the bolder 95%? Given that it usually has little correlation with reality, they're just as real.
Projects should be reported as 87% done. Even when there's the ominous "we'll solve that problem when we get to it" task maliciously eyeballing you from later in the project plan, or the "it doesn't work and we have no clue why?" runtme reality, still say 87% with confidence and pride.
Joel Spolsky, the well-known blogger and ISV owner, kicked up quite a storm recently with his piece entitled Language Wars [for those following the `debate', yes, I'm late to the party on this. I make it a general standard to avoid responding to blogs on here -- the whole blog thing is entirely too recursive -- but some recent reactions to his piece pushed me to post].
The article leads off with some pragmatic wisdom, advising enterprise-y, low-risk type shops to use well-known and well-proven technology stacks -- solid advice that's hard to argue with -- yet he then ends the piece with a comment about an in-house, next-generation, super-duper language being used to develop FogCreek's premiere product, FogBugz.
The discord was so great that most readers presumed that the Wasabi thing was a joke, or alternately that the rest of the article was the joke (which would have been an awesome revelation). Much confusion ensued, to the point that Joel had to put up a post clarifying that he was actually serious about the Wasabi thing
Aside from the seeming hypocrisy, what really instantiated some JoelCritic<T> instances (via the BlogCriticFactory) were Joel's comments about Ruby, where he seemingly indicated that it wasn't ready for prime time.
...but for Serious Business Stuff you really must recognize that there just isn't a lot of experience in the world building big mission critical web systems in Ruby on Rails, and I'm really not sure that you won't hit scaling problems, or problems interfacing with some old legacy thingamabob, or problems finding programmers who can understand the code, or whatnot...
...I for one am scared of Ruby because (1) it displays a stunning antipathy towards Unicode and (2) it's known to be slow, so if you become The Next MySpace, you'll be buying 5 times as many boxes as the .NET guy down the hall.
I'm sure Joel anticipated the backlash. Perhaps it was even the motivation behind the posting: The resulting torrent of discussion brought quite a few visitors to his blog, and earned him a lot of inbound links, both of which have definitely helped with his new business ventures. No publicity is bad publicity, they say, especially if it's timed to coincide with the launch of a new job board (as an aside, Ruby, Wikipedia, OSX, Python, Lisp, and ERLang are all terrible! People with the letters J or P in their names are jerks!).
Ruby is still new enough, and with a small enough community, that many of its users double as evangelists -- think of the Amiga computer, the BeOS operating system, or any other contextually-superior alternative embraced by a small enough group that many feel an ego-intersection with the technology, motivated to defend and advocate it when the opportunity arises. Linux once had such an attack-dog core of rabid enthusiasts, though as the user base has grown, and it has become more pedestrian, you really have to target a Linux-niche (such as a little used distro) if you're aiming to stir up a hornet's nest.
That entire lead-up was just some context for the actual topic of this entry: So-called premature optimization.
A common response to Joel's complaint that Ruby is slow or resource inefficient is the frequently incanted declaration that such complaints are nothing but "premature optimization!"
I've seen the same deflection shield used to defend abhorrent database designs, convoluted, overly-abstracted class designs or message patterns, and virtually anything else where a realist might proactively ponder "but won't performance be a problem doing it like this?", only to yield the response "You know, premature optimization is a classic beginners mistake!"
If you don't want to be lumped in with beginners, the lesson goes, it's best to pretend that performance simply doesn't matter. We'll cross that bridge when we get to it.
Premature optimization is the root of all evil (or at least most of it) in programming.
Donald Knuth
I remember the early days: I once spent about 16 work hours optimizing a date munging function, increasing its performance from something like 2 million iterations per second to 4 million iterations. In the grand scheme of things, the performance difference was completely negligible, but from the perspective of artificial benchmarks it seemed like tremendous progress was being made.
That was premature optimization.
Indeed, anyone who's done time in the software development industry can identify with what Mr. Knuth was saying, probably having been involved with (or responsible for) project plans gone awry when efforts focused on highly-complex caching infrastructures, or ultra-optimizing some seldom used edge function.
Yet what is arguable, and situation specific, is deciding what qualifies as premature, versus what is simply proactive, predictive, professional performance prognostications.
NOT ALL PERFORMANCE CONSIDERATIONS ARE PREMATURE OPTIMIZATION!
While there is no doubt that there is such a thing as premature optimization -- it is an evil distraction that sidetracks many projects -- there are critical decisions made early in a project that can cripple the performance potential (both resource efficiency, and resource maximum), making later optimizations enormously expensive, if not impossible without an entire rewrite.
Whether it's heavily normalizing the database (or its nefarious doppelgänger, the classic database-within-the-database: "This single table can handle anything! Just put a comma separated array of serialized objects in each of the 256 varbinary(max) columns! Look at the flexibility! Query it? Don't you bother me with your premature optimizations!"), creating an application design that's incongruent with caching, or choosing an inefficient platform.
There are credible performance considerations that need to be addressed at the outset, and revisited as development proceeds. It is absolute insanity, and entirely irresponsible professionally, to simply stick one's head in the sand and hope that some magical virtual machine improvements or subcolumn indexing decomposition and querying technology will occur before deployment, or before the economics of scaling come into play.
And speaking of scaling, the canard that the horizontal-scalabilty intrinsic with most web apps (unless you really screwed up the design -- as many people do -- and made horizontal scalability impossible) makes the problem a nonissue is absurd: Perhaps if your project has a high transaction value then you have the luxury of adding more servers to serve a small number of clients, yet for most real-world projects adding resources is a big, big deal. And it isn't simply the cost of a low-end Dell 1850: Whether you're colocating or hosting in an expensively rigged corporate server room, the cost of each server is substantial.
You end up in the dilemma that you're financially (or physically) limited to a set quantity of resources, having to limit or scale-back the functionality provided to each user due to the inefficiencies caused by early decisions. "Sorry we can't implement that cool AJAX type-ahead lookups because the callbacks would kill our servers - we're already saturating them with our stack of inefficiency, so there's no overhead left."
I think the lackadaisical attitude towards efficiency is a result of experience derived from countless unvisited or seldom used web apps deployed across millions of PCs, colocated with equally as spartanly used peers. When a site sees a dozen visitors in a day, it's easy to declare that performance is a seeming nonissue nowadays - that it's only a concern for game programmers and nuclear modelling engineers. Then one day the page gets mentioned on Digg or Reddit or Slashdot or BobOnHardware and in that potential moment of glory the app falls over and dies, again and again.
None of this really has anything to do with Ruby. Personally I haven't used it beyond the tutorials, though I do know that it does very, very poorly on the standardized benchmarks. However it is distressing seeing so many people dismiss Joel's comments (or comments about Python, or ERlang, or XML, or any other technology) as premature optimization.
