A lot of my work - both system consulting and software architecture/development - relies upon Microsoft technologies: Whether it's re-engineering a legacy system to take advantage of new SQL Server features for performance or functionality, overhauling a network infrastructure to leverage ActiveDirectory and the extensive platform security functionality, or developing a performant and scalable time-tracking application for an enterprise client, Microsoft is often a very important part of the equation.

Partly due to specialization (it's the tools we target), coupled with simply being the best choice in a lot of scenarios in our target market, we heavily rely on the Microsoft platform for ourselves and our customers. As a professional I can say with confidence that the platform is a secure, high performance, extremely scalable, robust one that compares very favourably against all competitors.

That wasn't always an accurate statement, though. Indeed, it is remarkable looking at the history of Microsoft and learning from their success: On paper it really is hard to believe that Microsoft maintained the market dominance that they did, and it's amazing that competitors couldn't capitalize on Microsoft's late entrance into a lot of markets, and their missteps in others.

Was Microsoft a master of timing, holding off on technologies and advances until the perfect time, or were they simply the beneficiary of a captive audience that was willing to wait however long Microsoft took, blind to the available alternatives?

I'll provide a couple of examples that I recall marvelling at as they occurred- these are hardly exhaustive, however I think it's a nice sampling.

Microsoft Maladies

• Microsoft core offerings were crippled by real/virtual mode limits until long after the 386 and 486 were prevalent. In a nutshell, this made software development a lot less pleasant, and the resulting applications more limited and unstable - I remember being enormously unhappy learning real-mode assembly on the x86 after dealing with the elegant, 32-bit flat world of Motorola 68000 assembly. It seemed so primitive that it still existed, or that a software company continued to rely upon it long after it was obsolete and irrelevant in hardware.
• Microsoft's "operating system" for years was simply the DOS command line, and a set of utilities and software interrupt handlers. While Mac users were busy with a rich graphical user interface, we in the DOS world were anxiously awaiting fantastic new features like DELTREE, and maybe a new version of EMM386 to deal with real mode nonsense. It amazes me now to recall actually going to a store and paying real money for a stack of 3 1/2" DOS 5 upgrade disks...6 years after I was programming applications on a richer 4MB platform, here I was excited that himem.sys could free up some of the critical 640KB of low memory.
• Microsoft toyed with windowing systems, finally creating something credible and successful in 1990 (Windows 3.0). In contrast a variety of competitors had fully-integrated, rich, usable, robust Windowing systems many years before - The 1984 Apple Mac being an obvious example, along with the 1985 Atari ST and Amiga...even options on the Commodore 64. I was an Atari ST fanatic in those days, and I marvelled at how primitive the PC world remained even years later.
• It wasn't until Windows 95/NT that memory protection was utilized to avoid processes stomping on each other's memory. Again, many, many years after most competitors had implemented this basic functionality. Instead we dealt with the normal occurence of misbehaving apps taking down the entire system as a fact of life.
• It wasn't until Windows 95/NT that preemptive multitasking was available in Windows. Prior to this a single misbehaving application could capture the CPU's attention and never let it go (never yielding), which was a fairly typical event. The Amiga featured pre-emptive multitasking a decade earlier.
• Microsoft released Windows 95 without a web browser, remarkably enough, finally releasing a barely changed version of the NCSA's Mosaic in the Plus! pack.
• Microsoft 95 was pretty much a security nightmare. Not only was its software far-from-ready to be connected on the public internet - I remember being the unhappy victims of winnuke and friends when I made people unhappy on IRC (you can't please all of the people all of the time), it also had no real file/object security of consequence. While NT was built as a "multi-user" system from a security and kernel perspective, many of the shell and utilities were user unaware, undermining this capability.
• Microsoft's web technologies were far behind the times until Bill Gates' famous speech that changed their direction, reacting to Netscape's lead rather than charting the course. Internet Explorer quickly ramped up and became the dominant web platform - until it became so powerful that the team was disbanded.
• Alternative 3D rendering APIs (Glide and OpenGL) led the way in an area where eventually DirectX would emerge dominant.

I recall during my early courtship with the PC simply marvelling at how incredibly obsolete the platform seemed to be compared to competitors like the Amiga and the Mac introduced years earlier - from graphics capabilities to software to hardware: Everything about it seemed so backwards in comparison to the superior alternatives, yet customers stuck with it. This was the platform that Microsoft wed themselves to, so surely they would suffer as well, right?

Microsoft's insistence on legacy compatibility led to a platform that moved much slower than competitors - Competitors that had the liberty of just tossing it all out and starting from scratch with whatever whizz-bang feature the newest chips offered. Maybe they could run super-stable and super-fast, and offer the developers an elegant platform upon which to perform their magic...but could it run Commander Keen 1 through 3? Could it run that ancient text database app?

Not All Negatives

Of course it's easy to focus on the deficiencies and imagine that they wrote the whole story, but in reality the situation was much more complex. Windows, for instance, pioneered widely-used video card acceleration (I still remember that shiny new Diamond Speedstar 24x. 24-bit graphics, coupled with hardware acceleration of 3D primitives. It was good times running those benchmarks. Of course the Amiga fanatics will point out that it supported hardware acceleration, just as the STe featured a hardware blitter chip, but the interaction between acceleration and the GDI in Windows really set the bar), and Microsoft's push greatly accelerated the adoption of optical media. Windows For Workgroups brought inexpensive networking to a lot of shops (NetBEUI was imperfect, but it was an easy transition to TCP/IP), and Windows in general represented a "good enough" platform for a lot of users. Internet Explorer, for all of its ActiveX "holes" and CSS quirks introduced the rich web model that we rely upon today.

En Route to 64-bit x86

This all comes to mind as the x86-64 transition accelerates: More and more users are starting to switch to 64-bit capable systems, and the 2/4GB limits of our machines is actually becoming a rational limit among desktop users: Everyday users are shouldered against a limit that seemed almost theoretically large just a few short years ago.

Of course Microsoft has been releasing incomplete 64-bit options for years (for instance you could get a 64-bit version of SQL Server 2000 for the Itanium platform, barring a laundry list of exclusions and limitations, and way back with NT 3.1 Microsoft supported 64-bit processors, albeit in 32-bit mode). Now that 64-bit support is finally becoming a critical factor, Microsoft has a wide gauntlet of support ready, and is finally ready to deliver.

Once again when the market really cares, Microsoft is ready. For years some have been talking about the advantage of various operating systems, such as Linux, being availabile on cutting edge processors and 64-bit platforms. For years that has been paraded as an advantage to customers who continued to run their platform on a standard old x86-32 foundation. Yet now that those limits are being reached, and the platform needs to accommodate new levels of capability and performance, Microsoft is ready. Another deficiency overcome.

The Question

Looking at the platform now - the stability, security, and feature set of Windows 2003, a lot of it already existing in XP - it really does seem like a tremendous window of opportunity for the competition has passed: What used to be a crop full of delectable low hanging fruit is now a well protected enclave featuring armed guards.

If competitors couldn't make inroads before, how do they have a chance now? If Linux couldn't capture the desktop market against a monstrosity like Windows Me!, what chance does it really have against XP?

The most obvious answer is web applications - render the operating system irrelevant and you don't really have to compete.

Tagged: [Software Development], [Programming], [Software-Development]

VCRs and PVRs - Small Usability Improvements Yields Huge Usage Changes

How often did you use the scheduling functionality of your VCR to record your favourite television shows, decoupling yourself from the rigorous schedule imposed by the television networks?

The common answer, overwhelmingly, is never. Few bothered using the scheduling functionality, even when it would be beneficial to their quality of life.

This inspired endless jokes about the complexity of "programming the VCR". Even the few brave "wizards" who did bother scheduling recordings generally did so rarely: The hassle of managing tapes, manually setting schedule times, and then having the uncertain-quality result unavailable until completion simply wasn't worthwhile to most people. Many times it didn't work out, and they discovered that they actually recorded 8am instead of 8pm. Whoops!

Even the introduction of Guide+ - a system that allowed you to record a program by punching in a short code - changed the situation little: To many it still wasn't worth the marginal hassle.

The functionality to time shift was there, but few leveraged it.

This topic came up after becoming engaged in an interesting discussion about PVRs versus VCRs, and why the former is inspiring panic and behaviour changes among the television networks, while the latter was largely ignored. Consider that virtually every household in the West had one or more VCRs, yet only a very small percentage have a PVR today (though obviously it's a much greater percentage among the net savvy). Why the concern about functionality we've had for well over a decade?

The reason for the panic, of course, is that the seemingly minor usability and functional improvements of the PVR dramatically increased the usage and utility of the technology: Instead of rummaging for Guide+ numbers in the back of the newspaper, or worse - configuring start and end times manually - one simply pulls up an online listing, selects the programs they want, and selects to record them. The quality is superlative, it takes just a few moments, and they gain the added ability to quickly skip past commercials. Many choose to automatically record every new episode, saving even more time. To put the icing on the cake, there's no hassle dealing with the tapes.

simply reducing the complexity or number of steps marginally can lead to market dominance

There is a valuable lesson to be learned from this: Seemingly minor advances in usability can tremendously alter marketplace success (the VCR was, of course, a great success in marketplace saturation, but that was almost entirely on the merit of playback of pre-recorded content. Few used it to actually record content). Even when it seems like a marketplace need has been functionally satisfied, simply reducing the complexity or number of steps marginally - or reducing the barriers to entry - can lead to market dominance (or market creation). A PVR isn't just a VCR with a hard drive - It completely changed the equation.

Software For Every Need

Consider the software market: By all appearances it looks to be a saturated market - with a solution for every need - but the remarkable thing is that much of it remains completely unused and unadopted. There are countless domains where solutions sit collecting binary dust because the complexity or barrier to entry is too high.

Skype, in contrast, blazed a path of glory and achieved virtual overnight success, yet really it's just yet-another IP voice technology (like we've had since the mid-90s. Sure it added the distributed net, but that's a feature that is a marginal improvement at best). It offered a clear, usable interface, firewall avoidance, and a simple directory for finding the other person, and bam it is getting bought out for $2.6 billion - for doing what had been done by countless competitors in a seemingly commodity market for years before.

FogCreek software has had success simply taking some open source software and putting a pretty face on it, offering a small value-add (avoiding configuring your firewalls) - Making money charging money in a market that people thought was saturated with free alternatives. The web could really be considered a Gopher 2.0, but improved usability enough to be embraced by the everydayman. Bam, the webolution. HTML is absurdly trivial, yet the marginal usability advance of blogs are what made everyone a writer. CSS and JavaScript are both highly accessible technologies, and you can get started quickly by viewing the source of sites you like, again vastly accelerating the transition from initial exposure to actually doing something with it.

What About the Professionals?

Even when targetting highly-trained professionals, immediate "usability" remains critical. Remarkably many of the successful back-end technologies are those that were easy to get started with.

Extraordinary to think that multi-year projects and massive web applications of tremendous scale were built on chosen technologies because they offered a painless, 10-minute getting started setup and tutorial - letting someone start pushing out code immediately - yet in talks with peers I've found that this is frequently the case. Indeed, I will admit to this irrational behaviour myself - several times I considered implementing a project in J2ME (targeting cell phones), but the hassle of setting up a J2ME development platform, and then the pertinent modules for the various phones, served as such a discouragement that I abandoned the project rather than wasting 4 hours dealing with that. In the longer term of a project it's completely irrational, yet it happens.

Of course much of the ASP development community evolved not because ASP was the best platform that was being chosen on merit, but rather because a lot of shops had a Windows NT box sitting there with IIS on it, and they started dropping ASP scripts on it (other languages, like PHP, required additional installations = more trouble). Soon enough these were ASP shops, even though it was almost accidental. Few of them really seriously evaluated the various alternatives.

Of course this was by design: Microsoft, who I spoke about earlier, understands this resistance to learning well. They have entered countless markets with seemingly inferior offerings (at least at first), but because it's there (Microsoft used to rely upon "everything on" by default) and it's easy to use, the marketplace adopts it. SQL Server is a fantastic database system (I personally believe it was one of the best, and is now the best with SQL Server 2005), but a lot of its growth came about largely because it was a trivial install with a simple, ultra-low barrier to entry GUI: Joe Developer installs it from the MSDN discs, prods it for a while, and soon enough he's building the enterprise data system on it. All because it was so accessible and easy to use [Of course many of those database don't use transactions (or they don't properly), and they host terrible schemas, but it got it used]. On the Windows platform a lot of admins did the "install everything" technique, and slowly they sorted it all out and utilized it. This was the way that Microsoft entrenched itself into corporate networks.

Contrast this with other areas where Microsoft hasn't followed this philosophy, and where the results have been much less positive - Even for critical back-end technologies like Biztalk and Sharepoint (both of which yafla provides solutions and consulting for), where you would think it would be soberly analyzed by experts over months of analysis before deployment (and thus requiring significant upfront configuration should be a non-issue), they often see little adoption simply because the install or initial configuration discourages fly-by investigation. Without the initial investigation there is no one to champion it, so it goes unused (despite being fantastic products).

There are countless examples of products whithering because the first install required 40 steps, and then doing the first "hello world" type of project was an enormous hassle. On the flip side a lot of questionable technologies and solutions have permeated largely because it was usable immediately, with little up-front investment.

Minimize Barriers to Entry - Make Your Software Initially Easy

If you make software products, ensure that Getting Started is as painless as possible, and advanced customization options are saved until the user has some experience with the product (literally it should install and configure everything, and start the user off with a Hello World template solution): Even if your customers will need to spend hundreds of man hours specializing it for their needs, they need to see something they can poke at and interact with almost immediately, giving them a sense of accomplishment to motivate them to continue on. 

Once you've gotten the initial time investment, it's much, much easier to require a more involved understanding, and to demand that the user commit themselves to some educational time by the fireplace with the documentation.

We're a very impatient bunch these days, and this is critical if you want success.

If, on the other hand, you're looking at potential markets for software products, examine the usage patterns of PVRs versus VCRs - While the software world might seem full of existing solutions, really the field is wide open for usable solutions. Make an easier to use mousetrap and much of the world will beat a path to your door.

Tagged: [Software Development], [Programming], [Software-Development], [Usability]

One of the huge stars of the early Internet was e-cards - static, animated-GIF, or Flash alternatives to a standard ink and paper Hallmark card, not to mention that you could save yourself a couple of bucks and the hassle of going out and buying and mailing a traditional card (the biggest boon to men like myself was that you could quickly send one off on the day in question - no event pre-planning necessary).

One of the biggest successes of the e-card phenomena was Blue Mountain. My first experience with Blue Mountain was actually by mistake - I was looking for some ski conditions, and instead of http://www.bluemountain.ca/ I entered http://www.bluemountain.com. Of course I capitalized on my mistake, immediately sending some goofy e-cards to friends and relatives. Of course many of the recipients started using the service as well. I suspect that the tipping point for that company - the contagious source vector - came when winter rolled around in Ontario and the powder was just right, and lots of misdirected Ontarians figured they might as well use the service since they were there.

During those early days e-cards were sent in bulk. It was good times in the e-card business, and marketing was easy: Every recipient was encouraged to send their own e-cards, so growth was exponential.

Excite @ Home bought Blue Mountain for a blistering $780 million dollars in 1999.

Of course, as with most quick-ascent fads, people started to question e-cards: It the sentiment was so cheap, and required so little personal effort, was it really a substitute for a real card? Was it just a thoughtless form of spam, quickly discarded and ignored? It didn't take long for that sort of feeling to pervade the marketplace.

If this were an episode of Behind the Music, dark and disturbing music would be playing. The Dire Straights of e-cards were upon the industry.

Excite @ Home unloaded Blue Mountain for $35 million in 2001. If the business were individually valuated today, I highly doubt it would come anywhere close to even that number.

This thought came to mind given the almost-absence of e-cards I've seen this holiday season. Perhaps it's just me, and my network of contacts is unique in its reliance upon traditional cards, but it is a remarkable contrast. I did receive one e-card - from a web host - and quite honestly it bordered on a bit offensive getting a holiday greetings card that I know was spun off from a mass mailing script.

Project Completion

You just finished a gruelling six-month project to create the Greatest Software Widget Ever Made. Your blood, sweat and tears have been invested into what you believe is a first-rate solution, whether solving a corporate need, or building a better mousetrap to bring untold riches to your ISV.

You email news of your accomplishment to peers and/or clients, proudly sitting back to await the certain accolades.

Amidst the expected congratulations, one reply makes you uneasy. "So it basically does what Project X does?" it asks.

Surely they must be mistaken, you think.

You fumble through the feature sheets for Project X to discover that, sure enough, on paper it does everything that your solution does, and in some ways it is arguably superior: Maybe it supports a richer set of features, is cross platform where yours isn't, or it's a component of a more-complete integrated solution where the whole is greater than the parts.

On paper, at least, it is a better solution.

The Damage

If you're an ISV, you've just discovered that you have more competition than you thought you did. This could impact your market in a wide range of ways.

• Perhaps it's a very expensive product from a company like BEA. You can easily underprice them, and it represents little threat to your product (it might actually help you make sales, giving you a much more expensive product to contrast with in sales pitches). Great stuff!
• Perhaps it's the complex potential use of an existing system that requires such a high level of expensive modules, customizing and programming that it turns out to be much more expensive and risky to actually use.
• Maybe it's an esoteric open-source project with a highly convoluted code-base, few developers behind it, and it requires significant knowledge to configure and use. It represents only a theoretical risk.
• In the worst-case scenario, the alternative is a free, comprehensive, elegant and usable alternative that is quickly becoming widely known. Your market has been devastated, outside of selling in hopes that your customers are blissfully unaware (which is rightfully morally difficult for most people).

If you're a corporate developer, it could simply be a demotivating signal of time wasted (developers are heavily motivated by seeing their solutions doing "good" in the world. Dumping code they invested time and passion into is heart-wrenchingly demotivating. "What have you been working on?" "Well, I spent the last year working on a project that was dumped for Microsoft BizPoint 2009 before delivery"); it could just be ammo for the office Machiavellian (I've was criticized in the past by just such a coworker for not leveraging a "possible solution" that wasn't even invented until two years after my project was deployed, so such Machiavellian Jr's aren't even bound by the constraints of time); or it could be a career killer.

In the corporate space things can get even uglier, as products from the big names like Microsoft will be preferred, however ill-equipped for the task they actually are: If they marginally seem to apply to the problem, there will be those who push for it. Sometimes this strategically makes sense (e.g. if you adopt a new product early, it'll likely evolve quickly if it's developed by a company with the resources that Microsoft has, for instance), but often it's just corporate hubris. Some organizations have a love-hate (mostly hate) relationship with their IT department, so any opportunity to undermine and reduce the dependency on them, even if it involves a more expensive, less-flexible, less-empowering solution, will be pursued. I worked at one shop where a large team of developers worked for years perfecting and expanding in-house solutions, while at the same time the organization continually and openly quested to replace everything they did and were doing with externally sourced solutions. The effect on motivation was profoundly negative, but it was the result of a Better Not Be Invented Here mentality, coupled with a negativity opinion of IT.

The Smaller Scale

Of course such an overlap of functionality or use isn't limited to whole products. It applies to even the smallest component of our projects.

Developers face this sort of issue regularly: From cryptography libraries to database abstraction tools to MVC frameworks to simple coding standards - For virtually every single need, there are solutions that purport to solve all ills and fulfill all needs. Somewhere amidst the 100,000+ version 0.1 projects on sourceforge, the huge catalog of software and feature lists provided by organizations like Microsoft and Oracle, or the millions of projects on the web, and even amongst the libraries and frameworks already included with our platform, there are possible competitors.

From a more optimistic perspective, they are possible allies - code and existing research and knowledge that you can leverage to make your solution better. Why waste time building a dubious non-core cryptography function just to enable security, if you can just use the System.Security.Cryptography collection of classes? Why re-invent the wheel when a lot of standard solutions are crystallized in industry patterns?

The problem is that you have to know about the knowledge, competitors and/or allies that you can leverage.

The Problem

Several months back I wrote that internal code reuse can be dangerous, the premise being that most teams don't spend enough time investigating what is already available (often available in a superior, more widely-known fashion) - be it coding standards, frameworks, or whole projects, a lot of the code base in many organizations is completely redundant, inferior duplications of what already exists, created purely out of lack of awareness. Even when resources are within grasp they're often ignored - A huge number of in-house .NET "utility libraries" duplicate what exists in the .NET Framework. All because no one bothered looking through the admittingly-massive MSDN Library before undertaking a task themselves.

While sometimes this occurs because of the Not Invented Here syndrome, more frequently it's simply ignorance - Either intentional ignorance (such as not bothering doing proper competitor/options research before getting started), or unintentional ignorance (there is such a huge number of projects out there, often with convoluted explanations of what they do, and difficult and time consuming setups to test it out yourself, that it can be impossible to know what already exists. For some small components and projects it can literally be faster to just do it than it is to investigate the alternatives).

Huge code repositories grow larger with second-rate solutions, products are developed that enter the market dead-on-arrival, and developers have the passion beaten out of them.

The Paralysis

As bad as all of this sounds - making solutions to discover that they're second rate or obsolete, or huge internal code repositories filled with duplications of better alternatives - the problem exists even where these problems aren't evident (in fact a secondary-effect occurs because of the prevention of these issues): Many software developers are simply decision-paralyzed. They're unable to proceed with any project because the number of possible standards, components, and frameworks that they need to consider is unending.

This is made worse by the fact that there are tens of thousands of software products and components making claims that they can't back up (especially in realm of nebulous, all-empassing applications. These high level applications often require tremendous configuration and programming to be usable, but the appearance is that they're a drop-in-place solution). Sourceforge, for instance, is full of tens of thousands of version 0.1 applications, where the purported feature list could be be described as a wish list. Other products entail tremendous expense, or bring along undesired dependencies or complexity.

Researching, and then understanding every caveat and issue with every potential solution is a huge undertaking.

The Solution

If only it were so easy to simply rattle off a solution.

Nonetheless, a huge step in the right direction would be simpy allocating adequate time upfront for "competitor research" before committing to a project, or even before undertaking a component. This is a tremendous undertaking these days, yet in timelines it's often forgotten or considered satisfied by a quick 5 minute Google search.

If you're developing a web application to store documents, could it be satisfied by Sharepoint? Community Server? Zope? A derivative of Media Wiki? If you've committed to one of those projects, but you just need to extend some of the functionality (say you want to display the EXIF data of photos), have you committed the time to investigate the alternatives?

For virtually every need, there are a slew of theoretical solutions. The truth, however, is that many of those solutions aren't a good fit - many of them are simply terrible - but you need to be informed and prepared when peers, coworkers and clients ask about them in comparison.

Even if you don't find an appropriate solution already available (in most cases you won't), document all of your findings: If Biztalk looks superficially like it's the right solution, but for some reason it isn't appropriate, be prepared for the inevitable grilling you'll get every single time someone who knows about Biztalk and then hears about your project gets within earshot.

Be prepared!

Tagged: [Software Development], [Programming], [Software-Development]

The Expense of Errors

A widely demanded feature delivered with Visual Studio 2005 is "Edit and Continue"; which is the ability to alter running debug code to incorporate code-changes on the fly: You're debugging and realize that you initialized a variable to the wrong value, or your loop control has a off-by-one error. You pause the run, quickly hash out the changes, it builds it into the running image, and you continue debugging from where you were.

Great feature that can be a tremendous time saver, avoiding having to stop the session, make the changes, rebuild everything, and then begin the developer test session anew.

Is it, coupled with similar tool advances over the years, making programmers generally sloppier, though?

Sloppy Programming Habits

Observing the habits of many peer developers in the field, I would say that it and similar advances absolutely have made us more careless in general: The less expensive errors become, the fewer checks and mental effort we'll expend ensuring that they don't get into the code in the first place. We're continually pushing the onus of catching errors one level higher.

To contrast with a slightly earlier time, there was a time, way back when (circa 1990), when I was plugging away with DJGPP (GCC for MSDOS), editing the source files in a simple DOS text editor, exiting out, building (very time consuming, with few benefits like precompilation), and then running. It was such an onerous, expensive process that I put a significant amount of care and concern into every single line of code. I would follow-up careful coding by going back and auditing every single function and interaction to ensure that it was syntactically accurate, but more importantly that it was logically accurate.

The cost of an error making it to the next level was high enough that I was very motivated to avoid them in the first place.

After such a personal code audit, I was very confident in the code, and it was very rare that an error made it any further: The cost of an error making it to the next level was high enough that I was very motivated to avoid them in the first place. The original level of quality was high enough that few additional checks were actually needed - it simply worked correctly for all scenarios.

Of course I had it incredibly easy in contrast to those who programmed before (I already had the benefit of a significantly easier development process). I'm sure the folks who programmed punch cards redoubled and tripled the effort again, achieving amazingly high at-origin quality levels in their code: You can't just spit something out when you're printing and sorting punch cards, and then feeding it into the mainframe during your tight allotted time window. Nor did programming assembly in the 8-bit days leave much room for errors.

Contrast this with the habits of many developers today (myself included at times): Spit out a bunch of code, occasionally hitting compile/syntax check to automatically detect gross syntax errors. Build and run, and if it blows up then follow the exception back to the error and correct it. Drop into breakpoints and watch what values are to ensure they're what you wanted (a modern variation of printf debugging), and if they aren't then use edit and continue to quickly hash in some changes. Keep debugging. Run the TDD sets to ensure that the superficial, incomplete collection of tests "guarantee" that the code is "perfect". 

Toss the result over the wall to the QA department. They're likely running a macro script that tests a small sub-section of the code, so there are few guarantees there either. In the corporate space, they'll throw it over the wall to the UA testers who again will likely only catch the most obvious of errors.

Deal with the inevitable problem when failures occur in the field, pointing out their inevitability given the numerous layers of quality control you have in place.

Of course some developers will strongly object to even the possibility of such a scenario: Their code is flawless at inception, crafted with the utmost of care and concern, and they need never evaluate their habits or tool usage because they couldn't possibly come closer to perfection. That level of ridiculous denial is destructive on any team or project, and I can offer no advice on how to solve or manage it (though it's the foolishness of the inexperienced, so generally developers grow out of such bravado with time). Instead I choose to deal in the real world, with real developers on real projects in real organizations.

Additional Checks Are No Guarantee

For all of the process (including layers of QA, UA, regression testing, and so on), many errors aren't caught at many shops until the code reaches the field, which is why it's critical that they don't enter into the code in the first place.

the addition of layers can paradoxically increase the probability that errors will be introduced in the first place

Indeed, sometimes the addition of extra layers can paradoxically increase the probability that errors will be introduced in the first place: At one very large organization where I observed development firsthand, developers would hand their obviously flawed code (it was clear that there wasn't even a superficial quality check) over the wall, doing so knowing both that there was a QA department that should catch these things (and if they didn't then it's their fault if it makes it further, exonerating the developers even more), and if that department does find a fault it came as a largely ignored problem report that held few ramifications or negative implications.

Change precisely what was documented as defective, rebuild and resubmit.

Eventually the QA department would pass on the code to the UA department, which was a set of user testers that simply relied upon the comforting idea that the developers and QA surely would have found any possible faults. UA could be relied upon largely to restate long-known system limitations instead of verifying the changes.

All of these layers relieved developers, and each of the other layers, of the real responsibility of defective code. Advanced tools facilitated sloppy coding in the first place, and layer upon layers of ineffective checks ensured that there was little actual responsibility for faults that made it to the field. In the corporate space where developers generally don't have a passion for the software they're creating, the result was often of questionable quality.

False Efficiency

It would be an interesting experiment to have two concurrent mid-sized projects, each completing the same task, with one development team having a modern complement of development tools, and the other with no ability to automatically syntax check, run automated tests, or debug in any way outside of a small number of scheduled debug builds and test sessions. It would be interesting to evaluate both the overall timeline (did the tools save much time?), and the quality levels of the resulting product.

I believe that the results would be very surprizing to many software developers. In real world projects (e.g. not pre-project timelines, but actual post-mortem results), approximately one half of development is dedicated to finding, and fixing, software faults. Making the per-item cost of faults cheaper may reduce the per-fault cost, but it also might increase the frequency of faults to the point of being a net loss.

A comment that I frequently hear relates to the efficiency of development - That modern tools make us so much more efficient. Under the right conditions, and with proper usage, this is certainly the case. Edit and Continue, for example, could be a very useful feature once every blue moon. Yet by the outpouring of demand for that feature, one would think that developers were crippled by the inability to alter running code: The responsibility to craft quality code before hitting build was just too overwhelming. This is a sign that quality code craftsmanship is on the decline.

Tagged: [Software Development], [Programming], [Software-Development]

...is that they're often fact-deficient.

The rumor that's been making the rounds over the past 24-hours is speculation that Microsoft has bought, or is buying, Opera (the makers of the excellent Opera web browser). The source of this multi-hundred-million-dollar rumor, involving two publicly-traded companies, is one small, random, tech "ezine". Nonetheless it's been repeated on Slashdot, Digg, along with many other sites and blogs.

The indirect birthplace of the rumor - perhaps the source of the plot for someone's fiction - was John C. Dvorak's commentary yesterday that Microsoft should buy Opera, given that they're letting the IE browser rust on the Mac. Opera, Dvorak claimed, would provide Microsoft with a cross-platform solution.

The problem with Dvorak's hypothesis, among many other logic gaps, is that Microsoft doesn't want a cross-platform solution. Internet Explorer on the Mac wasn't abandoned because it was too difficult to maintain, and it certainly wasn't that Microsoft couldn't keep up. Instead, it was strategically abandoned because Microsoft couldn't rationalize spending the time and effort to create a browser for someone else's platform. It also brought Microsoft under additional scrutiny: It's one thing for the monopoly operating system vendor to use some of those profits to enable their own platform, but it's quite another to provide a free browser to edge out competitors on other platforms as well. Internet Explorer on the Mac undermined Microsoft's bundling proposition as well (the whole "it's a part of the operating system...that you can also run on other operating systems...").

Ultimately this rumor is so void of any logical foundation that it is an indictment of the group-think sites that it has gotten the attention that it has (though it got the originator website exactly what they wanted, which is a lot of attention). Once again I'm setting myself up for a hearty serving of egg-on-the-face, boldly proclaiming that it is very unlikely that this story is true, but that's exactly what I'm saying. At least the Google-buying-Opera nonsense of a month or so ago could be mentally rationalized to some degree, but Microsoft-buying-Opera is just over the top ridiculous.

On top of all of that, there are some legalities that are certainly going to start biting small internet papers and blogs that perpetrate this sort of rumor mongering: Making false statements about publicly traded companies in ways that can impact their valuations might seem like a harmless, fun way of boosting the hits, but there are legal restrictions on that sort of fiction. I'm not saying with certainty that this particular story is untrue, but it seems highly likely to be the case.

UPDATE: While scanning the discussion boards, I've noticed that a common technique to keep the hype alive is to focus on the mobile market as Opera's killer valuation: Opera has a micro-browser that runs on cellphones, therefore that legitimizes the purported buyout (as only the few true believers would think that Microsoft replacing IE7 with Opera is even remotely possible).

There are several problems with this.

• Microsoft already competes in the mobile space with the PocketPC OS, and a mobile version of Internet Explorer
• Even then, there is very little money to be made having your browser on a cell phone - this isn't IE versus Netscape of 1997. This is 2005, and the primary concern is to be the site that the micro-browser goes to, not the browser itself. In the case of cell-phones, there is little chance that the providers would allow Microsoft to direct it to a Microsoft property, so even that minor advantage is lost
• Many of Opera's customers (who are the hardware vendors - not Joe on the street) for the mobile browser are Microsoft competitors. They would dump it in a heartbeat if it was acquired by Microsoft

The mobile angle does nothing to rationalize the scenario.

[The static location of this piece can be found at this address]

How Ridiculous

Rediculous - Results 1 - 10 of about 3,800,000 for rediculous.

It is ridiculous that such an obvious misspelling has become so prolific (correctness by repeated assertion), yet it's a great example of how contagious an incorrect spelling can be: Given that language is largely learned by example, it is inevitable that an endless exposure to malformed spelling will eventually infect the language of others, gathering a widening net of victims.

This is more of a problem now more than ever, given that many of us have supplanted - or entirely replaced - the professional writing in our lives (newspapers, books, professional papers) with the amateur writing of bloggers and forum posters: The "good" influences - carefully authored, carefully edited professional writing - have given way to carelessly hashed-out entries by time-pressured bloggers and marginally-literate forum posters, in a domain where the accepted rules of netiquette strongly discourage pointing out spelling or grammar mistakes.

Those who point out errors in grammar or spelling are quickly marginalized as "Grammar Nazis". Ignorance rules the day, and the social pressure encouraging good spelling and grammar has dramatically declined.

English Is Non-Trivial, But Spelling is Standardized

English is a very difficult language, with a tremendous array of conflicting influences, and a byzantine array of specialized rules and conditions.

It is, for instance, very difficult to conform to all norms of grammar given that many of them are subjective and conflicting (and many self-appointed gurus have themselves made embarrassing errors). I have absolutely no doubt that this entry, for example, has over a dozen real or subjective grammar problems: From the incorrect placement of a comma, to the overzealous use of a compound adjective, to the use of a colon where a dash would suffice.

I certainly make no claim of perfection. Where I find that I've made an error (and I heartily welcome emails to this effect), I try to correct them as quickly as possible.

Nonetheless, spelling is standardized (with minor regional variations), so unless one is intentionally trying to extend or adapt the language, some effort should be exerted to check the standards references to ensure that one's usage is conformant, just as one would ensure that their CSS or HTML was compliant with the pertinent standards.

The Cost

The impact of the continued exposure to incorrect spelling and grammar can be extraordinary to observe. I've seen people corrected dozens of times, yet rediculous is so ingrained in their mind that they just can't break the habit. Soon enough other participants are perpetuating the misspelling, with the forum slowly diverging from correct English into some bizarre forum-localized lingo-ignoramus.

It might seem harmless, but this incorrect spelling starts infecting their professional writings (emails, instant messages, documents, signs, business cards - a domain where the laissez-faire attitude of the online world isn't acceptable), making them look ignorant and careless. That's if the fear of the same hasn't discouraged written discourse altogether (which is sadly very common. I've encountered plenty of professional acquaintances who avoid the written word like the plague).

It can even reduce the comprehension efficiency of written materials, as the reader's brain tries to rationalize the correct spelling on the paper with what they have stored in their memory cells.

It reduces general literacy.

If a reader's first exposure to analagous (analogous) or ancilliary (ancillary) are in a hastily written blog entry or forum post, naturally they're going to adopt the incorrect variant, perpetuating it to other entries and posts. Like a virus the misspelling infects new victims.

Of course it should be noted that language is indeed a "living" thing, and it does evolve and change over time - the English we speak today differs greatly from the English of yore - but the sort of ignorance that I'm describing has nothing to do with extending or adapting the language. Instead it's simple contagious laziness.

Showing Regard For Readers

Good form or not, I am regularly going back and rewording old entries for improved clarity and readability, and occasionally even to correct spelling mistakes that made it under the radar (I have some eagle-eyed readers that very helpfully point out some of these errors. Rather than being irritated by the "grammar nazis", I am very appreciative to have the extra sets of eyeballs).

I do this primarily to ease consumption by readers: While the initial entry might have been rushed when too little time was available - but I thought the information or perspective were useful for someone - the entries live on and see far more traffic over time than at the outset. A correction here and there, and the refinement and rewording of a paragraph or two to make it more clear and concise, takes me a few moments, yet it saves dozens or hundreds of readers time in the future (and improves their comprehension of the content).

I consider the effort very worthwhile.

Furthermore, I try to run all entries through an up to date spell-checker before the initial publishing. To make the process more palatable, I have trained the spell-checker with all of my domain-specific terminology (the false-negative rate of spell-checkers is one of the primary reasons most people avoid them).

I don't want to appear ignorant by misspelling a common word, and I don't want to save myself a little time at the cost of every reader's time. I also don't want to pollute the vocabulary of readers with believable misspellings.

Industry Solutions

Just as one eagerly sticks a W3C validation banner on their page declaring their compliance with some level of specification, it would be intriguing to advocate a "spelling and grammar" standard mark. One that simply declares that the author actually cares, and does exert some effort to meet some minimal level of correctness in spelling and grammar. It would be a public sign indicating that they are open and thankful for comments and corrections regarding the same.

Furthermore, it would be advantageous if the major search engines - including blog aggregators and search engines - allowed one to refine results by grammar and spelling, optionally scoring academically correct content higher in the results. While sloppy spelling is no guarantee that the content isn't of value, there is a noteworthy correlation between the care and concern put into the spelling and grammar of an entry and the value of the actual content contained within: If someone couldn't bother spell-checking their entry, the factual content of their entry naturally has to come into question as well.

In the forum and blogging world, it would be beneficial if more tools supported convenient and efficient automatic spell-checking (the fact that no major browser has incorporated native TEXTAREA spell-checking thus far is a travesty. Any of them could have a killer feature if they simply added Word-like squiggly underlining of suspect words, with easy alternative corrections). As it is, many tools have nothing at all, and the few that do often host a ridiculously unintuitive, hacked-in partial solution.

Let's clean up English on the internet.

[TRAFFIC NOTE: This story, one of the few general interest posts I've made on here, has appeared on reddit, sending quite a few users this way. For those who've actually read this far, if you found this entry interesting I would appreciate if you could give an arrow up to it on reddit. Alternately if you think this is a dud, please give it an arrow down. Thanks!]