Project Completion

You just finished a gruelling six-month project to create the Greatest Software Widget Ever Made. Your blood, sweat and tears have been invested into what you believe is a first-rate solution, whether solving a corporate need, or building a better mousetrap to bring untold riches to your ISV.

You email news of your accomplishment to peers and/or clients, proudly sitting back to await the certain accolades.

Amidst the expected congratulations, one reply makes you uneasy. "So it basically does what Project X does?" it asks.

Surely they must be mistaken, you think.

You fumble through the feature sheets for Project X to discover that, sure enough, on paper it does everything that your solution does, and in some ways it is arguably superior: Maybe it supports a richer set of features, is cross platform where yours isn't, or it's a component of a more-complete integrated solution where the whole is greater than the parts.

On paper, at least, it is a better solution.

The Damage

If you're an ISV, you've just discovered that you have more competition than you thought you did. This could impact your market in a wide range of ways.

• Perhaps it's a very expensive product from a company like BEA. You can easily underprice them, and it represents little threat to your product (it might actually help you make sales, giving you a much more expensive product to contrast with in sales pitches). Great stuff!
• Perhaps it's the complex potential use of an existing system that requires such a high level of expensive modules, customizing and programming that it turns out to be much more expensive and risky to actually use.
• Maybe it's an esoteric open-source project with a highly convoluted code-base, few developers behind it, and it requires significant knowledge to configure and use. It represents only a theoretical risk.
• In the worst-case scenario, the alternative is a free, comprehensive, elegant and usable alternative that is quickly becoming widely known. Your market has been devastated, outside of selling in hopes that your customers are blissfully unaware (which is rightfully morally difficult for most people).

If you're a corporate developer, it could simply be a demotivating signal of time wasted (developers are heavily motivated by seeing their solutions doing "good" in the world. Dumping code they invested time and passion into is heart-wrenchingly demotivating. "What have you been working on?" "Well, I spent the last year working on a project that was dumped for Microsoft BizPoint 2009 before delivery"); it could just be ammo for the office Machiavellian (I've was criticized in the past by just such a coworker for not leveraging a "possible solution" that wasn't even invented until two years after my project was deployed, so such Machiavellian Jr's aren't even bound by the constraints of time); or it could be a career killer.

In the corporate space things can get even uglier, as products from the big names like Microsoft will be preferred, however ill-equipped for the task they actually are: If they marginally seem to apply to the problem, there will be those who push for it. Sometimes this strategically makes sense (e.g. if you adopt a new product early, it'll likely evolve quickly if it's developed by a company with the resources that Microsoft has, for instance), but often it's just corporate hubris. Some organizations have a love-hate (mostly hate) relationship with their IT department, so any opportunity to undermine and reduce the dependency on them, even if it involves a more expensive, less-flexible, less-empowering solution, will be pursued. I worked at one shop where a large team of developers worked for years perfecting and expanding in-house solutions, while at the same time the organization continually and openly quested to replace everything they did and were doing with externally sourced solutions. The effect on motivation was profoundly negative, but it was the result of a Better Not Be Invented Here mentality, coupled with a negativity opinion of IT.

The Smaller Scale

Of course such an overlap of functionality or use isn't limited to whole products. It applies to even the smallest component of our projects.

Developers face this sort of issue regularly: From cryptography libraries to database abstraction tools to MVC frameworks to simple coding standards - For virtually every single need, there are solutions that purport to solve all ills and fulfill all needs. Somewhere amidst the 100,000+ version 0.1 projects on sourceforge, the huge catalog of software and feature lists provided by organizations like Microsoft and Oracle, or the millions of projects on the web, and even amongst the libraries and frameworks already included with our platform, there are possible competitors.

From a more optimistic perspective, they are possible allies - code and existing research and knowledge that you can leverage to make your solution better. Why waste time building a dubious non-core cryptography function just to enable security, if you can just use the System.Security.Cryptography collection of classes? Why re-invent the wheel when a lot of standard solutions are crystallized in industry patterns?

The problem is that you have to know about the knowledge, competitors and/or allies that you can leverage.

The Problem

Several months back I wrote that internal code reuse can be dangerous, the premise being that most teams don't spend enough time investigating what is already available (often available in a superior, more widely-known fashion) - be it coding standards, frameworks, or whole projects, a lot of the code base in many organizations is completely redundant, inferior duplications of what already exists, created purely out of lack of awareness. Even when resources are within grasp they're often ignored - A huge number of in-house .NET "utility libraries" duplicate what exists in the .NET Framework. All because no one bothered looking through the admittingly-massive MSDN Library before undertaking a task themselves.

While sometimes this occurs because of the Not Invented Here syndrome, more frequently it's simply ignorance - Either intentional ignorance (such as not bothering doing proper competitor/options research before getting started), or unintentional ignorance (there is such a huge number of projects out there, often with convoluted explanations of what they do, and difficult and time consuming setups to test it out yourself, that it can be impossible to know what already exists. For some small components and projects it can literally be faster to just do it than it is to investigate the alternatives).

Huge code repositories grow larger with second-rate solutions, products are developed that enter the market dead-on-arrival, and developers have the passion beaten out of them.

The Paralysis

As bad as all of this sounds - making solutions to discover that they're second rate or obsolete, or huge internal code repositories filled with duplications of better alternatives - the problem exists even where these problems aren't evident (in fact a secondary-effect occurs because of the prevention of these issues): Many software developers are simply decision-paralyzed. They're unable to proceed with any project because the number of possible standards, components, and frameworks that they need to consider is unending.

This is made worse by the fact that there are tens of thousands of software products and components making claims that they can't back up (especially in realm of nebulous, all-empassing applications. These high level applications often require tremendous configuration and programming to be usable, but the appearance is that they're a drop-in-place solution). Sourceforge, for instance, is full of tens of thousands of version 0.1 applications, where the purported feature list could be be described as a wish list. Other products entail tremendous expense, or bring along undesired dependencies or complexity.

Researching, and then understanding every caveat and issue with every potential solution is a huge undertaking.

The Solution

If only it were so easy to simply rattle off a solution.

Nonetheless, a huge step in the right direction would be simpy allocating adequate time upfront for "competitor research" before committing to a project, or even before undertaking a component. This is a tremendous undertaking these days, yet in timelines it's often forgotten or considered satisfied by a quick 5 minute Google search.

If you're developing a web application to store documents, could it be satisfied by Sharepoint? Community Server? Zope? A derivative of Media Wiki? If you've committed to one of those projects, but you just need to extend some of the functionality (say you want to display the EXIF data of photos), have you committed the time to investigate the alternatives?

For virtually every need, there are a slew of theoretical solutions. The truth, however, is that many of those solutions aren't a good fit - many of them are simply terrible - but you need to be informed and prepared when peers, coworkers and clients ask about them in comparison.

Even if you don't find an appropriate solution already available (in most cases you won't), document all of your findings: If Biztalk looks superficially like it's the right solution, but for some reason it isn't appropriate, be prepared for the inevitable grilling you'll get every single time someone who knows about Biztalk and then hears about your project gets within earshot.

Be prepared!

Tagged: [Software Development], [Programming], [Software-Development]