In the pursuit of home improvement I've had to remove and reinstall several toilets over the past year, sometimes redoing the same toilet more than once due to mistakes made on the first outing: all of the preparation in the world sometimes isn't enough if you haven't practically experienced a task before.

I've gotten to enjoy the removal and installation of wax rings, the wonderful varieties of flanges, and the unfortunate results of using too thin of a wax ring when the newly installed flooring raise the toilet's height just a little too far.

Sure I could pay someone to do this, but sometimes I enjoy, err, getting my hands dirty and learning these things myself.

The thing I marvel at is how terribly hackish the interface between toilet and sewage system really is -- you have to align a very heavy toilet perfectly over two held-up-only-by-good-intentions bolts, often back-breakingly leaning over in very tight confines while setting it down just perfectly on the bolts; bolts that are barely contained by a flange that is the end piece of some fixed plumbing, meaning that if you overtighten then you're breaking a pipe that's a massive pain to repair.

It's close to impossible to gauge the difference between not tight enough and too tight with these sorts of things, but it's a costly mistake if you go too far one way or the other (too tight and you have a busted flange or toilet base, but if it's too loose you could have movement that breaks the seal, which causes floor warpage, which breaks the seal more, which causes more warpage, which pushes the unit up breaking the flange): Sort of like the oil change technicians who stripped the oil plug and didn't bother telling me until it popped off, creating an environmental disaster in my driveway, or the auto mechanics that over-cranked the spark plug, stripping the aluminum head.

Of course the toilet interface demonstratably works, given that there are billions of non-leaking, non-broken facilities in operation right now, but technically it is, in my opinion, a complete hackjob of an interface.

In many ways it's like HTML and browsers that will happily adapt to many transgressions of the already loose spec.

Our world has endless numbers of these "there has to be a better way" designs, but because of low expectations, the inertia of the status quo, lots of workarounds, and the training and experience of those who have to deal with it, it's fine.

I bring this up because software is endlessly compared to other disciplines, usually with the software industry cast in a poor light. 

"Why can't we build software like we build bridges?", we endlessly hear.

People say this because they have no idea how hacked out (and often over-engineered and costly -- have you looked at the cost of building a little road over a gap these days?) the business of building bridges, or airplanes, or toilets, often is.

It isn't as perfectly engineered as they probably imagine.

Sidenote - How in the world does an entirely ambiguous word like "biweekly" or "bimonthly" continue to be used? Biweekly is sometimes used to indicate every other week, other times used to indicate twice a week. Bimonthly has the same confusion, either meaning every two months, or twice a month.

For years I've argued against ridiculous software patents: While I'm a fervent believer of the innovation fostering environment provided by IP protection and rights, the stream of trivial software patents has reached a torrential pace. We're now at a point where it's impossible to create any software solution or website without infringing upon the trivial "IP" of hundreds of patent holders, leading to the unintended consequence that innovation is suppressed because of the natural litigation risk presented by patent trolls.

One of the more recent abusers of the patent system is none other than Microsoft. They're even patenting methods they're taking from other products now (sure, Microsoft's patent wouldn't hold up during the course of a real lawsuit, but few opponents would have the resources to even bring it that far).

Such a comment inevitably summons the Microsoft-does-no-wrong defenders who repeatedly declare that Microsoft is different, and that these were defensive patents. Microsoft, we are told, is just building a patent shield in case a patent troll comes gunning for them (ignore the absurd foundation of the whole "defensive patent" argument -- patent trolls seldom have any interest in cross-licensing, and they seldom have published products that a target can counter-assault with dubious patents).

Maybe not.

When Microsoft's growth curve seemed limitless, they overlooked casual piracy (which is how they gained dominance), had limited invasive copy protection mechanisms, and played nicely in the software development community. As their hegemony has faced real competition, some of it coming from the open source world (given that capitalist competitors were easy to squash), and their growth has been stagnant and has the potential of reversing course in the coming years, Microsoft has changed for the worse. Now they're enlisting jackbooted squads of anti-piracy teams, and infesting their products with activation and "genuine advantage" bugs.

Now they're starting to threaten competitors, and worse the customers of competitors, with their patent trolls.

The risk presented by Microsoft's so-called "defensive" patents is exactly what is happening. They've been offensive patents all along.

Expect it to only get worse.

Over the past 24 hours I've logged several hundred inbound visitors vectored over from http://www.dzone.com (I swear that I seldom check the stats -- honest to goodness cross my heart -- but I just happened to right after getting up this morning. It caught my eye as it wasn't one of the usual suspects). Fairly interesting programming-specific Digg/Reddit-like site, with a useful collection of links.

Just thought I'd mention it for those readers looking for something different. This isn't a reciprocity link, but rather that I truly found it a decent site with different content than the regulars.

Sidenote: One thing that I've always found fascinating about sites like that is the incredibly small percentage of users who actually "vote". I've been Digg'd, Slashdotted, Reddit'd, and now dzone'd, and I'd say that less than 5% of users who follow the links actually declare their feelings (in the case of Digg I would put the percentage even lower). While some of the sites don't allow for a negative vote, Reddit does but even there less than 1 in 10 link navigator expresses their feeling one way or the other.

I recently finished the book Dreaming in Code, grabbing a copy after it started appearing in all of the "customers who bought this book also bought"* sections for other books I was contemplating.

Wonderful book.

It documents the progress (or lack thereof) and missteps that occurred during the lengthy birthing of Chandler, an open source Outlook/Exchange "Killer".

Started with a lengthy, largely abstract wish list, and an ability to use whatever technologies and techniques that they wanted -- financed by the deep pockets of the undemanding Mitch Kapor -- Chandler was the sort of project that developers dreamily contemplate while analyzing the patterns on their cube wall, imagining how much better it would be than the tight confines and demanding deliverables of their current reality.

Chandler was the sort of project where the only apparent limitation seemed to be the collective imaginations of the contributors. The sky was the limit!

The result of such free reign and lack of demands can only be classified as a disaster: After a long period of development by a fairly large and expensive team, the releases thus far have been a rather dubious calendar product, in a very alpha state, and a server product that backtracks on many of the initial goals (for instance the "P2P solves everything" silver bullet).

It's difficult to categorize this book: It really isn't a how-to book at all. Nor is it really a book for a layman, as it frequently goes into a level of detail that could be daunting (sometimes the level of technical detail seems a bit like filler, not really playing a part in the telling of the story. Unnecessary detail for the story, but too little detail to be of interest for a technical analysis).

Really it's a how-not-to book: A reporting of many of the classic mistakes of software development happening in a real, visible project, and how they impacted the progress. The OSAF (Open Source Applications Foundation) team certainly isn't the first to make these mistakes, and they certainly won't be the last, but they had an uncommon honesty and openness to allow a reporter to document their every step.

I would imagine Mitch figured this would all turn out quite a bit differently, with Chandler rocketing to Firefox-like success, and this book documenting the new age approach to software creation.

This book came to mind yesterday when I hit publish on the "Most Destructive Metric In Software Development" entry. In that entry I mentioned that developers abhor using other people's code when they have the option of spinning something "original" themselves, and this materializes several times during the Chandler project. After eliminating the Zope ZODB (a Python object database) from consideration, for instance, and going off and going through several iterations of their own database, the individual behind the decision declared that they "never really looked at ZODB".

Classic.

* - Around four years ago we bought the great book "The Baby Name Wizard", and found it to be a helpful in choosing names for our first child. We're now a month away from welcoming our third child, and have begun the name contemplation game, and I just received a "People who bought the Baby Name Wizard also like..." email from Amazon. Their system has more inputs than I imagined.

Software development is a difficult task to meter.

It's not for lack of trying.

For decades consultants have been evangelizing methods which, they claim, would allow an unskilled, casual observer to easily measure and compare productivity in a contextually agnostic way.

Their ultimate goal: To allow a drop-in manager, with only a superficial knowledge of the activities, skills, and complexities of a task or project, to easily compute metrics by which to dole out the frequency and intensity of whippings and rewards.

[Aside: Before anyone incorrectly presumes any of this is critical of software development managers as a group or individually, realize that it is nothing of a sort: I start with a brief analysis of the goal of such simplistic measures -- most organizations would like positions, including management, to be lower-skill and easier (cheaper) to fill, and such a simplification of the role is definitely in their interest, just as many dream of the panacea of no-skill, factory-type software development -- and then actually question the fact that developers themselves are often guilty of quoting these metrics. 9 times out of 10, developers have only themselves to blame for a lot of the problems with the profession. This is not yet another boring us-versus-them war cry pandering piece, like those that top the meme charts frequently]

The same methods -- if they worked as promised -- could be used to chart project progress ("We're 7868.2 ConsultaMarks towards the 11273.9 estimated for the entire project!").

Instead of relying upon the from-the-trenches observations of Randal the development group manager -- a grizzled vet of software development who manages with a hands-on style by becoming intricately aware of the domain challenges and unique contributions of each team member -- Lynn, the parachuted in middle manager, wants some simple numbers that can be sorted like her mutual fund returns, giving her some available sacrificial lambs when the next diversion-from-massive-executive-fumbles headcount reduction comes due.

Many proposed solutions have come and gone, with the most persistent being the infamous SLOC (Source Lines of Code)/LOC measure.

Source Lines Of Code

SLOC, if you haven't been afflicted with it, is an easily computed count of the number of lines of code in a given project/component/object (although first you have to agree on the definition of a "line of code", and this is a point of debate among SLOC champions). It's often used to count the number of lines of tested, complete code added by a particular contributor (easily accomplished with many source code repositories), allowing for the easy creation of nice little charts like the one above.

SLOC does have some quasi-legitimate uses: Given a common programming language and domain complexity, SLOC magnitude differences have a moderate correlation with general project size, and at the method level it is a rough indicator of gross complexity (see the article FxCop & Cyclomatic Complexity for a discussion of a loosely related metric, which is the number of intermediate language instructions generated from a method).

Applied at the individual or group level, usually as a cheap substitute for good management and project awareness, SLOC measurements are likely to encourage very destructive behaviors: Copy/paste coding, limited reuse of existing code found elsewhere in the organization and the industry, little motivation to prune code where necessary, overly convoluted coding, motivation for employees to only take on trivial coding tasks, and so on.

The Lemon Slice Lemon Roast

Envision a system that ranked cooks by the number of lemons they use to provide a restaurant's service each night: You're going to end up with a lot of dishes featuring copious stacks of lemons, even if ultimately it compromises the quality and organizational health of the establishment. While in some situations you could conceivably roughly compare overall restaurant success by the number of lemons they go through in a period, the comparison only holds true if all else remains equal (e.g. if otherwise the restaurants are very comparable, such as two restaurants serving Thai food): A deli restaurant might use very few lemons despite a healthy customer turnover, where an equally successful Greek restaurant might go through hundreds.

Far more logical would be to measure the number of dishes served -- while still imperfect, it would be much more useful than the LemonMetric. There is no comparable measure, with a similar level of granularity, as "dishes served" in software development (don't even think of mentioning the highly ambiguous "function point" metric as a simile).

Preaching To The Absentee Choir

"Geez...we all know that there are significant problems with the SLOC metric!" many will inevitably retort. "This is old news. You're preaching to the choir!"

"...but having said that, I saw a recent article that claimed that the average developer does {X} lines of vetted code a year. Are they really that slow? Me and my team must generate at least 20{X} a month! I hear that some superstars are responsible for 200,000 SLOC a year. They must be awesome!"

Comments just like that are probably being typed into a TEXTAREA at this very moment.

Why do so many comments about productivity -- even in the comfort of secret No-PHB hideouts -- inevitably elicit gloating commentary about personal SLOC accomplishments? Why do we hear gushing superlatives about the "superstars who push out 100s of thousands of SLOC a year"?

Why do so many in this industry perpetuate this destructive myth?

~SLOC

Let me flip this metric on its head, and state that, if anything, for a certain domain of project, and a certain class of developer, a high rate of SLOC can actually indicate poor programming practices.

In the nascent days of software development, many teams had a compiler or an interpret and that was pretty much it. They were responsible for building the majority of functionality from scratch. The pace of SLOC creation was tremendous (especially given that much of that implementation was trivial, allowing them to code as fast as they typed. Little time needed to be spent problem solving or planning: It doesn't require a superstar to code yet another string copy function).

As time went on, organizations compiled volumes of reusable internal code for all of their domain specific problems.

From an individual developer perspective, no longer was it acceptable to simply "run and start coding". Now you had to spend some of your time learning, assessing, and implementing shared internal code in your projects.

And it wasn't just inhouse: The frameworks and libraries provided with our tools have been growing by leaps and bounds, immediately solving a huge range of traditional problems and tasks with well tested, robust, feature rich solutions.

In the industry as a whole, code sharing has become widespread, with excellent code being available for virtually all common (and even uncommon) tasks.

So many solutions are available in the industry and supplied within our libraries/frameworks, that even organizational code reuse can be indicative of a problem.

Yet somewhere out there someone is hand-writing an FTP client implementation. Somewhere developers are wasting a tremendous number of man-hours by poorly, and unintentionally, duplicating code that exists in the frameworks and libraries that they're already using, or which can be easily found in license compatible open source projects.

Not Invented Here

A part of the reason for this is laziness -- it's a real bother having to look through the documentation and amongst search engine results, and that's hardly as much fun as just coding. Another part of the reason is a classic perception flaw that virtually all developers suffer from: Endless optimism about the capabilities and quality of the code we produce -- which we always think we'll finish much quicker than we really will -- coupled with an unreasonable pessimism about the applicability or worth of code we could source from another group in the organization, or from an external source.

I'm often guilty of these failures of perception, as are the overwhelming majority of developers.

Conclusion

Rarely does a developer actually tread across new ground (and I'm certainly not just talking about business back-end "CRUD" developers -- even in signal processing, embedded development, game development, and other less common branches of software development, most of the "solution" is the integration of existing work in novel ways, adding an envelope and façade of customization).

For the rest of us, our job is partly to generate the generally small amount of niche-specific code, usually aiming to build it with the most concise -- aka minimal -- code necessary, with the bulk of our time being in the analysis and integration of the extraordinary volumes of available solutions.

Where niche, custom code is necessary, generally it will be for a non-trivial task, and the SLOC pace will be unavoidably glacial.

For the overwhelming majority of developers in the industry, the only value of SLOC measures is as a warning sign, not an indication of progress.

My toddler son is a rambunctious little button pusher.

Where there's an On button to be pressed, he's there pressing it.

Again and again and again. On. Off. On. Off. On. Off...

He's an equal opportunity button pusher, so it isn't just power buttons that receive his attention (although he does show a definite preference for power buttons, having figured out the international icon most are adorned with). He also patronizes volume controls (thankfully the speakers have automatic cutoff circuits above a certain power level), input selections, DVD tray eject/return buttons, reset buttons, and every other interface element that affords pushing.

There's a limited ability to reason or debate with a toddler, and my warnings about the peril such activities presents to the vulnerable little ICs seems to have limited effect.

Enclosing every device in protective cages just doesn't seem reasonable.

So I end up with toonies ($2 coins here in Canada) taped over the power button of televisions (it buys a bit of time), computers with the power buttons disabled and the reset buttons disconnected, and for other devices (e.g. stereos, DVD players, radios) it's just a constant state of vigilance, with us ready to physically intercede when he prepares for a button mashing attack.

Oh, for want of a simple control to disable the front panel on these devices. Ultimately most of them are software controlled now, so a simple boolean IsIgnoreFrontPanel would suffice, perhaps with a complex 7-simultaneous-pressed-buttons override to re-enable the front panel.

It would save a lot of these devices the electronic stress of being constantly cycled.

On the same vein, I'd love for a "Microsoft Bob" style simplified interface for my preschooler daughter's PC. Even with the limited potential damage possible with the limited account she uses, context menus and unintended shortcuts are never intentional, and are always just a nuisance.

Various hypotheses have come forth to explain why we have a disproportionate number of ideas and epiphanies in the shower. Explanations as varied as guessing that it's due to the aromatherapy of shampoos, or the brain soothing effect of warm water on our scalp.

I have a simpler explanation: It's one of the very few times when we actually disengage from the world and think. Lathering and rinsing tends to be a very automatic activity, and given that we don't have the option of reading the paper or browsing websites, we're forced to converse with, and be entertained by, our own mind.

A similar result can be experience on your daily commute (this only really works if you drive by yourself, presuming that the traffic isn't infuriating and distracting. I've found commuter trains, buses and carpooling to have no such meditative effect, and instead to be highly detrimental to thought): Turn off the radio and just relax on the drive.

It is astounding how many decisions and ideas are resolved on a 30 minute commute in this island of tranquility. It is the activity closest to meditation in my daily schedule.