Jeff Atwood, of Coding Horror fame, recently rebutted my post "Beginners and Hacks", which itself was a reply to his post "C# and the Compilation Tax".

Jeff makes some great points, but at the outset I have to disagree with his statement "The present model of software development is clearly monkeys all the way down. And if you're offended to be lumped in with the infinite monkey brigade, I'd say that's incontestable proof that you're one of us."

No, Jeff, I don't develop via the Infinite Monkeys Model. It disturbs me that any professional in this industry would volunteer for such a pejorative.

While humility is often a good thing, there is a limit. Every developer can't be Linus Torvalds or John Carmack, but every single developer should still have professional self-respect, and a desire to do and be the best that they can.

As for my denial of membership in the worldwide IMB representing "incontestable proof" that I'm among that group, that comment had me reminiscing about a shop I worked in about a decade ago: A new hire had proposed a questionable set of development changes, some of which I was passionately opposed to. He dismissed such disagreement via a hilarious bit of circular reasoning--

a) If you passionately disagree, you are being "defensive"

b) If you're being defensive per the definition given in a), it must be because you are wrong.

It's a simple, comforting way of dismissing opposing perspectives: Everyone who disagrees is just being defensive because they're wrong. It was so remarkable that it has always stuck with me as an example of self-delusional perception.

Jeff goes on to compare his apparent utter dependence on continuous compilation code checking with squiggle-line spell-checking. Even if I were to accept that simile, which I don't at all, let's humor that comparison for a moment.

I've written about the importance of correct spelling before, and have lauded the integration of automatic, continuous spellchecking in Firefox. I'm typing this entry in Microsoft Word, which has helpfully alerted me to several misspellings (mostly the result of typos).

I greatly appreciate these tools, and how they help me with the craft of writing.

Yet I'm not a professional writer. I am, in actuality, a hack and a beginner.

By noting that differentiation, am I then saying that a professional, dedicated-to-the-craft writer would actively abhor such a tool (see the Frank Navasky character from You've Got Mail as just such an anti-technology luddite)?

Of course not, and that is not and has never been the argument I'm making. Those who jump to such a conclusion are just being defensive, and thus, we have learned, must be wrong. No I'm not calling for editing in notepad, or making shoes like we made them 150 years ago.

Instead I'd wager that you'd find the average professional writer, dedicated to the craft of putting words to print, has dramatically less dependency on such accoutrements than "beginners and hacks": They have elevated their creations to the point where something as rudimentary as spelling no longer represents a significant part of their "problem". They compose their creations so carefully that they're less likely to have such errors in the first place: When every line is a conscientious, careful, considered work of art, it's less likely that a typo-detection utility is as important.

For a blowhard blogger like me, vomiting paragraphs of raw thought into an editor, this sort of handholding is much more important, and the use of spell-checking actually speaks directly to my point. Writing is not my craft, and these literary creations aren't craftsmanship. I've even been known to mix up it's and its on occasion, to the delight of my critics.

This brings us to the crux of the whole "debate": It was never about the advanced functionality of tools, or even the use of said features or whether they "annoy" me or not, but instead I'm speaking to a growing trend of laziness and carelessness in coding, where developers emit screens of code (probably gloating about their remarkable LOC achievements), and after spending as much time fixing up the many automatically detected errors they spend weeks trying to diagnose the much more insidious logic, design and usage errors that almost certainly permeate their creation.

If their work is so carelessly authored that they consider continuous automated correctness checks a heavily leaned upon, necessary feature of their environment, then I wouldn't put much stock in the quality otherwise.

That is the problem that I argued against, simply stating that when you feel naked and abandoned without these assistants, finding yourself automatically doing frequent compilations to catch egregious mistakes, then you've probably lost touch of the craft, and one's work isn't getting the loving attention it deserves.

Every now and then I browse over to the popular Coding Horror blog: It's like turning on an episode of the A-Team and collapsing into a comfortable couch on a hot summer day, ice cubes clinking around in a sweaty cup of lemonade.

It's a nice way to zone out, distracting my consciousness from a vexing problem in hopes that a background process running in the Brain.ThreadPool completes and returns a solution.

A recent entry, C# and the Compilation Tax, piqued my interest, reminding me of a prior entry I posted about Edit and Continue (where I discussed the potential over-reliance on automated helpers.)

In his post, the Coding Horror author observes that in actual use .NET has evolved into a single language environment, with C# increasingly dominating (though it's doppelgänger, VB.NET, is only a marginally different variant -- C# is the Lexus ES300 to VB.NET's Toyota Camry).

Who knew? Would you also believe that the Java Virtual Machine can be targeted by languages other than Java?

Cats and dogs getting along! Microsoft and Linux making up (see also: Microsoft the Patent Troll)! Robert Mugabe the economic development champion!

He goes on to note his disappointment that the Visual Studio IDE doesn't support automatic, continuous compilation for C# projects, which would enable it to quickly highlight egregious mistakes in his code.

In the comments to his post, I offered up the opinion that the reliance upon such a piece of functionality might be detrimental to the craft of software development.

Jeff,

I encourage you to go a week with minimal automated code-correctness checks. When you've gotten to the point where you rely upon continuous automated checks (which can miss a tremendous number of problems where the code is syntactically and type correct, but logically broken), you've seriously harmed your craftsmanship.

[Link]

Dennis Forbes on May 15, 2007 07:10 AM

Mr Atwood replied-

Dennis,

Ah yes, that old logical trap. Better tooling makes us weak! It's a crutch we begin to rely on that cheapens our craft!

This is even debunked all the way back in Mythical Man-Month:

"There is a widespread recognition that debugging is the hard and slow part of system programming, and slow turnaround is the bane of debugging. So the logic of interactive programming seems inexorable."
http://www.codinghorror.com/blog/archives/000026.html

And then, as for the logical fallacy of "easy tools make you weak":

"So let's get real. Bad programmers write bad code. Good programmers write good code. RAD lets bad programmers write bad code faster. RAD does NOT cause good programmers to suddenly start writing bad code."
http://www.codinghorror.com/blog/archives/000090.html

This isn't a zero-sum game; better tools let good programmers work faster. The bad programmers, well, it doesn't really matter what you give them because the output will be the same: disappointing.

Logical fallacy...zero-sum...a reference to the Mythical Man-Month...add in "orthogonal" and you've got a Score:5 on your hands.

In between appeals to authority -- Jeff is a big fan of Code Complete and the Mythical Man-Month, always ready to quote them as the indisputable, final word on all matters relating to software development -- Jeff makes the critical, err, "logical fallacy" of presuming that my questioning of his automatic compilation dependency (to avoid assigning literal integers to strings, apparently) is an indictment of all accouterments of software development.

Hardly, Jeff. It might make for a convenient strawman, but it's certainly not the argument being presented.

I'm not calling for a return to punch-card programming, or even back to the days when we'd reference our 50lbs of Visual C++ reference material to lookup the parameters of an API function.

I even think IntelliSense is quite a great little bit of functionality.

When you're randomly typing garbage into your editor, complaining that the continuous invocation of the compilation shortcut is cramping your style (and your fingers), however, you've developed a serious problem.

Sometimes these features are there to coddle a beginner, carefully keeping them within the painted lines and away from the dangerous electrical sockets along the wall. That would explain why it was a more important feature in VB.NET than C#...not that VB.NET is any more trivial -- it's just a syntactic variant -- but it is the language that beginner programmers are generally guided into.

And to his last point: My experience has been that the best developers just naturally start using less and less "helpers", to the extreme where you have incontestably great developers like Linus Torvalds arguing against fundamental helpers like interactive debuggers.

Many readers may be wondering why the kernel does not have any more advanced debugging features built into it. The answer, quite simply, is that Linus does not believe in interactive debuggers. He fears that they lead to poor fixes, those which patch up symptoms rather than addressing the real cause of problems. Thus, no built-in debuggers.

I'm certainly nowhere near as extreme as Mr. Torvalds -- indeed, apparently Linus has softened his stance somewhat -- but I wouldn't dare to claim his experienced perspective "debunked" because Fred Brooks -- operating in an environment where what is defined as "immediacy" would be glacially slow by today's standards, and whose metrics had nothing to do with Edit and Continue or continuous automatic compilation -- can be selectively reinterpreted and applied against a completely different world.

Sorry, Jeff, but I don't buy the infinite monkeys on an infinite number of keyboards model of software development. I can only envision tools like continuous compilation and edit and continue as the hand-holding of beginners, and the crutch of hacks.

We've been using Microsoft's Team Foundation Server for version control and basic work item/requirements/bug tracking for about 9 months now. All in all it's a good tool, though really it still feels like a version 0.8 beta that got pushed out the door a little early.

The Good

• It's a world better than Visual SourceSafe
  
• Superb integration with Visual Studio
  
• Performance is great locally, but also over a low-speed pipe
  
• Security is well-defined and adequate
  
• Branching and merging works well
  
• Multiple files are checked in as a changeset, rather than individually
  
• The requirements/work items/bugs functionality is half-decent, albeit very unpolished and feature-poor

The Bad

• Some basic functionalities, such as rollback, are missing (which is why I say that it seems like a pre-release beta in ways). There is a powertool hack that lets you rollback a changeset, in which case it checks out all of the affected files, does a get of the pre-checked out files, and then checks in the old file(s), leaving the "rolled back" version in the history.
  
• The application tier is incredibly fragile. If an enterprising team member decides to do some clean-up directly in the TfsVersionControl data-tier database (getting around missing functionality in the tools -- for instance there is no way to permanently delete, aka destroy, in the tools, remarkably, so if someone checks in a 500MB file and you want to remove it, you're forced to do it directly in the database), you will discover that a single missing related record -- the database doesn't define or enforce foreign-keys, so it isn't going to block the DELETE command there -- will cause the application tier to die a hundred deaths, excepting out on null values and other inanities. This is made far worse by the fact that the application tier caches a lot of lookup data, so check-ins/outs will work for a while after these related records were moved, making a simple database rollback impossible. Instead you need to go through every database manually rationalizing all of the data, determining where the application tier is dying.
  
  The worst part is that many of the unfound lookups should be complete non-events, either not displaying the unrelated records, or putting in placeholders while indicating that an error occurred. The idea that it takes down the entire version control system is completely unreasonable, and it really made me question the implementation.
  
• Offline support is non-existent in the front-end tools, with it expecting a constant connection to the source control web services. There is a dubious powertool that manually works by removing the project from source control, and upon reconnection you tell it to reattach and then do a sync.

The Ugly

• The application tier won't install on a domain controller, or on a 64-bit system, both limitation being entirely unreasonable and hindering for small shops.

All in all it's pretty decent, but I think they called it done a little early.

Software development is a difficult task to meter.

It's not for lack of trying.

For decades consultants have been evangelizing methods which, they claim, would allow an unskilled, casual observer to easily measure and compare productivity in a contextually agnostic way.

Their ultimate goal: To allow a drop-in manager, with only a superficial knowledge of the activities, skills, and complexities of a task or project, to easily compute metrics by which to dole out the frequency and intensity of whippings and rewards.

[Aside: Before anyone incorrectly presumes any of this is critical of software development managers as a group or individually, realize that it is nothing of a sort: I start with a brief analysis of the goal of such simplistic measures -- most organizations would like positions, including management, to be lower-skill and easier (cheaper) to fill, and such a simplification of the role is definitely in their interest, just as many dream of the panacea of no-skill, factory-type software development -- and then actually question the fact that developers themselves are often guilty of quoting these metrics. 9 times out of 10, developers have only themselves to blame for a lot of the problems with the profession. This is not yet another boring us-versus-them war cry pandering piece, like those that top the meme charts frequently]

The same methods -- if they worked as promised -- could be used to chart project progress ("We're 7868.2 ConsultaMarks towards the 11273.9 estimated for the entire project!").

Instead of relying upon the from-the-trenches observations of Randal the development group manager -- a grizzled vet of software development who manages with a hands-on style by becoming intricately aware of the domain challenges and unique contributions of each team member -- Lynn, the parachuted in middle manager, wants some simple numbers that can be sorted like her mutual fund returns, giving her some available sacrificial lambs when the next diversion-from-massive-executive-fumbles headcount reduction comes due.

Many proposed solutions have come and gone, with the most persistent being the infamous SLOC (Source Lines of Code)/LOC measure.

Source Lines Of Code

SLOC, if you haven't been afflicted with it, is an easily computed count of the number of lines of code in a given project/component/object (although first you have to agree on the definition of a "line of code", and this is a point of debate among SLOC champions). It's often used to count the number of lines of tested, complete code added by a particular contributor (easily accomplished with many source code repositories), allowing for the easy creation of nice little charts like the one above.

SLOC does have some quasi-legitimate uses: Given a common programming language and domain complexity, SLOC magnitude differences have a moderate correlation with general project size, and at the method level it is a rough indicator of gross complexity (see the article FxCop & Cyclomatic Complexity for a discussion of a loosely related metric, which is the number of intermediate language instructions generated from a method).

Applied at the individual or group level, usually as a cheap substitute for good management and project awareness, SLOC measurements are likely to encourage very destructive behaviors: Copy/paste coding, limited reuse of existing code found elsewhere in the organization and the industry, little motivation to prune code where necessary, overly convoluted coding, motivation for employees to only take on trivial coding tasks, and so on.

The Lemon Slice Lemon Roast

Envision a system that ranked cooks by the number of lemons they use to provide a restaurant's service each night: You're going to end up with a lot of dishes featuring copious stacks of lemons, even if ultimately it compromises the quality and organizational health of the establishment. While in some situations you could conceivably roughly compare overall restaurant success by the number of lemons they go through in a period, the comparison only holds true if all else remains equal (e.g. if otherwise the restaurants are very comparable, such as two restaurants serving Thai food): A deli restaurant might use very few lemons despite a healthy customer turnover, where an equally successful Greek restaurant might go through hundreds.

Far more logical would be to measure the number of dishes served -- while still imperfect, it would be much more useful than the LemonMetric. There is no comparable measure, with a similar level of granularity, as "dishes served" in software development (don't even think of mentioning the highly ambiguous "function point" metric as a simile).

Preaching To The Absentee Choir

"Geez...we all know that there are significant problems with the SLOC metric!" many will inevitably retort. "This is old news. You're preaching to the choir!"

"...but having said that, I saw a recent article that claimed that the average developer does {X} lines of vetted code a year. Are they really that slow? Me and my team must generate at least 20{X} a month! I hear that some superstars are responsible for 200,000 SLOC a year. They must be awesome!"

Comments just like that are probably being typed into a TEXTAREA at this very moment.

Why do so many comments about productivity -- even in the comfort of secret No-PHB hideouts -- inevitably elicit gloating commentary about personal SLOC accomplishments? Why do we hear gushing superlatives about the "superstars who push out 100s of thousands of SLOC a year"?

Why do so many in this industry perpetuate this destructive myth?

~SLOC

Let me flip this metric on its head, and state that, if anything, for a certain domain of project, and a certain class of developer, a high rate of SLOC can actually indicate poor programming practices.

In the nascent days of software development, many teams had a compiler or an interpret and that was pretty much it. They were responsible for building the majority of functionality from scratch. The pace of SLOC creation was tremendous (especially given that much of that implementation was trivial, allowing them to code as fast as they typed. Little time needed to be spent problem solving or planning: It doesn't require a superstar to code yet another string copy function).

As time went on, organizations compiled volumes of reusable internal code for all of their domain specific problems.

From an individual developer perspective, no longer was it acceptable to simply "run and start coding". Now you had to spend some of your time learning, assessing, and implementing shared internal code in your projects.

And it wasn't just inhouse: The frameworks and libraries provided with our tools have been growing by leaps and bounds, immediately solving a huge range of traditional problems and tasks with well tested, robust, feature rich solutions.

In the industry as a whole, code sharing has become widespread, with excellent code being available for virtually all common (and even uncommon) tasks.

So many solutions are available in the industry and supplied within our libraries/frameworks, that even organizational code reuse can be indicative of a problem.

Yet somewhere out there someone is hand-writing an FTP client implementation. Somewhere developers are wasting a tremendous number of man-hours by poorly, and unintentionally, duplicating code that exists in the frameworks and libraries that they're already using, or which can be easily found in license compatible open source projects.

Not Invented Here

A part of the reason for this is laziness -- it's a real bother having to look through the documentation and amongst search engine results, and that's hardly as much fun as just coding. Another part of the reason is a classic perception flaw that virtually all developers suffer from: Endless optimism about the capabilities and quality of the code we produce -- which we always think we'll finish much quicker than we really will -- coupled with an unreasonable pessimism about the applicability or worth of code we could source from another group in the organization, or from an external source.

I'm often guilty of these failures of perception, as are the overwhelming majority of developers.

Conclusion

Rarely does a developer actually tread across new ground (and I'm certainly not just talking about business back-end "CRUD" developers -- even in signal processing, embedded development, game development, and other less common branches of software development, most of the "solution" is the integration of existing work in novel ways, adding an envelope and façade of customization).

For the rest of us, our job is partly to generate the generally small amount of niche-specific code, usually aiming to build it with the most concise -- aka minimal -- code necessary, with the bulk of our time being in the analysis and integration of the extraordinary volumes of available solutions.

Where niche, custom code is necessary, generally it will be for a non-trivial task, and the SLOC pace will be unavoidably glacial.

For the overwhelming majority of developers in the industry, the only value of SLOC measures is as a warning sign, not an indication of progress.

The CYA Application Security Model is the practice of implementing so-called security obstructions primarily to absolve the vendor from blame if something goes awry during everyday operations. This model is usually sold under the pretense of improving user education, or encouraging safer application usage, but that's of minimal actual concern (in reality the opposite outcome -- more risky application usage -- is probable).

An example of the CYA ASM in action is one that pops up a seemingly endless stream of confirmation "Are you really sure you want to do that?" dialog boxes, warning the user against doing what should be completely normal, benign activities.

This pestering, progress-inhibiting assault of a million warnings and confirmations application behaviour is certain to cause the user to enable a "turn off all security" mode (for instance adding every site to "trusted sites" in Internet Explorer), paradoxically making the security situation infinitely worse, but for the vendor this often the desired outcome: At least then they can smirk and blame it on the userbase if what should be a harmless activity compromises their machine.

Didn't you heed the "The Internet could be harmful to security!" dialog box when you attempted to connect to the internet?

The well-known Hanlon's razor states-

Never attribute to malice that which can be adequately explained by stupidity.

While it's a seemingly pessimistic perspective on the capacity of one's fellow human, it is an undeniable truth that we often mistake carelessness, thoughtlessness, or outright ignorance as malicious intent.

Yet it's a more serene existence -- to the benefit of one's lifespan -- to simply assume that the person who dangerously cut you off on the road, for instance,  is just a moron deserving a bit of sympathy, rather than considering him or her a roadway foe challenging you to a deadly battle of wills.

From a software development perspective, however, I think an inverted variation would serve the industry well.

Never blame others until adequately considering the possibility of your own (negligence | carelessness | stupidity).

As a general rule, the denizens of the software development profession -- it certainly isn't limited to this profession, but given that it's the general focus of this blog it's the one I comment upon -- have a tremendous capacity for assuming the worst of others, far before considering the unsavoury prospect that maybe -- just maybe -- it's actually their own mistake or lack of knowledge that's the cause of the issues they face.

It is far too common to cast a wide net of blame, declaring that Microsoft's products are screwing up, the documentation is all wrong, the server is malfunctioning (maybe because of cosmic ray particles toggling memory bits), the installation tool is a dud, and one's coworkers are surely idiots insidiously and maliciously changing code just to make one's brilliant code poetry fail to achieve its momentous glory.

After such hand-waving, blame-weaving dramatics, in most cases the developer realizes that they skipped an obvious step in the instructions, or they forgot to get latest of the entire branch, or they were copying the wrong file or looking at the wrong folder or running the wrong executable, or they were using the class entirely wrong, or they completely misunderstood how the operating system security system works, or they set a global setting a week back that completely changed how the application functions, or they ignored the email and documentation and group meeting detailing system changes, and so on.

They quietly retreat -- don't expect a retraction -- until they repeat the same mistake the next time something doesn't go exactly as imagined.

I've met these people in the industry. I've worked with these people. I've been one of these people.

I think we can all relate to situations where we've railed against a company, a product or a person, only to have the embarrassing realization that we were simply doing something dumb.

And it's not even that doing something dumb is noteworthy: We're humans, and we're bound to make mistakes. The problem is that we often don't even give a moment of time to even the possibility that we could be at fault, instead just assuming the worst of others.

It's far more beneficial to both productivity and team morale to have a little bit of self-doubt in these situations: Assume the worst of yourself before assuming the worst in others.

Epilogue

After hashing out this entry, I wondered why it wasn't appearing on the public blog. After berating various products and services, I remembered that I recently outsourced my DNS (for the reasons described here, using the service recommended by a reader), and forgot to add an entry for the FTP server. Whoops.

As a completely offtopic aside, one of the reasons I switched DNS providers was to have support for a domain SPF record. While it does nothing to stop the tide of pump-and-dump investment scam spams, at least it allows those recipients utilizing the service to immediately dump-bin those that claim to come from yafla (I get about 100 bounces a day, and who knows how more actually get through), knowing that the from: address was forged.

Why is it that "90% done" (and its partner in crime - the ubiquitous "almost done!") is the progress report for virtually any project, over virtually all of its life-cycle?

Why has 90% become the fictional number of choice? Why not the more conservative 80%, or the bolder 95%? Given that it usually has little correlation with reality, they're just as real.

Projects should be reported as 87% done. Even when there's the ominous "we'll solve that problem when we get to it" task maliciously eyeballing you from later in the project plan, or the "it doesn't work and we have no clue why?" runtme reality, still say 87% with confidence and pride.