Dennis Forbes on Pragmatic Software Development
Subscribe to RSS
 
Tuesday, February 20 2007

Software development is a difficult task to meter.

It's not for lack of trying.

For decades consultants have been evangelizing methods which, they claim, would allow an unskilled, casual observer to easily measure and compare productivity in a contextually agnostic way.

Their ultimate goal: To allow a drop-in manager, with only a superficial knowledge of the activities, skills, and complexities of a task or project, to easily compute metrics by which to dole out the frequency and intensity of whippings and rewards.

[Aside: Before anyone incorrectly presumes any of this is critical of software development managers as a group or individually, realize that it is nothing of a sort: I start with a brief analysis of the goal of such simplistic measures -- most organizations would like positions, including management, to be lower-skill and easier (cheaper) to fill, and such a simplification of the role is definitely in their interest, just as many dream of the panacea of no-skill, factory-type software development -- and then actually question the fact that developers themselves are often guilty of quoting these metrics. 9 times out of 10, developers have only themselves to blame for a lot of the problems with the profession. This is not yet another boring us-versus-them war cry pandering piece, like those that top the meme charts frequently]

February ConsultaMark(SM) ProductoMatrix(TM) Results
Cog Output Proposed Action
Tom 117.6 2% Raise At Year End
Amy 111.2 1% Raise At Year End
Jacob 92.7 Forced Overtime
Serene 85.5 Replace LCD with the 14" VGA monitor from the server room
Nellis 68.0 Creative Dismissal

The same methods -- if they worked as promised -- could be used to chart project progress ("We're 7868.2 ConsultaMarks towards the 11273.9 estimated for the entire project!").

Instead of relying upon the from-the-trenches observations of Randal the development group manager -- a grizzled vet of software development who manages with a hands-on style by becoming intricately aware of the domain challenges and unique contributions of each team member -- Lynn, the parachuted in middle manager, wants some simple numbers that can be sorted like her mutual fund returns, giving her some available sacrificial lambs when the next diversion-from-massive-executive-fumbles headcount reduction comes due.

Many proposed solutions have come and gone, with the most persistent being the infamous SLOC (Source Lines of Code)/LOC measure.

Source Lines Of Code

skyway_lift_bridge

SLOC, if you haven't been afflicted with it, is an easily computed count of the number of lines of code in a given project/component/object (although first you have to agree on the definition of a "line of code", and this is a point of debate among SLOC champions). It's often used to count the number of lines of tested, complete code added by a particular contributor (easily accomplished with many source code repositories), allowing for the easy creation of nice little charts like the one above.

SLOC does have some quasi-legitimate uses: Given a common programming language and domain complexity, SLOC magnitude differences have a moderate correlation with general project size, and at the method level it is a rough indicator of gross complexity (see the article FxCop & Cyclomatic Complexity for a discussion of a loosely related metric, which is the number of intermediate language instructions generated from a method).

Applied at the individual or group level, usually as a cheap substitute for good management and project awareness, SLOC measurements are likely to encourage very destructive behaviors: Copy/paste coding, limited reuse of existing code found elsewhere in the organization and the industry, little motivation to prune code where necessary, overly convoluted coding, motivation for employees to only take on trivial coding tasks, and so on.

The Lemon Slice Lemon Roast

Envision a system that ranked cooks by the number of lemons they use to provide a restaurant's service each night: You're going to end up with a lot of dishes featuring copious stacks of lemons, even if ultimately it compromises the quality and organizational health of the establishment. While in some situations you could conceivably roughly compare overall restaurant success by the number of lemons they go through in a period, the comparison only holds true if all else remains equal (e.g. if otherwise the restaurants are very comparable, such as two restaurants serving Thai food): A deli restaurant might use very few lemons despite a healthy customer turnover, where an equally successful Greek restaurant might go through hundreds.

Far more logical would be to measure the number of dishes served -- while still imperfect, it would be much more useful than the LemonMetric. There is no comparable measure, with a similar level of granularity, as "dishes served" in software development (don't even think of mentioning the highly ambiguous "function point" metric as a simile).

Preaching To The Absentee Choir

"Geez...we all know that there are significant problems with the SLOC metric!" many will inevitably retort. "This is old news. You're preaching to the choir!"

"...but having said that, I saw a recent article that claimed that the average developer does {X} lines of vetted code a year. Are they really that slow? Me and my team must generate at least 20{X} a month! I hear that some superstars are responsible for 200,000 SLOC a year. They must be awesome!"

Comments just like that are probably being typed into a TEXTAREA at this very moment.

coffees

Why do so many comments about productivity -- even in the comfort of secret No-PHB hideouts -- inevitably elicit gloating commentary about personal SLOC accomplishments? Why do we hear gushing superlatives about the "superstars who push out 100s of thousands of SLOC a year"?

Why do so many in this industry perpetuate this destructive myth?

~SLOC

Let me flip this metric on its head, and state that, if anything, for a certain domain of project, and a certain class of developer, a high rate of SLOC can actually indicate poor programming practices.

In the nascent days of software development, many teams had a compiler or an interpret and that was pretty much it. They were responsible for building the majority of functionality from scratch. The pace of SLOC creation was tremendous (especially given that much of that implementation was trivial, allowing them to code as fast as they typed. Little time needed to be spent problem solving or planning: It doesn't require a superstar to code yet another string copy function).

As time went on, organizations compiled volumes of reusable internal code for all of their domain specific problems.

From an individual developer perspective, no longer was it acceptable to simply "run and start coding". Now you had to spend some of your time learning, assessing, and implementing shared internal code in your projects.

And it wasn't just inhouse: The frameworks and libraries provided with our tools have been growing by leaps and bounds, immediately solving a huge range of traditional problems and tasks with well tested, robust, feature rich solutions.

In the industry as a whole, code sharing has become widespread, with excellent code being available for virtually all common (and even uncommon) tasks.

So many solutions are available in the industry and supplied within our libraries/frameworks, that even organizational code reuse can be indicative of a problem.

Yet somewhere out there someone is hand-writing an FTP client implementation. Somewhere developers are wasting a tremendous number of man-hours by poorly, and unintentionally, duplicating code that exists in the frameworks and libraries that they're already using, or which can be easily found in license compatible open source projects.

Not Invented Here

A part of the reason for this is laziness -- it's a real bother having to look through the documentation and amongst search engine results, and that's hardly as much fun as just coding. Another part of the reason is a classic perception flaw that virtually all developers suffer from: Endless optimism about the capabilities and quality of the code we produce -- which we always think we'll finish much quicker than we really will -- coupled with an unreasonable pessimism about the applicability or worth of code we could source from another group in the organization, or from an external source.

I'm often guilty of these failures of perception, as are the overwhelming majority of developers.

Conclusion

Rarely does a developer actually tread across new ground (and I'm certainly not just talking about business back-end "CRUD" developers -- even in signal processing, embedded development, game development, and other less common branches of software development, most of the "solution" is the integration of existing work in novel ways, adding an envelope and façade of customization).

For the rest of us, our job is partly to generate the generally small amount of niche-specific code, usually aiming to build it with the most concise -- aka minimal -- code necessary, with the bulk of our time being in the analysis and integration of the extraordinary volumes of available solutions.

Where niche, custom code is necessary, generally it will be for a non-trivial task, and the SLOC pace will be unavoidably glacial.

For the overwhelming majority of developers in the industry, the only value of SLOC measures is as a warning sign, not an indication of progress.

Reader Comments

I prefer to measure people on a non-numerical basis. We are individuals, after all, management bullshit notwithstanding.

I like to think in terms of roughly Wildly Unacceptable, Borderline, Acceptable, and Excellent. Anything more than four categories doesn't help me much, I find.

How about categories for a report-card on a developer?

My favourite non-metric report-card areas:


1. Does this developer know the technical domain? (Win32 API, SQL, embedded real-time systems, or whatever they are supposed to know)

2. Do they make progress, solve problems, and create solutions independently?

3. Do they use a reasonable set of developer best-practices? (Version control, backups, etc etc)

4. Are they capable of solving difficult problems, or do cry and give up?

5. Does this person know how to Google, or do other things to find pre-built components, that will get a project done without writing everything herself?

And so on. There are so many degrees-of-freedom when assessing a developer's performance, that any aggregate system is going to have flaws. But whenever you lead a team, you're at least going to order, if not number, your team and determine their performance. I think a categorical grade with commments (a report card format) is better than some stupid number, by a long shot, because it's something you can share with your team, and say, 'here are your obligatory strengths and weaknesses, as observed by me, and here's what to work on this coming year'.

What do you think?

Warren
WPostma @ 2/20/2007 2:06:50 PM
I'm in total agreement with you Warren, and your axis of rankings are logical and a very similar to my own.

The difference between your rankings and a largely artificial ranking like SLOC -- and it's why I applaud the former and boo the latter -- is that your rankings require an expert who is actually involved with the work to make worthwhile assessments. The latter is an artificial hands-off "black box" type of metric that is geared more for uninvolved and unskilled readers.
Dennis Forbes @ 2/20/2007 2:27:39 PM
I'm not a developer - I'm a manager so bear with me.

One of the reasons I see programmers writing first and looking for existing code second is due to a couple of factors.

1) Writing new code is fun.
2) It's easier for a programmer to write new code than to understand someone else's code and make the modifications needed to support the requirements.
Miles Archer @ 2/20/2007 7:10:46 PM
nice post - i fully agree with your statements (no criticism, but your statements are well known in the agile community for a long time)

As i always say, we don't need too many people, producing to many code, but (fewer) people that produce less (but the right) code.

Best regards

Mario
Mario Gleichmann @ 2/21/2007 9:02:50 AM
It's not just a matter of libraries, either. It also applies to original code; the worse the developer, the longer the code. I've seen horrific cases of long, repetitive code that ran on and on to accomplish the simplest tasks. I tend to judge my fellow programmers by an inverse SLOC metric: the fewer lines of clear, well-commented code it takes them to accomplish a given task, the better they are.

One of the best programmers I know of is Alan Kay, who invented object-oriented programming. He's currently leading a project to implement an entire operating system, with all the programming tools need to extend it and a nice gui, in just twenty thousand lines of understandable, easily-modifiable code. For anyone interested, here's some info:
http://jaortega.wordpress.com/2007/02/12/reinventing-programming/
http://weeklysqueak.wordpress.com/2007/02/15/complete-computing-system-in-20k-lines-of-code/
http://lambda-the-ultimate.org/node/2021

Miles, you're absolutely right in saying it's not always easy to reuse code. I would go further and say that if code isn't easy to reuse, that might be a demerit for the guy who wrote that code. (But it might also be a demerit for the manager, if too much time pressure was applied.)
Dennis Peterson @ 2/22/2007 8:20:27 AM

Add Comment

Name *:

Email Address:

(your email address is not displayed)
Website:

Comment *:


Dennis Forbes - Dennis Forbes is a Toronto-based software architect and technology writer