Getting back into the flow after the holiday break.

In addition to branded software, yafla also provides consulting and custom software development (in the Greater Toronto Area), which has kept me extremely busy over the past while. That's an area of the business that I haven't really written about (primarily because I keep client project and platform details strictly confidential), however I have a case study where the client is interested in getting their name out there, and the need - and from it the solution - are really interesting and noteworthy, so I might start working it in.

As an aside - both the survey component and yaflaColor are currently broken (as of this writing). Both have been updated to .NET 2.0, and for some reason my third-party host has an issue where helper assemblies are being locked by an outside process, possibly due to synchronization issues with the NAS. This should be resolved shortly. I've let them sit broken for the day simply because they're non-critical and it lets the 3rd party solve the problems on their end.

[The static location of this piece can be found at this address]

Introduction

Work habits and conditions vary dramatically between software development groups.

At one extreme of the spectrum are productive, tuned development teams delivering solutions on schedule and on budget, staffed with passionate experts happily building their skills and careers while providing valuable cutting-edge solutions. At the other extreme lie dysfunctional, under-utilized teams. The latter group endlessly catapults magnitudes past estimates of time and money, with a revolving door roster of contemptuously-treated employees.

Most shops fall somewhere in-between -- imperfect, but continually working towards better, more efficient practices and process, all while trying to provide a rewarding, fulfilling, and career-building experience for team members.

This entry is targeted at that continually-optimizing audience, and is written largely for development group managers and team/group leaders, though it's still informative for the general development community (lateral and upward management is critically important, and is the force that drives most process change).

These observations are gleaned from my experience with teams of varying sizes and types, acting in a variety of roles including management, team lead, developer, process consultant, mentor, mentee, and technology consultant. I have directly participated in, or observed, the development process in small, tightly-knit engineering groups stocked with professional electrical engineers, to mega corporations with walking-dead teams awaiting the next executive shuffle to mete out some doom on their orphaned projects.

I've learned from the successes, but I learned even more from the failures.

I've focused on best-practices that are optimal in virtually any team, and which are frequently evident problems, while avoiding those that are more likely to vary significantly between teams and organizations. Work environment, for example, is very subjective and context specific: Some developers and teams work optimally in private, quiet offices with comfy chairs and scheduled communications, while others thrive in hectic, noisy open offices, interfacing directly with the "customer" throughout the day, eagerly setting up development kiosks on whatever can serve as a seat. Some organizations need armies of BAs and BSAs and QAs and UAs, with layers upon layers of heavy process and checks built atop a traditional waterfall development model, while other shops work best with a highly agile, frequent delivery model, completed by multi-role development resources. It isn't possible to declare universal best-practices within those domains.

I also appreciate that most real-world teams have budgets, and that feel-good lists of .COM-style excess aren't realistic or helpful in the current IT climate. Every developer doesn't need every tool available, or every electronic toy on the market, to do their job effectively. I haven't tried to pander to the development community, blowing smoke about how entitled and empowered they should be, beyond the level that is actually beneficial to their career and the role.

On to the list!

Optimizing Software Development - Executive Summary

• Customize Your Development Process - Choose standards and processes that fit your team
• Use Source Control, And Use It Properly- Source control is a requisite of good development
• Formulate Written Standards- Don't rely on by-example or word of mouth standards
• Mandate Regular Status Updates- Have daily status updates
• Organize Your Information- Standardize your information archive structure
• Allocate Appropriate Time For Research- Research appropriately, or enjoy wasted effort and redundancy
• Accurately Track Development Time- Build valid historical data, and time accountability
• Focus On Your Competencies- Now when to outsource tasks to stay focused on your competencies
• Focus On Results Instead of Effort and Sacrifice - Sacrifice and effort should only matter when they yield results

Customize Your Development Process

Select processes, practices, and standards based upon your particular type of organization, type of project, and workforce, and then customize them for your specific needs.

Of course this suggestion seems ridiculously obvious, yet this is an industry rife with groups foolishly mimicking the process of those who appear to be successful.

Whether it's the previously agile vertical-solution shop trying to act like they're a bank, overloaded with process and CMM levels far beyond necessity because that looks like what the "big boys" do, or the banking group trying to adopt XP practices and frequent deliveries because it's the hip new trend that all the blogs are talking about, when their product only requires bi-yearly deliveries under a heavy layer of checks and balances: What works for one type of organization doesn't necessarily work for another, and adopting those processes under the wrong environment will bring defeat rather than success.

Don't act like you're developing life-critical flight control software when you make a P2P app, just as you shouldn't do the opposite. Don't pretend that you're an ISV, or assume that much of what's right for a retail software vendor is transferrable, if you're actually an in-house IT development shop.

This sort of cargo-cult mentality exists at a lower level as well. During the .COM boom, success apparently came from stocking up on Aeron chairs -- Get the chairs, and you'll do well. After the collapse the idiocy continued, with the same chairs now being demonized: Now they could only bring doom, and had to be removed post haste. It's a chair people. In the grand scheme of things, it really isn't going to make that much of a difference, positive or negative.

The same could be said about foosball tables; catered lunches; big, corporate-style meetings; having big cozy offices for everyone, or tiny open plan offices for everyone. Someone seemingly did it to their success, so therefore it was perceived as an applicable practice for everyone.

Just because someone else benefited from it doesn't mean that it would benefit your organization. It could very well hurt your organization.

Use Source Control, And Use It Properly

Source control is the foundation of good coding practices.

Not only does source control keep your code centralized (facilitating organized access for developers, along with easy backups and automated integrity assurances), with the proper use of branching and labels it can greatly increase the agility of your team: Having the comfort of making major changes in a code branch, while retaining the ability to revert to a labeled release or trunk, is liberating, reducing developer paranoia and ameliorating the risk of change.

Whether you're using Microsoft Visual Sourcesafe, Perforce, CVS, or one of the countless other available source control products, it is a critical component in your toolset. It's critical even if you're a single developer working alone in your one-man software development shop.

Source control is also invaluable for change management and auditing, historically tracking how much has changed, and where, over a given period of time, which is a benefit that often pays off handsomely years later. For example auditing the code history to determine when an errant algorithm was introduced to gauge how long it's been screwing up the calculations.

Adequately trained quality assurance teams can even use source control to determine what has changed between releases to tightly focus their testing efforts without relying on the often incomplete word of the developers.

The technique to use source control properly depends upon the tool that you're using, and the standards and needs of the group: Some groups use source control only to check-in production releases, while other groups check code in frequently, with every developer checking in multiple times per day, merging branches at convergence checkpoints. Ensure that you know the feature-set of your source control package inside and out, and that you're following a rational, best-benefit standard of use for your scenario.

If your usage is non-optimal because of limitations of your SCM tool, consider a product change. It really is that important.

Tip: Frequently perform project builds on a clean machine with limited network access by following development environment configuration instructions, and then getting and building from source control. During this process many teams discover hard coded paths, missing dependencies, and solution defects. It's better to detect it early than for development to grind to a halt the day before a big release when the corporate IT department decommissions that old server that supposedly wasn't in use for the past year.

Formulate Written Standards

Standards can be extraordinarily detailed, or as simple as noting which industry standards a team subscribes to (e.g. "All .NET coding will be done in conformance with the Microsoft .NET Design Guidelines"). It could even be the formalization of the lack of specific guidelines in a certain realm (e.g. "Put the braces wherever you'd like, and either tabs or spaces are fine"). Documented standards should cover all aspects of development, including development environment, tools, accepted languages, best-practices, check-in behaviour, naming guidelines, and so on.

Of course standards should be living documents, with occasional reassessments leading to the removal and addition of new points.

If, on the other hand, your standards are word-of-mouth, or "by example" (usually with many conflicting examples throughout the organization), you don't really have standards -- more likely it's acting as a demotivating, inefficient confusion for newer developers to the team, and a constant political skirmish between the veterans of the group: While good developers will happily adapt and conform to the idiosyncrasies and preferences of a group or project, it is frustrating and unproductive when they have to constantly go back and rename tables and rework classes because of unwritten "standards" that exist only in the minds of a few. When developers lose confidence in their code because of uncertainty over trivia such as spacing standards, something is seriously wrong, and people are focusing their care and concern on something that should be thoughtless.

As an added benefit, the process of documenting standards is often a time for rational debate, when "that's the way we've always done it" standards have to be legitimately rationalized and justified, or rightfully discarded. Many teams have improved their process, finally jettisoning unjustified legacy standards, during just such an exercise.

Having concise, documented standards (which can be the documented absence of standards in a particular realm) saves everyone from having doubts about whether they're conforming, eases new developers into the fold, and increases the percentage of good standards while removing the excess baggage. It also facilitates external assistance more smoothly.

Mandate Regular Status Updates

Every project, and every team, should have a daily or bi-weekly status updates by every member to their team leader or manager, or better still to every other member of the team (including from the manager to the team). This is regardless of your development methodology.

This can be a scrum, or a simple daily email stating what work the person is planning for the day, and what they achieved during the past day/period. This practice not only keeps everyone focused on the task at hand, with better time accountability (just like everyone else, and possibly worse as a group, developers are procrastinators), but it also gives a transparency of development that let's the manager, leads, team members, and stakeholders see where impediments exist, providing them an opportunity to help, or to recognize and respond to schedule slip early on, before it becomes a crisis. This transparency is one of the fundamental goals of the CMM process, and the benefits can be largely gained just by frequent, scheduled, mandatory and accountable communications.

Optimally this information will also exist in an archived, centralized, widely accessible location, such as an audio recording of a scrum, or a centralized collection of update notes.

It is critically important not only that this information is conveyed, but that it's actually consumed and retained as well -- management and the rest of the team must pay it proper heed, and they shouldn't wallow over during the day to ask what was previously stated. That undermines the entire process when it turns into a wasted effort that no-one pays attention to.

Organize Your Information

Having lots of specifications, standards, code libraries, discussions, solutions, and examples strewn across the topology of the network is of little use to anyone, and can be detrimental. Outdated documents will continually reappear, and misinformation takes on an unstoppable life of its own.

Centralize and standardize, ensuring that everyone knows exactly where to find all pertinent information on a moment's notice. Everyone should know exactly where they can find, and thus where they can store, specific types of information.

As it is, many teams start off with the "dumping bin"-style repository when they begin a project, planning on sorting it out later, however soon enough there are GBs of files -- many of which have little or nothing to do with the product or project -- confusingly cluttering a massive mess of a directory structure. This puts a serious damper on productivity.

Documents can't be found. People have to constantly interrupt each other to ask where to find something or where to store something. Eventually an expensive initiative will have to be undertaken to separate the wheat from the chaff, at a time when most people have forgotten why all of the various unrelated information is there in the first place. Undoubtedly good data will be removed, and bad data will be retained.

Before you create the first project or product file, decide exactly how you're going to organize the information, planning for the long term. Who will have access to it, and what permissions will they have? How they will access it? What will be contained where?

Tip: Documents such as specifications yield the same benefits of source control.

Allocate Appropriate Time For Research

The first step of any project shouldn't be to hit the IDE, or even to start high level design work. The first step should be to research and understand all competing or similar products, and any and all libraries and tools that could be leveraged to empower your solution.

As I write this, countless teams are spinning their wheels reinventing the Microsoft Application Blocks, or they're developing a data transformation service that could be integrated in Biztalk or SQL Server Integration Services less expensively (and much more flexibly). Perhaps they're developing a web portal that's just an incomplete, second-rate clone of SharePoint, Community Server or Zope.

There are even teams duplicating functionality already existing in the .NET Framework or standard template library because they never spent the time to look.

For virtually any need there are libraries and existing solutions that could either replace the need for the project altogether, or more likely could serve as a foundation of a more robust, more capable solution. If you're in the solution business, the desire to propose and implement the best solution should trump any desire to code for the heck of it.

Developers can't complete that sort of competitive research in an afternoon, or as a footnote, nor should you rely upon off-the-cuff commentary provided in meetings by individuals vaguely aware of the options.

While it varies by project and industry, such research usually requires a significant upfront investment of time before coding begins, allowing the stakeholders and development group to all feel entirely confident that the project is justified and will yield the best possible results. Document every finding and decision to save endlessly having to explain why one decided to develop what seems to be a clone of Biztalk.

Accurately Track Development Time

One of the primary reasons why many shops continue to grossly under-estimate the time required for projects is that they have limited, or fictional, historical real-world time tracking to base future projects upon -- they might have half-a-dozen developers working full-time, but the best they can account for at the end of the week is that a couple of small app issues got worked on.

This is sadly very typical.

Where there is some form of time tracking system in place, often it is of little value: Developers, as a general observation, will try to find ways to under-report the amount of time they spent on a particular development problem or project, as it's a bit of a badge to claim to have "thrown together" complex solutions in a "couple of hours". Furthermore few shops properly partition times (e.g. by project stage and task), further blurring the value.

Implement a time tracking system, and coach your team to accurately and regularly post times (daily, if not more frequently. Any less frequent and accuracy suffers dramatically (as someone with the bad habit of filling out timesheets at the end of the month, anxiously looking at file timestamps to figure out what I worked on when, I say this from personal experience).

Ensure that there is enough granularity that the time reporting is usable. For instance that you can see that competitor research actually took two weeks, instead of the foolishly anticipated 2 hours.

Avoid tacitly encouraging false timesheet entries: Don't act concerned when real world values start appearing, and most tasks are much more time-intensive than originally thought. Adapt and accept that reality differs greatly from the mirage that most people imagine that they see.

Focus On Your Competencies

You have your team of six hyper-intelligent and eager software developers, overloaded creating the next great scrum-tracking application. Based upon some user feedback, you realize that a small percentage of your users may want to interact with the system in a minimal way using J2ME equipped cell-phones, but your team is entirely staffed with .NET experts. Do you a) add a J2ME developer to the team, and just try to expand the J2ME product line, b) let one of your existing developers try their hand at J2ME, c) outsource that non-core component. Most shops would pick b without hesitation, incorrectly justifying it under the guise that it's "free" because it's covered by the payroll.

If you decide to try your hand at it in-house -- despite the considerable domain difference -- the results will almost certainly be a second-rate solution, delivered more slowly and more expensively than just outsourcing the need. This is worsened in that a resource originally dedicated to the core product has been diverted to a non-core need, and will come back to the core product with a need to spend time reacclimating.

This isn't to say that this is always a bad solution: Sometimes it entirely makes sense to build such skills in the core team (if this is a component that will need regular modification, or if it's an expanding customer base), not to mention that the work diversion can prove entertaining and rewarding for a developer.

Undertake non-core competencies -- be it building a one-off data bridge to a partner, putting up a corporate website, designing logos and icons, or countless other non-core needs -- with eyes wide open, realistically assessing the cost to do it in-house. Estimation of the "knowns" is already notoriously bad in software development, but the situation is far worse for non-core competencies, where developers are prone to extraordinary underestimations.

Realistically consider leveraging external solution providers (or even other teams within the organization) where it can keep your group focused -- with appropriate communications, teams will understand that it enhances their value and effectiveness, rather than presenting a risk to their employment. [Note: My organization provides outsourced consulting and development, so I do have a conflict of interests. However we always go in with the intention of empowering and furthering the existing team, and never to supplant them]

Focus On Results Instead Of Effort and Sacrifice

Getting the team working on nights and weekends means little if it saps their passion to produce, expensively inflates the bug count, and increases turnover. Yet sadly a lot of managers work under the delusion that effort and sacrifice are worthy substitutes for results. Indeed, many management techniques are built entirely around deriving maximal effort and demanding complete sacrifice, inventing artificial deadlines and manufactured crises to prod the troops.

The results are invariable destructive and counter-productive.

This is rooted in basic human psychology. Just look at a famous cough syrup with a heavy advertising presence: Surely it must work better because of the sacrifice required to ingest it? Contrast that with the presumption of self-destruction that most attribute to activities that people find enjoyable.

If it's fun or requires little sacrifice, then it can't be beneficial. If it is unpleasant, then it must be doing good. These foolish and unfounded notions have sold a lot of snake oils over the years, and they've supported a lot of human folly.

Savvy Machiavellian developers know how to exploit this management myopia, under-delivering until crunch time, and then putting in some superficial extra effort to get the project done, yielding kudos all around for going "above and beyond". Just by finally delivering long overdue results that they could have achieved in the normal workday. This is especially the case in shops with limited time reporting, and no regular status updates: Show up early in the morning (sending emails as validation), grunt and groan and gripe, shuffle around regularly, and then leave late (again sending emails to validate how committed one is), maybe VPNing in during the middle of the night to again demonstrate great sacrifice.

In some shops that alone is a substitute for any results at all.

On the flip-side, the "slacker" that comes in at 10am, leaves at 4pm, and takes extended lunches somehow keeps generating the bulk of the product design and code, but his apparent lack of effort and sacrifice will be dealt with at the next performance review.

Many heavy and onerous processes are undertaken, and then maintained, not because they've empirically proven themselves to be useful, but rather because the effort involved gives a mentally cheap illusion of achievement. From volumes of documents that no one ever references or validates, to arduous signature gathering exercises. Most of the time they're the mindless completion of a task with no real benefit to the project or the process, but it goes unquestioned as a part of the flow.

Discard all processes that are mechanically completed with little actual benefit. Tweak those that require more effort than necessary, maximizing the results while minimizing the effort.

Effort alone doesn't make your quality better, software better, or team smarter. Focus on results, and welcome and encourage results that come with minimal apparent effort or sacrifice.

Conclusion

Building a top-notch, effective software development team is a lot of hard work, coupled with a bit of luck, but hopefully some of these points have given a bit of food for thought that might encourage some development in the right direction. This list most certainly isn't comprehensive, and I've stayed away from domain-specific practices, but it is a start.

Tagged: [Software Development], [Programming], [Software-Development]

As mentioned previously, many of the "notables" are replicated to a cleaner layout over at http://www.yafla.com/dennisforbes/index.html. This was created to make the content a little more suitable for search engines, for those who don't like the "visual clutter" of a blog style format, and as a more friendly way of solidifying the larger, more time consuming posts (rather than having them quickly cycle out of view for casual readers).

Just CVS'd "Tidy" tonight and hooked it into the auto-rendering process (directly from the .NET transformation code) and it works great, making the resulting transformed code a little more universal, and less prone to flagging warnings in artificially strict processors.

One of the most interesting finally-hitting-the-real-world products shown at the Consumer Electronic Show is Sony's electronic portable reader. Most will scoff at the idea that this will change user habits -- we've had, and largely rejected, the idea of reading eBooks on little PDAs for years -- however there is a core difference with the new Sony device: electronic paper.

Instead of using an emissive technology like LCD (which is emissive through the backlight, with negligible reflective properties) or CRTs, electronic paper (which has been evolving in various niche products for going on a decade) actually changes the reflective property of the surface, absorbing or reflecting the ambient light, just like real paper. Secondly, this first generation release features a dot density greater than most current display technologies, display 170 dots per inch rather than the standard 72 dots per inch (fitting 800x600 in a relatively small display, versus the terrible 320x240 display of most PDAs). A static, reflective, relatively high resolution display not only allows for a good amount of information on the screen at once, but it'll be dramatically easier on the eyes as well.

Supposedly electronic paper in colour form should be available soon, which should be very interesting.

I'm going to keep my eye on this technology, and will really consider it if they can maintain or increase the dot density, but up the size to 8 1/2 x 11. It would be a killer device at that point.

Do you constantly have co-workers, friends and family sending you interesting and fascinating links, leaving you asking where they find them? Do you wish that you had the chops to be a promiscuous link propagator, letting everyone know about the funny hamster dancing Star Wars baby with a bad translation?

Just a few sites are all you need to get the best of the current memes, giving you a lot of source material for endless link emails. Soon enough you'll be a link Jedi, saying "That's ancient news. I saw that three hours ago" to coworkers.

http://www.slashdot.org

Slashdot is a technology focused site that's been around for some time, and it still holds a lot of influence (Slashdot's demise has been greatly exaggerated). While Slashdot is split into sections (for instance a section dedicate to Apache), anything meme worthy will appear on the front page as well.

Slashdot content generally comes from the users (including anonymous cowards) submitting "stories" with links of interest. Stories often consist of multiple links, sometimes just a Google search. A small editorial group at Slashdot selects from the submissions those that they think would be interesting to the community, either relegating it to a single section, or cross-posting to the front page. Due to this process, and a queue of sometimes dubious "news", Slashdot tends to be one of the slower disseminators of fast-spreading information, and you generally won't find chest-beating "we're great!" type stories.

One of Slashdot's greatest assets is the well-proven commenting system, with a user-moderated scoring of each post from -1 (usually indicative of a troll) to 5 (either a great karma-whore pandering to the group think, or a truly insightful post): You can usually glean some additional information, or at least get the lay of the land, by reading comments above a threshold of 3 or so.

http://www.digg.com

Digg is a relative newcomer that, while supposedly technology focused, is more often than not covering general interest links. It has gained mindshare very quickly.

Similar to Slashdot, Digg is split into sections (for instance hardware), again with the concept of a front page containing the best of all of the sections.

Also similar to Slashdot, Digg stories are courtesy of the users. Unlike Slashdot, though, all of the stories are posted immediately, and there is no editorial control at all (beyond dealing with user-reported spam/abuse type submissions). Instead it is up to the users of Digg to promote (by "Digging") those stories that they think are good, or ignore (or reporting as spam/abuse/etc) those that they think aren't. Web 2.0 content democracy in action, although there have been high-profile instances of people gaming the system, automatically creating hundreds of users to digg their own submissions right to the front page. With its rise in popularity, the number of spam stories on Digg has dramatically increased as well.

Due to the lack of editor-lag time-delay, Digg is a good place to find very quickly spreading information. Unfortunately it also suffers from a severe case of group-think, and it's pretty evident at this point that quite a few content providers and bloggers know exactly what to write to elicit a hearty digging: Many of the "front page" entries these days are clannish "Digg versus Someone Else" type stories, and more and more it's focusing on itself rather than the original goal, becoming primarily a banner for the bored to rally behind. Several highly-ranked front page stories have been completely ignorant, mistake-laden misdirections, but given that anyone can Digg, with no proof of any effort (like actually reading the linked site), they get promoted to the front page.

The comment system on Digg is truly terrible, and the general quality of comments is even worse.

Despite all of that, occasionally Digg is good for wildfire spreading news, and can serve as an entertaining diversion.

http://www.reddit.com

Reddit makes no claims to being a technology focused site, and the links are general interest. It has been gaining popularity, albeit nowhere near the levels of Slashdot and Digg.

Using a relatively minimalist interface, Reddit is similar to Digg in that it lets users submit links which are immediately published, and the user community can vote up/down stories, possibly right to the front page. To the best of my knowledge, Reddit has no categorization, and instead is just a lump of links with minimalist titles.

Reddit's comment system is barely used at all, but it's quite a good foundation if a community did evolve.

Reddit definitely needs some work, and some major gaps in the platform and infrastructure are readily apparent. Nonetheless it's a great place to scan for interesting new links, even if you do have to go by a barely useful title to decide if you want to follow it. I expect great things of Reddit when they start doing associative analysis. e.g. I don't care what every artificial account and loyal herd member thinks, but I would like if it could correlate people who "think like me", giving me a personalized frontpage based upon the selections of people who've shown similar tastes to me.

http://del.icio.us/

Delicious is a community bookmarking site where users can host their bookmarks (presuming they aren't something they want to be private), categorizing them through keywords.

Through the power of numbers, the growing trends and popular links can easily be determined. You can even look at what links are popular for a given keyword, for instance concerning Microsoft or SQL or .NET.

http://www.stumbleupon.com/

StumbleUpon really doesn't fit in the same category, but instead works as a toolbar for your browser in which you can thumbs up or down pages, as well as provide page reviews, as you're browsing. You can then "Stumble" around to interesting pages that other users enjoyed. Interesting tool, although it seems that a large percentage of stumbles lead to humorous Flash movies.

http://www.plastic.com/

Plastic isn't really a meme site or link propagator at all. Originally the work of a couple of http://www.suck.com (which was a great site) alumni, Plastic is like Slashdot but with a better community, more editorial original content (rather than just link propagation), and a good comment system. You won't find link-of-the-days at Plastic, but if you just want some interesting discussion it's a good place to look.

http://www.kuro5hin.org/

Kuro5hin is again very similar to Plastic, which is again very similar to Slashdot. Again it focuses more on original content and commentary over link propagation.

http://www.boingboing.net/

Slashdot run through a blog. You'll find many of the same stories that you find on the other meme propagation sites, but often with a nicer package.

http://www.technorati.com/pop/

Technorati's "popular web topics". Most blog content is just someone repeating what other blogs have said (and so on), so Technorati senses that and makes a popular listing. Seems suspiciously like a convenient place to put some affiliate links for books and movies.

http://www.technorati.com/pop/blogs/

The Technorati popular blog list -- Know who's getting the link love right now.

There are others, such as Fark, SomethingAwful, among many, however they're often not safe for work, nor are they appropriate for the sort of memes that you'd want to mass email to all of your friends.

Hopefully this list has provided a good starting point in making you a meme kung-fu champion, emailing and IMing the interesting links faster than your friends.

While there can be wisdom in groups, there can also be tremendous ignorance in groups. To blatantly rip off despair, inc., none of us is as dumb as all of us.

Never has this been clearer than the past couple of months -- a time that makes one question the real wisdom of the masses, and whether many accolades/ diggs/ arrow-ups/ mods is really a good sign of worthy content, or whether it's just an expression of the ignorance of crowds, or a real-world demonstration of low-value groupthink.

During this period we've heard - from virtually every "news propagation" site, all bombarding the same messages - that Google was releasing an amazing new online office suite. Turned out that it was really just the optional bundling of the Java runtime with the Google toolbar (because bundling is always in the consumers' best interest...).

We heard that Microsoft was buying Opera, and then that Google was buying Opera, both of which were untrue (and both of which are hard to rationalize with reality given both company's commitment to competitive alternate platforms). Both originated at very small sites, and were supposedly based upon "insider" tips and info (let's ignore the thorny legal issues about making such a claim regarding the acquisitions of public companies). They were quickly picked up and relayed by countless other sites.

Many of these completely nonsensical stories gained a sort of "truth through repeated assertion": The more sites picked up the same nonsensical story, the more people used their "votes" to give it more visibility, the more real some completely baseless meanderings on someone's blog or commentary site became. These stories took on a life of their own, and if you debated the validity of them, you'd get a laundry list of links to sites all repeating the same fabrication.

This same sequence of events played out among the Riya-fanboyz, many of whom blanketed the meme sites and blog ranks with unsupported claims that Google was buying Riya. In that case, if you followed the "source" back to its origin you would find two bloggers, each pointing to the other as the source of the rumor. Given that the vast majority of bloggers get their "news" from other blogs, this is hardly surprizing.

This is such a predictable event now that you can guarantee lots of hits, and good frontpage action on the meme sites, by simply sticking to the tried and true and claiming that Google is doing X. What is X? Whatever you want it to be. Whether it's running movie theaters, releasing an ultra-low margin PC against any common sense, amazingly building an entire operating system against all precedent, or continuing the Google Office Suite rumor (which includes such hilarious pearls as "GMail's WYSIWYG is 90% of Microsoft Word") -- Savvy bloggers and small-market "news" sites know that it's a sure thing to just imagine up whatever ridiculous concoction one can invent involving Google, and it'll quickly disseminate across the web, earning tens of thousands of links (glorious pagerank), reputation, and traffic that one can redirect to one's own personal projects and businesses.

What starts as baseless, or barely-educated, speculation quickly turns into reality as it propagates.

I've picked just a few of the most known nonsensical stories that took the community sites (both the editorially controlled like Slashdot, and the true community sites like Digg) by storm over the past months, but there have been countless others where someone posts unsupported anecdotes or personal attacks and it quickly jumps to the front page, eagerly given credibility by, in all truthfulness, the clueless: You don't have to know what they're talking about, or even bother reading the supporting "evidence", to help promote a site on these community sites. On some of them nefarious groups have been busy script-creating accounts and voting up their own stories, not even waiting around for the ignorance of anonymous, detached groups to do its thing.

There really isn't a point in all of this, but to say that readers need to consider most of the group think sites more akin to the World Weekly News than the New York Times. While there will be occasional gems, there is a tremendous amount of noise as well.

A current spike of gym-going activity got me thinking about how expensive each visit really has been: If one calculated the total fees that my wife and I have paid for our dual-membership, and divided that by the actual number of visits, I suspect the number would be somewhat scary, and certainly nothing like what we original envisioned.

We've been dutifully paying a monthly fee to the local GoodLife Fitness Club for about six years now. We do this to maintain our super-A+++++ memberships, despite the fact that during this period we've gone intervals of over a year without having used our membership once (the lead-up and follow-up of having a child, and then another, really isn't conducive to that sort of thing). Yet amazingly we keep on paying.

And we're not alone. Of the people I know with gym memberships, all tend towards occasional spurts of highly-visible gym activity (e.g. just before summer), followed by extended periods of non-usage. Eventually something spurs them into action again (e.g. a particular milestone in their life -- like rolling over a decade marker -- or some other unavoidable sign like an expanding midsection) and again they frequent the gym for a month or so, and then it slides again. Repeat.

I don't have the numbers, but I suspect that the entire fitness industry relies upon this behaviour -- They know that at any time only a very small percentage of their active membership roster will take advantage of the facilities, so they can grossly oversell the facilities. For instance a gym might have facilities for 600 users a day, but they can easily maintain a paying membership of tens of thousands without any need for additional equipment or space.

So how do they get us? It's simple.

• Sign-up/Initiative Fees - When we signed up we had the option of paying a marginal sign-up fee with a high monthly fee, or a larger sign-up fee (something like $400) with a vastly reduced monthly fee. Of course we paid the larger initiation fee, as the difference between the two was made up in something like six months of membership payments. Who could choose otherwise? It simply made good financial sense.
  
  Now that we've made that "investment", however, we have a motivation to maintain our membership lest we have to pay it again. Taking advantage of human idealism, they know that most will think something along the lines of "Well next month I'll be able to go a lot more...so I'll just keep paying it lest I have to pay $400 just for cancelling it prematurely". It is a brilliant tactic to avoid cancellations.
  
  Many fitness clubs don't have the draw to earn such a large upfront payment, so instead their sign-up/initiation fee is more theoretical: They're always having giveaways and scratch-card contests where you save 95% off the initiation fee. The purpose is the same, however, given that users will convince themselves not to cancel given the "value" of the waived $500 fee (that they "won", just like 99.9% of the other members) they have in their current membership.
  
  
• The Psychological Challenge of Cancelling - These sorts of memberships never give you a simple website to go to, or an automated phone-number to call. Instead it's usually something along the lines of "Come on in and fill out the membership cancellation form". Not only is that a bit of a hassle, but psychologically one has to present oneself to basically admit defeat: "Here I am. I'm lazy, and I just can't cut it or make the commitment like a responsible person. Where's that cancellation form?"
  
  
• The Frailty of Human Motivation - Gyms know that a lot of people are idealists about human motivation, thinking that the easy part (e.g. getting a gym membership, or buying a bunch of cans of paint, or setting a New Year's resolution) makes it more likely that they'll do the hard part (for instance actually going to the gym regularly, spending a weekend painting, or laying off ice-cream like they promised themselves). Once they've been snared into the gym membership trick, people delude themselves into believing that having a gym membership makes it more likely that they'll actually follow-through by going to the gym regularly, despite all historical precedent. It's a powerful force in keeping people paying a toll to feel that they've partly accomplished that task.
  

In there, somewhere, is a lesson. Whether it's a lesson on how to take advantage of people, or a lesson in saving yourself money, is up to the reader.

If you're in the process of considering a gym membership, for instance, check if your local facilities have day-rates (remarkably few do because they know it would undermine their operation), and if you find a good one then commit yourself to paying the day rates instead of a membership for the first several months. While a quick calculation will convince you that day rates are uneconomical compared to a membership, you'll probably be in for an eye-opener several months down the line when you look back at how often you really went.

If you provide a regular service to customers, take advantage of the gym-technique and publish a "sign-up" fee but then waive it for your "special" customers (e.g. all of them). Presuming the service is actually of value, I'm sure the retention rate will be incredibly high.