Dennis Forbes on Software and Technology   Subscribe to RSS


About the Author
Dennis Forbes Dennis Forbes is a Toronto-based software architect. While focused primarily on the .NET and SQL Server worlds, Dennis frequently ventures outside of this comfort zone into game development and image processing. He has been published in several industry magazines, has been quoted in the Wall Street Journal and has been interviewed by NPR.

He is a vice president and lead software architect at an innovative New York City hedge fund back-office services firm.

Dennis has been working on solutions for the financial, telecommunications, and power generation markets for over 15 years.




The Feed Bag
Feb 24 - TED

 
Monday, January 09 2006

As mentioned previously, many of the "notables" are replicated to a cleaner layout over at http://www.yafla.com/dennisforbes/index.html. This was created to make the content a little more suitable for search engines, for those who don't like the "visual clutter" of a blog style format, and as a more friendly way of solidifying the larger, more time consuming posts (rather than having them quickly cycle out of view for casual readers).

Just CVS'd "Tidy" tonight and hooked it into the auto-rendering process (directly from the .NET transformation code) and it works great, making the resulting transformed code a little more universal, and less prone to flagging warnings in artificially strict processors.

Monday, January 09 2006

One of the most interesting finally-hitting-the-real-world products shown at the Consumer Electronic Show is Sony's electronic portable reader. Most will scoff at the idea that this will change user habits -- we've had, and largely rejected, the idea of reading eBooks on little PDAs for years -- however there is a core difference with the new Sony device: electronic paper.

Instead of using an emissive technology like LCD (which is emissive through the backlight, with negligible reflective properties) or CRTs, electronic paper (which has been evolving in various niche products for going on a decade) actually changes the reflective property of the surface, absorbing or reflecting the ambient light, just like real paper. Secondly, this first generation release features a dot density greater than most current display technologies, display 170 dots per inch rather than the standard 72 dots per inch (fitting 800x600 in a relatively small display, versus the terrible 320x240 display of most PDAs). A static, reflective, relatively high resolution display not only allows for a good amount of information on the screen at once, but it'll be dramatically easier on the eyes as well.

Supposedly electronic paper in colour form should be available soon, which should be very interesting.

I'm going to keep my eye on this technology, and will really consider it if they can maintain or increase the dot density, but up the size to 8 1/2 x 11. It would be a killer device at that point.

  Personal 
Monday, January 09 2006

Do you constantly have co-workers, friends and family sending you interesting and fascinating links, leaving you asking where they find them? Do you wish that you had the chops to be a promiscuous link propagator, letting everyone know about the funny hamster dancing Star Wars baby with a bad translation?

Just a few sites are all you need to get the best of the current memes, giving you a lot of source material for endless link emails. Soon enough you'll be a link Jedi, saying "That's ancient news. I saw that three hours ago" to coworkers.

http://www.slashdot.org

Slashdot is a technology focused site that's been around for some time, and it still holds a lot of influence (Slashdot's demise has been greatly exaggerated). While Slashdot is split into sections (for instance a section dedicate to Apache), anything meme worthy will appear on the front page as well.

Slashdot content generally comes from the users (including anonymous cowards) submitting "stories" with links of interest. Stories often consist of multiple links, sometimes just a Google search. A small editorial group at Slashdot selects from the submissions those that they think would be interesting to the community, either relegating it to a single section, or cross-posting to the front page. Due to this process, and a queue of sometimes dubious "news", Slashdot tends to be one of the slower disseminators of fast-spreading information, and you generally won't find chest-beating "we're great!" type stories.

One of Slashdot's greatest assets is the well-proven commenting system, with a user-moderated scoring of each post from -1 (usually indicative of a troll) to 5 (either a great karma-whore pandering to the group think, or a truly insightful post): You can usually glean some additional information, or at least get the lay of the land, by reading comments above a threshold of 3 or so.

http://www.digg.com

Digg is a relative newcomer that, while supposedly technology focused, is more often than not covering general interest links. It has gained mindshare very quickly.

Similar to Slashdot, Digg is split into sections (for instance hardware), again with the concept of a front page containing the best of all of the sections.

Also similar to Slashdot, Digg stories are courtesy of the users. Unlike Slashdot, though, all of the stories are posted immediately, and there is no editorial control at all (beyond dealing with user-reported spam/abuse type submissions). Instead it is up to the users of Digg to promote (by "Digging") those stories that they think are good, or ignore (or reporting as spam/abuse/etc) those that they think aren't. Web 2.0 content democracy in action, although there have been high-profile instances of people gaming the system, automatically creating hundreds of users to digg their own submissions right to the front page. With its rise in popularity, the number of spam stories on Digg has dramatically increased as well.

Due to the lack of editor-lag time-delay, Digg is a good place to find very quickly spreading information. Unfortunately it also suffers from a severe case of group-think, and it's pretty evident at this point that quite a few content providers and bloggers know exactly what to write to elicit a hearty digging: Many of the "front page" entries these days are clannish "Digg versus Someone Else" type stories, and more and more it's focusing on itself rather than the original goal, becoming primarily a banner for the bored to rally behind. Several highly-ranked front page stories have been completely ignorant, mistake-laden misdirections, but given that anyone can Digg, with no proof of any effort (like actually reading the linked site), they get promoted to the front page.

The comment system on Digg is truly terrible, and the general quality of comments is even worse.

Despite all of that, occasionally Digg is good for wildfire spreading news, and can serve as an entertaining diversion.

http://www.reddit.com

Reddit makes no claims to being a technology focused site, and the links are general interest. It has been gaining popularity, albeit nowhere near the levels of Slashdot and Digg.

Using a relatively minimalist interface, Reddit is similar to Digg in that it lets users submit links which are immediately published, and the user community can vote up/down stories, possibly right to the front page. To the best of my knowledge, Reddit has no categorization, and instead is just a lump of links with minimalist titles.

Reddit's comment system is barely used at all, but it's quite a good foundation if a community did evolve.

Reddit definitely needs some work, and some major gaps in the platform and infrastructure are readily apparent. Nonetheless it's a great place to scan for interesting new links, even if you do have to go by a barely useful title to decide if you want to follow it. I expect great things of Reddit when they start doing associative analysis. e.g. I don't care what every artificial account and loyal herd member thinks, but I would like if it could correlate people who "think like me", giving me a personalized frontpage based upon the selections of people who've shown similar tastes to me.

http://del.icio.us/

Delicious is a community bookmarking site where users can host their bookmarks (presuming they aren't something they want to be private), categorizing them through keywords.

Through the power of numbers, the growing trends and popular links can easily be determined. You can even look at what links are popular for a given keyword, for instance concerning Microsoft or SQL or .NET.

http://www.stumbleupon.com/

StumbleUpon really doesn't fit in the same category, but instead works as a toolbar for your browser in which you can thumbs up or down pages, as well as provide page reviews, as you're browsing. You can then "Stumble" around to interesting pages that other users enjoyed. Interesting tool, although it seems that a large percentage of stumbles lead to humorous Flash movies.

http://www.plastic.com/

Plastic isn't really a meme site or link propagator at all. Originally the work of a couple of http://www.suck.com (which was a great site) alumni, Plastic is like Slashdot but with a better community, more editorial original content (rather than just link propagation), and a good comment system. You won't find link-of-the-days at Plastic, but if you just want some interesting discussion it's a good place to look.

http://www.kuro5hin.org/

Kuro5hin is again very similar to Plastic, which is again very similar to Slashdot. Again it focuses more on original content and commentary over link propagation.

http://www.boingboing.net/

Slashdot run through a blog. You'll find many of the same stories that you find on the other meme propagation sites, but often with a nicer package.

http://www.technorati.com/pop/

Technorati's "popular web topics". Most blog content is just someone repeating what other blogs have said (and so on), so Technorati senses that and makes a popular listing. Seems suspiciously like a convenient place to put some affiliate links for books and movies.

http://www.technorati.com/pop/blogs/

The Technorati popular blog list -- Know who's getting the link love right now.

 

There are others, such as Fark, SomethingAwful, among many, however they're often not safe for work, nor are they appropriate for the sort of memes that you'd want to mass email to all of your friends.

Hopefully this list has provided a good starting point in making you a meme kung-fu champion, emailing and IMing the interesting links faster than your friends.

  Blogging 
Tuesday, January 10 2006

While there can be wisdom in groups, there can also be tremendous ignorance in groups. To blatantly rip off despair, inc., none of us is as dumb as all of us.

Never has this been clearer than the past couple of months -- a time that makes one question the real wisdom of the masses, and whether many accolades/ diggs/ arrow-ups/ mods is really a good sign of worthy content, or whether it's just an expression of the ignorance of crowds, or a real-world demonstration of low-value groupthink.

When Goldfish Escape

During this period we've heard - from virtually every "news propagation" site, all bombarding the same messages - that Google was releasing an amazing new online office suite. Turned out that it was really just the optional bundling of the Java runtime with the Google toolbar (because bundling is always in the consumers' best interest...).

We heard that Microsoft was buying Opera, and then that Google was buying Opera, both of which were untrue (and both of which are hard to rationalize with reality given both company's commitment to competitive alternate platforms). Both originated at very small sites, and were supposedly based upon "insider" tips and info (let's ignore the thorny legal issues about making such a claim regarding the acquisitions of public companies). They were quickly picked up and relayed by countless other sites.

Many of these completely nonsensical stories gained a sort of "truth through repeated assertion": The more sites picked up the same nonsensical story, the more people used their "votes" to give it more visibility, the more real some completely baseless meanderings on someone's blog or commentary site became. These stories took on a life of their own, and if you debated the validity of them, you'd get a laundry list of links to sites all repeating the same fabrication.

This same sequence of events played out among the Riya-fanboyz, many of whom blanketed the meme sites and blog ranks with unsupported claims that Google was buying Riya. In that case, if you followed the "source" back to its origin you would find two bloggers, each pointing to the other as the source of the rumor. Given that the vast majority of bloggers get their "news" from other blogs, this is hardly surprizing.

This is such a predictable event now that you can guarantee lots of hits, and good frontpage action on the meme sites, by simply sticking to the tried and true and claiming that Google is doing X. What is X? Whatever you want it to be. Whether it's running movie theaters, releasing an ultra-low margin PC against any common sense, amazingly building an entire operating system against all precedent, or continuing the Google Office Suite rumor (which includes such hilarious pearls as "GMail's WYSIWYG is 90% of Microsoft Word") -- Savvy bloggers and small-market "news" sites know that it's a sure thing to just imagine up whatever ridiculous concoction one can invent involving Google, and it'll quickly disseminate across the web, earning tens of thousands of links (glorious pagerank), reputation, and traffic that one can redirect to one's own personal projects and businesses.

What starts as baseless, or barely-educated, speculation quickly turns into reality as it propagates.

I've picked just a few of the most known nonsensical stories that took the community sites (both the editorially controlled like Slashdot, and the true community sites like Digg) by storm over the past months, but there have been countless others where someone posts unsupported anecdotes or personal attacks and it quickly jumps to the front page, eagerly given credibility by, in all truthfulness, the clueless: You don't have to know what they're talking about, or even bother reading the supporting "evidence", to help promote a site on these community sites. On some of them nefarious groups have been busy script-creating accounts and voting up their own stories, not even waiting around for the ignorance of anonymous, detached groups to do its thing.

There really isn't a point in all of this, but to say that readers need to consider most of the group think sites more akin to the World Weekly News than the New York Times. While there will be occasional gems, there is a tremendous amount of noise as well.

  Blogging   Personal 
Wednesday, January 11 2006

A current spike of gym-going activity got me thinking about how expensive each visit really has been: If one calculated the total fees that my wife and I have paid for our dual-membership, and divided that by the actual number of visits, I suspect the number would be somewhat scary, and certainly nothing like what we original envisioned.

We've been dutifully paying a monthly fee to the local GoodLife Fitness Club for about six years now. We do this to maintain our super-A+++++ memberships, despite the fact that during this period we've gone intervals of over a year without having used our membership once (the lead-up and follow-up of having a child, and then another, really isn't conducive to that sort of thing). Yet amazingly we keep on paying.

And we're not alone. Of the people I know with gym memberships, all tend towards occasional spurts of highly-visible gym activity (e.g. just before summer), followed by extended periods of non-usage. Eventually something spurs them into action again (e.g. a particular milestone in their life -- like rolling over a decade marker -- or some other unavoidable sign like an expanding midsection) and again they frequent the gym for a month or so, and then it slides again. Repeat.

I don't have the numbers, but I suspect that the entire fitness industry relies upon this behaviour -- They know that at any time only a very small percentage of their active membership roster will take advantage of the facilities, so they can grossly oversell the facilities. For instance a gym might have facilities for 600 users a day, but they can easily maintain a paying membership of tens of thousands without any need for additional equipment or space.

So how do they get us? It's simple.

  • Sign-up/Initiative Fees - When we signed up we had the option of paying a marginal sign-up fee with a high monthly fee, or a larger sign-up fee (something like $400) with a vastly reduced monthly fee. Of course we paid the larger initiation fee, as the difference between the two was made up in something like six months of membership payments. Who could choose otherwise? It simply made good financial sense.

    Now that we've made that "investment", however, we have a motivation to maintain our membership lest we have to pay it again. Taking advantage of human idealism, they know that most will think something along the lines of "Well next month I'll be able to go a lot more...so I'll just keep paying it lest I have to pay $400 just for cancelling it prematurely". It is a brilliant tactic to avoid cancellations.

    Many fitness clubs don't have the draw to earn such a large upfront payment, so instead their sign-up/initiation fee is more theoretical: They're always having giveaways and scratch-card contests where you save 95% off the initiation fee. The purpose is the same, however, given that users will convince themselves not to cancel given the "value" of the waived $500 fee (that they "won", just like 99.9% of the other members) they have in their current membership.

  • The Psychological Challenge of Cancelling - These sorts of memberships never give you a simple website to go to, or an automated phone-number to call. Instead it's usually something along the lines of "Come on in and fill out the membership cancellation form". Not only is that a bit of a hassle, but psychologically one has to present oneself to basically admit defeat: "Here I am. I'm lazy, and I just can't cut it or make the commitment like a responsible person. Where's that cancellation form?"

  • The Frailty of Human Motivation - Gyms know that a lot of people are idealists about human motivation, thinking that the easy part (e.g. getting a gym membership, or buying a bunch of cans of paint, or setting a New Year's resolution) makes it more likely that they'll do the hard part (for instance actually going to the gym regularly, spending a weekend painting, or laying off ice-cream like they promised themselves). Once they've been snared into the gym membership trick, people delude themselves into believing that having a gym membership makes it more likely that they'll actually follow-through by going to the gym regularly, despite all historical precedent. It's a powerful force in keeping people paying a toll to feel that they've partly accomplished that task.

In there, somewhere, is a lesson. Whether it's a lesson on how to take advantage of people, or a lesson in saving yourself money, is up to the reader.

If you're in the process of considering a gym membership, for instance, check if your local facilities have day-rates (remarkably few do because they know it would undermine their operation), and if you find a good one then commit yourself to paying the day rates instead of a membership for the first several months. While a quick calculation will convince you that day rates are uneconomical compared to a membership, you'll probably be in for an eye-opener several months down the line when you look back at how often you really went.

If you provide a regular service to customers, take advantage of the gym-technique and publish a "sign-up" fee but then waive it for your "special" customers (e.g. all of them). Presuming the service is actually of value, I'm sure the retention rate will be incredibly high.

  Personal 
Wednesday, January 11 2006

an overwhelming percentage of software developers imagine themselves to be far above the norm - they themselves conveniently fit that top 2% echelon, they believe

Several readers have emailed to ask how my "Optimal Software Development Processes and Practices" entry -- which was surprisingly well received -- compared to several better-known software development guideline papers. For instance Joel Spolsky's superlative "The Joel Test: 12 Steps To Better Code".

"Read them all and then adopt what sounds right to you," I replied.

IMG_3307

To expand upon my answer a bit, my entry is a listing of high-level management practices that apply to virtually any team involved in virtually any type of software project. If you have accountable software developers; you're focused on solving the right problem, using the right platform, tools, and technologies (given that they've done the research, and they're not tied up having camelCase/PascalCase wars, developing a redundant GFA Basic solution for the Atari ST); you have a history of accurate, detailed time information to draw upon for estimates and to really know the capacity of your team -- which usually indicates a grizzled, experienced management that is more attuned with reality than is the norm; you have full transparency of how the project is progressing and what is getting done; then you have a much better probability of success.

Yet these basics -- these core fundamentals -- are sadly missing from many software development teams.

It is, however, fairly typical of papers of this genre to look for something seemingly unique -- some sort of hook -- that'll grab the masses as the silver bullet. One which they can quickly implement in their process (for instance a poorly used and abused bug tracking application, or a hashed out daily build script that has been failing for months but no one has noticed because it has no utility, or partnering up developers in pair programming, or implementing code reviews, or pursuing test-driven development) and claim success. "We score a 9! Awesome!" they gloat, high fiving and then returning to the continuing cycle of project failures.

One common silver-bullet hook these days is the classic "only hire the best!" mantra, and I thought it worthy of special mention.

"Only hire the 2% of software developers!" these advice papers crow. "We ensure that our software is top notch by rigorously putting our interviewees through 19 interviews, by asking clever brain-teasers from How Would You Move Mount Fuji, and then by having them design and implement an embedded operating system on a whiteboard!" This is usually accompanied by a "how to spot a great developer!" listing which could better be described as "How the author would find themselves in a pile of applicants".

This arrogance isn't limited to the authors of such silver-bullet recommendations, though. In conversations and forum threads discussing such a top X% recommendation, one quickly comes to realize that an overwhelming percentage of software developers imagine themselves to be far above the norm -- they themselves conveniently fit that top 2% echelon, they believe, so of course they agree: Let the banks and the IT shops take the other 98% (the duds), because the top 2% are too busy making webmail interfaces and bug tracking applications (as ridiculous as that sounds).

The problem with such a "only hire the best!" proclamation, however, is that it's meaningless from the start -- How do you select the top X%, given that the number of attributes of a software development team member is virtually infinite? How do you quantify prospective talent universally like this given that what really matters to you varies dramatically based upon what you're developing, what fit the developer will be in a team, and how you're going to manage them?

If you're a poor manager and you fail to utilize a talent properly, or if you choose the wrong talent for a particular position, it can be an absolute disaster, or at least a non-optimal situation.

The top 2% in one realm was the bottom 2% in the other.

To draw from personal observations, I've known some remarkably intelligent developers who would absolutely storm a path of success in one type of project, in one type of role, basically designing, implementing, and delivering a remarkable project almost single-handedly. Yet they would wallow fruitlessly for months when put on another project. Whether it was because they had no motivation or passion for the new project, or that they simply couldn't adapt to the new requirements -- which is remarkably common. Great developers, young and old, often have a niche where they are remarkably strong, outside of which they falter -- the results were the same: A great developer was being wasted on a project that didn't leverage their strengths, and a project had a deadweight superstar that couldn't catch on. The top 2% in one realm was the bottom 2% in the other.

There are many projects and situations in which John Carmack and Linus Torvalds would be complete disasters.

There are many who'll disagree with me. "That's hogwash!" they'll say. "I can do anything well, it's just that IIS is a piece of crap, and the MSDN documentation is all wrong, and Microsoft is evil, and there's this strange behaviour with the MSXML Library -- that wouldn't happen on Linux! -- and that really screwed up my timeline, and....". Sure.

Of course this could be countered by claiming that the "top 2%" refers to whatever pet dogma the speaker subscribes to. "The top 2% writes test cases before writing code, comments all code (in RoR of course), and does their timesheets by 4:30pm every day".

And thus it is explained how every developer can claim to be the top 2%, and how every elite-herder can claim to hire only the top 2%: They simply adapt the meaning to describe whatever practices they adhere to, whatever techniques they use, and their personal skillset, or by simply describing the candidates who made themselves available to them, and who accepted the position.

In that way we can all be #1.

Friday, January 13 2006

I've described why I blog several times previously, including within my very first blog-style entry back on September 4th (this blog just passed its 4 month birthday). The motivations are the same motivations that have pushed me to post online "papers" for about a decade now: Reputation, a bit of an outlet for thoughts (it is therapeutic), and of course to maintain or gain some namespace in the internet world (which really means PageRank these days), or at least to avoid namespace loss.

The namespace competition is much more competitive these days than it was a few short years ago, with the technical ease of blogging encouraging a lot of very capable entrants. If you're stand still, then you're falling behind quickly.

It really has been an uphill battle trying to get my thoughts noticed, but I'm finally at a remarkable point where I post something -- something written late at night in a brief interlude between ending "real work" for the night and hitting the hibernate button -- and discover that many thousands of visitors have passed through the next day. Often something makes its way onto the meme sites courtesy of one of the readers who thinks it's worth sharing (thank you kindly to those good folks). That's pretty neat, and is very rewarding.

visitors
The Reddit Effect

It's especially satisfying given that I seldom spend more than 3 hours on an entry, even on the longer pieces, usually spreading it out over several days. On an average entry -- for instance the spelling entry (which also saw many thousands of visitors per day), or understanding daylight savings time -- I spend approximately 30 minutes, going back on re-reads to clean up the flow. There are also coffee-break length entries like this one.

I usually have the general concept fermenting in my mind for a while, and I type quite quickly so it's really just a process of transcribing it out and dealing with the technical errata (like Radio Userland's propensity for being overly helpful).

To make them a bit more visually appealing, I tend to take a few more oddball pictures during day to day activities than I normally would, but that's fun and is hardly a chore.

Entries are often much more readable after about a week, and if blogs had a wiki-style history you would see constant minor wording changes and paragraph reworks.

The few pieces that have earned considerable attention have been real eye-openers, and in some ways they encourage one to move to the dark side: It's very easy to see, for instance, exactly the type of content that earns a lot of attention, and it would be terribly easy to write pandering pieces, saying everything that the crowd wants to hear, and that which they want restated (which is what many use the popularity-measuring sites for -- to push their own agenda by promoting sites that state their opinion, and by silencing dissenting opinion by punishing those that don't. Look at Slashdot comment moderation as a great example of this).

It has been a great experience, and I greatly appreciate everyone who stops by. Thank you very much for a moment of your time.

  Blogging 
Earlier EntriesLater Entries

Dennis Forbes