Dennis Forbes on Pragmatic Software Development
Subscribe to RSS
 
Monday, March 13 2006

A recurring argument on software development boards, often with a backstory involving a developer teaming up with an idea-man and then having second-thoughts about the equity split, debates the relative value of ideas versus implementation. Who is worth more to the team: The one who did the hard work of coding, or the guy who thought up the web app in the first place?

The predominate opinion is usually that the implementation is extraordinarily valuable, while the idea is close to worthless. "Ideas are cheap" the posters often claim. "There are an endless number of good ideas out there."

"Anyone can think up good ideas. Few can put them into practice."

So I ask you: What value are ideas? If you were a developer that teamed up with someone who had a clever idea for a web app, even perhaps just a couple of interesting twists that give it a potential competitive edge, how much value would you give their contribution?

This is a very contentious issue in the software development community, and it's one I plan on addressing in an upcoming entry.

P.S. This entry uses an inline IFRAME for a survey, and it may not appear in some aggregators or RSS readers. If that's the case, visit the entry directly.

Tuesday, March 07 2006

Summary: Fake job listings, or real jobs backed by a selection process that filters out everyone, can earn your firm adversaries and detractors. They need to be identified for what they are -- a cheap, transparent gimmick.

A Typical Model

Following a fairly typical business model in this niche, yafla is a small consulting and contract development shop in a mid-sized market, using that business income to support the development of derivative products and businesses (such as the soon-to-come www.360notes.com), along with licensed ISV technologies.

We love the consulting/development market. We're extremely good at it, we enjoy the diversity and challenge, and we have no plans of ever abandoning it (at least until the world's software and IT needs have been completely satisfied once and for all). Yet of course we'd also love to build the next great shrinkwrap or web app hit, and are working towards that goal as well.

One thing we haven't done, however, is to pretend that we're bigger, more important, or more exclusive than we really are through the cheap but common technique of bogus or never-ending job listings -- We haven't built a pompous careers section bulked up with imaginary roles. We haven't provided descriptions of how extraordinary yafla talent is that only walk-on-water technical magicians with PhDs should consider joining the ranks of Extraordinary Gentlepeople.

The Lofty Requirements and Imaginary Jobs

under_skyway

What I'm talking about, of course, is the sadly prevalent practice of many organizations, from tiny to huge, to endlessly list artificial positions that don't actually exist, or to advertise theoretical positions that no one short of a deity could actually fill. This is a technique utilized by a diverse range of companies, from the tiny start-up that immediately has a prominent "careers" section comprising one third of their web property, to the huge organization like Microsoft's Canada division that lists the same positions for a year on end.

In this case I was spurred to write after seeing yet another micro-web firm -- one which I know has less than half a dozen employees and hasn't grown in years -- with a comprehensive job listing section including endless exhortations for only the most extraordinarily gifted and rare to join this exclusive crew. I imagine the endless ranks of prospects wasting their time jockeying for positions that will never be filled and it saddens me a bit.

Advertising false or theoretical positions is a cheap way of conveying the illusion that the perpetrator is an elite company on the go, so in demand that human resources needs outweigh the need to acquire clients (sort of a social proof), providing lofty requirements in hopes that you'll believe that all of the existing staff achieves the same. It attempts to cow competitors into thinking that they're so successful that they're chomping at the bit for new employees, but are thwarted by a workforce that is incapable of meeting their amazing skill requirements.

In response to these ads, countless prospective applicants carefully craft cover letters and customized resumes. They eagerly apply, imagining their future with this fabulous organization.

Many times their application either times out and yield an automated rejection letter, or immediately gets rejected by a human resources department that is filtering applications by a wide range of criteria unstated in the job posting (the classic being the secret compensation cutoff -- ask for less and you'll get through and underpaid, but waste your time submitting for a job that doesn't fulfill your financial requirements and you'll get instantly rejected).

Not named Smith. Not named Singh. Too much corporate experience. Too little corporate experience. Too presumptuous in its use of the greeting "A very good day to you". The arbitrary and often ridiculous filtering criteria can effectively eliminate anyone.

Cheap and easy for HR, so everyone wins. Right?

The Negative Repercussions

Of course they don't.

On the other end of the equation is a real person that wasted their time and effort creating a submission for jobs that often don't even exist, or which they don't meet due to hidden requirements -- people who often already have jobs and are sacrificing their limited free time -- only for their efforts to sit stagnating in an inbox or database for years. If they're lucky they might get rashly rejected despite fulfilling all of the requirements, and then some, when some arbitrary time-out mechanism fires off a form rejection letter, or a haphazard HR rep blanket rejects all pending resumes.

The resentment builds. The goodwill of the organization suffers.

These rejectees often have decision making authority in their organization, and over the years they grow into more powerful roles. For years they nurse the wound that their perfect application for the big XZZ corp position was rejected out of hand, without as much as a phone call or personal contact allowing them to demonstrate their worth. Often it subconsciously colours their choices, be it as simple as supporting and advocating open source products in defiance of the careless Microsoft rep that canned their submission, or second guessing whether the firm should automatically move to .NET, or whether there should be a competitive showdown between it and J2EE. Even a minor antagonism can substantially change internal advocacy.

Of course choices should be made based upon empirical facts and unbiased analysis, but as human beings we are consciously and subconsciously affected by emotions and biasSmaller firms of course have less ramifications for partaking in this abuse.

Smaller firms often have no downside to printing such fake ads -- apart from killing a bit of your soul abusing people like that -- given that spurned suitors often have no business relationship with them outside of vying for a job. These firms use such ads in the theory that "no publicity is bad publicity". Compare this to Microsoft that relies upon the decisions of millions of people, and suffers from a million tiny wounds when an HR department gets sloppy.

My Personal Experience

I can personally relate to this problem to a degree.

Many years back, when I first moved to the greater Toronto area, I applied to a Microsoft Canada position that was pretty much a perfect match for my skills -- the ad seemed like it was written specifically with me in mind: I had what seemed to be a perfect mix of skills, experience, education and certification.

I had watched this job listing sit on the job site for many months, and figured I'd finally give it a try, so I wasted an hour of my time putting together a perfect resume, and then reformatting it for their absolutely terrible, archaic online resume building tool. I looked forward to the inevitable telephone interview when I would wow them more with my incredible communication skills and demonstrated intelligence.

A week later I got a form rejection letter informing me that I wasn't suitable for the position.

No phone call. No email questions. No follow-ups at all. Just a blanket, uncontestable (from a no-return bogus email address, which is a classy touch) rejection by the machine of HR at Microsoft.

The job listing stayed up for as long as I bothered watching. Every day that it sat there unfilled I stewed over the fact that I wasn't given even the courtesy of a brief phone interview.

I've never been a Microsoft zealot, and nor did that organization represent my dream job, but it admittingly did burn getting rejected in such a fashion (I'd much rather have bombed an interview and had myself to blame). My choices of Microsoft technology have always been driven by facts and pragmatism, so it wasn't like I was going to stomp my feet and embrace Linux to spite the Microsoft machine, but it did make me give alternatives a second look.

In all honesty it's probably made me a little more antagonistic towards Microsoft Canada employees, making me question "so what's so special about this guy? I don't get a phone call yet this idiot works there?"

For someone more emotionally invested, though, it probably could have made them an enemy. I have to wonder how many avowed Microsoft enemies, spreading the anti-Microsoft word far and wide, were molded when the human resources machine rashly stomped on their dreams.

The same question could be raised for other organizations with the same hiring practices, and for startups that build a facade of size by listing numerous positions that don't actually exist.

Not Everyone Is Employee Material

Of course many applicants aren't appropriate or optimal for the position they're seeking, so job seekers have to accept the fact that even getting past the initial filter is a remote possibility (especially given the one-click applications possible at many of the job sites: A single listing can yield tens of thousands of mechanically submitted applications, so even the best prospect can get lost in the noise, not to mention that listed positions are often internally filled, and listed only as a matter of process), and as always getting considered is often best achieved by network contacts and organizational insiders rather than cold resume submissions.

Nonetheless, fake job listings are quickly transparent to all, and unfilled roles that waste the applicant's time can earn an organization lifetime enemies.

On top of that, the practice of keeping some requirements secret wastes everyone's time. Apart from often unstated diversity staffing requirements, organizations often fail to publish the compensation band for a job. For prospects with financial needs beyond the position, it is an enormous waste of time and effort, and it fills HR inboxes with prospects that will never accept the position.

Monday, March 06 2006

Mozilla has announced the winners of the previously mentioned extend Firefox contest, with the three grand-prize victors carrying home a beefy PC, along with a considerable amount of developer credibility. Category winners earned some decent prizes as well.

lift_bridge

Congratulations to the winners on a job well done.

Having said that, I've tried out the winning extensions, and I have to confess that I'm underwhelmed. Not to take away from their accomplishments, but for a challenge of such magnitude, for a core product with millions of software developer fans, I expected some awe-inspiring, revolutionary products to emerge.

Firefox has an enormously robust and feature rich extension model, where almost anything is possible, yet the roster of available extensions is dominated by trivial tools with simplistic, archaic interfaces, too many of which seem like hack jobs (the exception being the extensions by big corporations -- Corporations that spit and polish their offering to reap the benefits of tracking your habits and encouraging you to use of their search).

Of course I'm complaining about something that is generally free, so as the old saying goes, you get what you pay for...In fact you get far more than you pay for, but it demonstrates that there are limits to the sacrifice and resources someone will commit to a product that they find difficult to monetize.

Not only is it close to impossible to achieve revenue from an extension unless you're pushing a different product (such as search), but the skills and technologies that you learn building extensions is hard to leverage for professional gain (e.g. knowing XUL and the Mozilla extension API is of marginal value outside of building extensions, where the C skills gained doing kernel hacking has tremendous professional value).

Nonetheless, at least the winners weren't gadget clocks. Gadget clocks and basic arithmetic web service examples always strike me as a sign of a technology or platform that is being oversold.

Friday, March 03 2006

I've received a couple of fantastic comments about troubles that people have faced adding items from here to their del.icio.us bookmarks, namely because Radio Userland uses a constant title for all entries (and del.icio.us automatically uses the title, so three different entries get the same title if you fail to manually override its choice). The common title problem was one of the reasons I created the notables static listing, though of course that listing is just a subsection of entries.

To help with this issue, I've added quicklinks below each entry to add it to your del.icio.us bookmarks, furl bookmarks, to Digg it or to Reddit it (which will link to an existing entry if one is already on there), and to check for Technorati links (there are seldom Technorati links because most of the readers here aren't bloggers, or they aren't the sort of bloggers that comment on every site they visit. I'd get a big boost in the Technorati rankings if I started pandering to the incestuous blogging community). I've mirrored these items to the static section as well.

  .NET   Blogging   IT   Software Development   SQL 
Wednesday, March 01 2006

[I don't normally post entries about other people's blog entries, however this relates to a topic that I've meant to cover for some time.]

Caught an interesting blog entry about Web 2.0 hype via Reddit the other day, and thought it worth a mention. It's a rather cynical take on the current "Web 2.0" phenomenon, noting that many of the recent Web 2.0 sites and services have no rational current or future business model.

To recycle a rather dated joke--

  1. Make a really cool AJAX-equipped, streaming-video website with das Blinkenlights
  2. Get lots of visitors and users
  3. ???
  4. Profit!

This is happening to a scale not seen since the disastrous .COM bubble of 1999-2001, and many of the rising stars of the internet world seem to exist purely in the hope that Yahoo or Google or Microsoft will snatch them up for tens of millions in their quest for Internet dominance. Sometimes it really does happen.

Most of us would love to be on the cheque receiving end of such an acquisition.

Being a software developer, entrepreneur, and technology consultant, I've been in contact with a lot of people who dream of branching out on their own, making their own killer software application or hugely popular web application. They imagine how they'll be sitting on the pile of lucre when the Adsense revenue and subscription fees start pouring in, or when the huge buy-out cheque comes in the mail, all while enjoying a reputation as a technical genius (or, in common media parlance, as an "internet whiz").

I offer my support and encouragement for their plans, but temper it with a bit of skepticism. My doubt arises because 9 times out of 10, or more like 97 times out of 100, their plan involves--

  • Making a killer web application for geeks to hang out. Maybe make subscription levels and members only areas for revenue
  • -or- Making cool component libraries for software developers, maybe some cool AJAX-like components
  • -or- Making tools for the software development community (bug tracking, process management, etc)
  • -or- Making cool web development tools, like an aggregator to mash up all of the Google Maps and Flickr photos with Amazon ISBNs
  • -or- Something else to cater to the technically adept geek community.
Rockton World's Fair Llama

This is what they know, so naturally they want to spin their observations and experience directly into business success. Whether they're pro-bloggers, or elite software developers, they feel that they know the needs of the market -- they know the itches that need scratching -- not to mention their natural desire to earn some credibility among their peer group of geeks and developers.

There are a number of problems with this plan.

  • Every other software developer/web app creator is thinking and doing exactly the same thing: A small market is being grossly oversupplied by for-profit ventures, not to mention a huge cadre selfless gratis/libre advocates.
  • The target is, on average, a smarter crowd. Smarter not only in general intelligence, but more importantly in the specifics of the products and technologies that you're offering.
  • This target market has been observed as being very thrifty (just ask how the tip-reliant in Las Vegas felt about Comdex).
  • A sad reality is that some of the target market resents seeing peers financially succeed. If you make a commercial product or solution, expect your forums to be overloaded with the resentful pointing out every marginally related open source competitor or freeware. Expect users to claim their constitutional right to use your information and services free of advertisements and restraints.
  • The target market is technically capable of circumventing ads, copy protection, and just about everything else you might throw up to earn or protect revenue.
  • The market is being conditioned to believe that software and services should be free (as in beer), or at least the costs should be submarined into something else.

This isn't to say that revenue isn't possible -- there are quite a few solution providers and websites doing very well targeting the developer/geek audience -- but instead is just an observation that it's one of the most difficult markets to tap. While many other domains sit underserved with a client base open and willing to paying fair rates for solutions, most dreaming software developers still peg their dreams on building the next Slashdot or tech aggregator or AJAX component suite or SQL formatter.

IMG_4108

Software developers wage war over their saturated marketplace, building marvels of technical excellence in hopes of getting a piece of the table scraps, all while ignoring the much larger market (one in which people are actually paying $0.99 a day for a service to SMS jokes to their cell phone. In the tech world you can expect a fight to the death if a site demanded a $5/year subscription fee). I marvel seeing many of the simplistic, technically-deficient solutions yielding huge returns in other domains -- earlier today I had to deal with a widely used "business group membership" product, and I was astounded to see the enormous fees that went along with a subpar, circa-1995 era solution.

A week or so I mentioned a desire to build a Firefox extension to perform site-specific time tracking and blocking. I also commented that "obviously extensions aren't a potential revenue market". Not ever project has to be a revenue project (there are greater rewards than monetary rewards in many quests), however not everyone agreed with my assessment.

[as an aside, a user comment noted that there is a recently released extension called TimeTracker, which displays your daily total browsing time. While that product is extremely basic right now, it does discourage me from continuing on my extension quest -- I don't want to interfere or seem to "rip off" the obvious growth path of TimeTracker]

My reasons for believing that the extension market isn't a viable revenue market are largely documented above. The only current revenue model for extensions, I believe, is by serving a master other than the user (e.g. Adware/spyware, or as a branch of a much greater service such as Google and their Google toolbar).

Monday, February 27 2006

Many users have a small number of passwords -- often only one -- that they use everywhere. For their corporate domain account, their blog, their photo site, their email, their banking and PayPal accounts, and their discussion groups, one key opens them all. Despite the incredible risks involved with this practice, it is more prevalent than ever.

Password reuse is often the habit of jaded users who've been bitten by the "lost password" bug a few too many times, especially against sites that they seldom visit (or even frequently visited sites in cases where they've relied upon their browser or password utility to remember all of the variations...a hard drive crash or system migration leaves them helplessly flailing about, unable to access dozens of sites).

It's so much easier to remember one password than it is to remember dozens.

Of course, few will actually admit to recycling passwords like this, and instead it's the exceptions using unique 20 character random sequences that are most likely to speak out. Yet impromptu prodding of acquaintances, clients, and contacts, along with the results of several recent security surveys, has me convinced that these security best-practice aficionados are the exception, and a large number of users, perhaps even a majority, are dangerously reusing the same password prolifically.

If someone discovers your password on site A, there's a very good probability they can use it to access site B, and C, and D, and E, and so on. The security of your account relies upon the faith of a lot of people who you shouldn't have faith in, not to mention that it depends upon the weakest link before it all potentially unravels.

Despite all of the safeguards that I've put in place in the architecture and design of 360notes, I've tried to minimize the potential damage if an exploit ever did happen by eliminating any unnecessary information where possible (reducing the surface attack area). Following this philosophy, not only do I not want to store your password -- of course I only store a hash and not your original password, which should be a universal practice even among "low value" sites -- but I never want your probably-reused password ever hitting the site in the first place.

Instead of sending your password to be hashed on the server, I want it hashed on the client end, before it even gets sent down the wire.

This wouldn't be a possibility without JavaScript, however the functionality makes JavaScript pretty much mandatory, so in this case it's reasonable to require it for even the basic functionality of the site. As such, the account creation and logon system incorporates functionality that hashes a combination of your username, password, and the domain on the client end, passing through the hash to the server as your "password". As a secondary benefit, the server can generate single-use variants (salts of sorts) which it provides with the form. If such a variant is provided, after the client script has created the hash, it then hashes the first hash with the variant, which the server can do as well, offering basic line encryption as well presuming that the server is tracking the variants, and ensuring they are server provided and not reused (it doesn't replace SSL, so there is still avenues for man-in-the-middle attacks and untrusted remote servers masquerading as the official site, however it's a step in the right direction where SSL can't or won't be used).

The SHA1 algorithm is well known, and in this case I decided to go with the excellent SHA1 implementation by Paul Johnston. Implementing it was trivial, and a simple example demonstrating how to use it for this purpose follows.

<script language="JavaScript" src="sha1.js" type="text/javascript"></script>
...
<form name="loginForm" id="loginForm">
<input id="passwordHash" type="hidden" value="">
Email Address: <input id="emailAddress" type="text" size="20"><br/>
Password: <input id="password" type="password" size="20"><br/>
<input type=button onclick="DoLogon();" value="Logon" />
<input type=hidden id="variant" value="" />
</form>
<script language="JavaScript">

function DoLogon()
{
  var domain = "
www.360notes.com";
  var username = document.getElementById("emailAddress").value;
  var password = document.getElementById("password").value;

  var hashString = domain + "|" + username + "|" + password;
  var hash = hex_sha1(hashString);
 
  /* Variant - trivial "encryption" if the server has provided
     a tracked, single use pseudo-salt. */
  var variant = document.getElementById("variant").value;
  if (variant.length > 0)
  {
    hash = hex_sha1(hash + "|" + variant);
  }

  document.getElementById("passwordHash").value = hash;
 
  /* Remove the password element from the form before submitting */
  document.getElementById("loginForm").removeChild(document.getElementById("password"));
   
  /* Submit the form. */
  document.loginForm.submit();
}

Voila. Now I never know that you use the password 4muppet8 on every site, and instead I only ever see a unique hash specific for this domain.

Of course this scheme still suffers a critical weakness: If, somehow, a nefarious agent could replace the server side scripts, and somehow my remote server validation scripts failed, they could simply alter it to pass through the original password. While that scenario is far more remote and unlikely than the already remote and unlikely database delving or line monitoring, it does demonstrate why the optimal situation would be intrinsic browser support: Instead of creating a site-specific custom script to secure and individualize the password for a specific domain, which allows users to reuse passwords without actually giving the password to any specific site, the browser should support this functionality directly, and it should be evident in the UI.

In addition to the password input type, there should be a secure password type (with obvious, non-spoofable graphical indicators that it is a secure pasword box) as a basic HTML element, automatically incorporating this sort of enhancement. HTTP already supports digest autentication, which is similar, but unfortunately it is incompatible with the form logon approach commonly used, not to mention that it has its own failings.

Thursday, February 23 2006

I've always wanted to mention Quake 2 on here, so here goes.

One of the big advantages .NET brought to desktop applications, at least in regards to official Microsoft dogma, was XCopy deployments: Instead of long, convoluted setups installing dozens of components into the system, shared libraries into system folders, and registry settings in the registry (and maybe some win.ini settings for real historical fun), XCopy allows you to build applications that can be "installed" by simply copying a folder.

Everything old is new again. Years back this was the DOS way, and in many cases it's still the UNIX way.

Some time back my favourite game on the Windows platform -- I did, and though the time is extraordinarily rare still do, play "shooter" games -- was Quake2, in particular with the ActionQuake mod. This game lived in my computing ecosystem for quite a while, not only because it was great fun, but also because the application existed as a directory "island" of sorts: John Carmack and crew had disregarded the Windows developer guidelines, ignoring centralized libraries, components, and registry settings, and stored settings in little config files in the application directory, with mods, extensions and libraries appeared in that directory or subdirectories.

I could "install" the application on a new machine, including all of my settings and extensions, just by copying the Quake2 directory to the new PC. Similarly, if I installed a new harddrive I could move it there and it was fully functional immediately. No complaints about missing libraries, or ridiculous dependencies upon fixed drive letters or fixed paths.

It just worked, and happily adapted to wherever it found itself.

This seemingly trivial "feature" made this application live on my hard drive far longer than it would have otherwise. Without the hassle of reinstalling the app everytime I upgraded machines or reinstalled the OS --  a nuisance that led to many apps getting left behind -- which would also have meant reinstalling all of the patches and mods, and then laborious reconfiguring the settings to a close proximity of my tastes, it just migrated with me. It was always there waiting to provide a quick diversion during a time of thought.

With XCopy deployment, Microsoft has shown that they've seen the light, and have realized that the whole "tentacles throughout the system" approach has been a terrible mistake.

With IIS 7 we might finally see the same sort of benefit for web applications. As it is IIS is a bit of a mixed-up configuration mess, with many directory-specific settings being stored in the IIS metabase existing somewhere else (on Windows 2003 it's a convenient, well-documented XML file that you can find at %sysdir%\\inetsrv\\metabase.xml) -- mime types, directory and file security, where virtual directories/web apps start, cache settings, etc. With IIS you can't simply XCopy the app and have full configuration, but instead you have to have appropriate permissions (generally administratrive) to open the IIS administrative console and set, for instance, that your CSS files should be cacheable for a day but the frequently rewritten XML file shouldn't be cached all, which tells IIS to add the appropriate HTTP headers to each.

Compare this to Apache, which lets you configure the vast majority of directory and file specific settings via .htaccess files in each directory, saving the "system wide" httpd.conf for settings that are truly web server wide. Configuration is logical and single point, and an application can be migrated with virtually all related setup with tremendous ease. A remote developer with access to only his folder of the web app has the ability to configure things as they should be configured without ridiculously requiring administrative rights.

IIS 7 adds this sort of functionality, moving a lot of the virtual directory and folder/file settings into files that you put in your web app file structure (obviously it won't, or rather won't under ideal conditions, allow these files to be read by web users). No more hydra-setup where half the setup exists over there and half exists over here.

Everything old is new again.

Earlier EntriesLater Entries

Dennis Forbes - Dennis Forbes is a Toronto-based software architect and technology writer