Dennis Forbes on Software and Technology   Subscribe to RSS


About the Author
Dennis Forbes Dennis Forbes is a Toronto-based software architect. While focused primarily on the .NET and SQL Server worlds, Dennis frequently ventures outside of this comfort zone into game development and image processing. He has been published in several industry magazines, has been quoted in the Wall Street Journal and has been interviewed by NPR.

He is a vice president and lead software architect at an innovative New York City hedge fund back-office services firm.

Dennis has been working on solutions for the financial, telecommunications, and power generation markets for over 15 years.




The Feed Bag
Feb 24 - TED

 
Wednesday, December 06 2006

While I signed up for the original Monad public beta, I never really gave it more than a cursory look: It seemed really incomplete and unpolished. It also irked me somewhat that Microsoft couldn't just embrace one of the existing scripting languages -- say python or perl -- even if they were invented elsewhere, but had to go and invent something new (although I've grudgingly come to appreciate their reasoning).

In mid-October it came out in release form, having been renamed PowerShell.

I've finally got a chance to try to incorporate it, and thus far it looks very nice. While this isn't something that you're going to build a product upon, good automation scripts are instrumental in good development practices, eliminating inefficiency, and the morale-suckage that goes along with repetitive, manual tasks, but more importantly eliminating the inevitable error when people are given such tasks.

It's well worth a close look for automation scripts on the Windows platform. Certainly beats .bat.

Reading through the documentation, however, the following gave me a good chuckle.

One major advantage of using objects is that it makes it much easier to pipeline command, that is, to pass the output of one command to another command as input...

...Windows PowerShell provides a new interactive model that is based on objects, rather than text...

...In the following example, the result of an IpConfig command is passed to a Findstr command. The pipeline operator (|) sends the result of the command on its left to the command on its right. In Microsoft® Windows® PowerShell, you don't need to manipulate strings or calculate data offsets...

PS> ipconfig | findstr "Address"

Ah...good stuff.

Of course in that case the vaunted example isn't using objects (well I suppose the result is a set of string objects, but the example makes no use of them, and completely fails to convince as to the difference), it's just using the stdout of ipconfig, and that command of course works in a classic cmd.exe (or even command.com) session.

Of course some people are surprized that Windows has any command line functionality. In the examples for PureJPEG

Friday, December 15 2006

I spent much of Wednesday in Pearson Airport's decrepit (but soon to be demolished in the staggeringly expensive upgrade of the entire airport) Terminal 2, wondering what the deal was with our flight that was supposed to be boarding.

The help desk sat vacant -- as it had for hours -- and the nearest information board was literally about 1300 feet down a walkway towards the terminal hub. I'd made this walk several times (for epic adventures such as the "going to buy a bottle of water, because the change machine down here is broken and I didn't dare try bringing change through airport security because I have enough junk to worry about already" journey) and it was really starting to get old.

There was approximately one (1) dual-outlet power gangplate in the entire facility -- which I managed to win from a gaggle of nervous-eyed crackberry users, all of us desperately fighting for the ability to siphon some precious electrons (I like to call it "invisible oil") into our drained batteries.

Yet despite this seeming hostility towards the accoutrements of modern life, the facility is surprizingly equipped with ubiquitous, free wireless.

The wireless allowed me to make a secure VPN connection elsewhere (I wouldn't trust anything over an unencrypted channel on public wireless), actually making use of the time to get some work done. It really is empowering checking code out of and into Team Foundation Services, doing database changes, and then rolling out deployments, all from some random chair in some random airport.

It also allowed me to visit Air Canada's website to check on the status of the flight.

Whoops, maybe not. Looks like there was some sort of java error on their middleware (which I know because they strangely feed the entire error stack to end users, rather than having a more professionally refined default error page). This stayed this way for as long as I bothered to continue checking, and made the website completely useless for this task when I actually needed it.

Coming back on Thursday, a late night had me crunched for time, so I wanted to check the schedule for New Jersey Transit's train to Newark Airport from New York's Penn Station. Just needed to know when I needed to catch one -- I'd never been on it before, and had no idea what the frequency was -- and how long it'd take.

No dice.

The online train schedule information system was down. Just had to hoof it and hope. Luckily there was one in the station, ready to go. Maybe there's always one queued, but I wouldn't know it from their information system.

Today I wanted to order a gift from the Future Shop's website. After a detailed error message, it then flipped to a message proclaiming that they were doing "routine maintenance".

Then there are the amateurish downtimes that have occurred on some of the large meme sites, when moves or upgrades that should be seamless end up causing hours of outages. I'm a huge fan of Flickr, but even their recent moves were far more disruptive than they should have been (though they certainly have much more of a justification -- namely petabytes of pictures on a top tier website -- than some jokers running a "lists of links" type website).

These are hardly isolated incidents, and I'm not trying to pick on particular organizations. It just happens to be the most recent frustrating demonstration that the web isn't where it should be, and far too many teams consider reliability to be much less of a concern than it rightly should be.

I've been doing software for long enough to know that few systems are foolproof, and that sometimes eventualities conspire against the best laid plans of the most considerate, skillful world class teams, but these sorts of should-be-exception situations are happening with increasing frequency.

Despite all of the improvements in computer science, and the advances in the platforms that we're developing against, the net direction seems to be downwards, with reliability apparently coming after all else.

This isn't a good trend.

QuickNotes

Office 2007 is quite nice, though from my experience it's a little bit unstable. Between Word crashing while attempting to use the blog editing functionality, and Excel crashing at the oddest of times (the same experience being had on two completely different make and model of machines, with very different software stacks and few commonalities), it seems to be a fairly regular occurence. Finally the blog functionality in Word just stopped working altogether. On the bright side the document recovery is extremely capable, and I've never lost even a keystroke of typing, as it seems to be keeping a realtime backup going.

Saturday, December 16 2006

A bit of a ruckus has arisen over the purported breach of security at the groupthink site Reddit.

It seems that the Reddit folks were storing the user's passwords in plaintext, so a recent data loss or integrity compromise of some sort has them warning users to change their password just in case their backup tape -- if that story is right -- gets in the hand of the desperate-for-high-karma-Reddit-accounts drug cartels.

Many are calling this a blatant mistake on the part of the Reddit crew, declaring that password's should never be stored in plaintext. The Reddit crew and defenders have stated that the plaintext passwords are used to allow them to email the password to the user, which is a tenuous argument but I suppose they went for the KISS model (which is pretty much the modus operendi of Reddit. They recently rolled out a CAPTCHA implementation that is laughably vulnerable out of the gate, but it is the simplest implementation possible).

What is most disturbing to me, however, are the declarations that this is much more of a problem than Reddit alone. People are crying foul because they believe that their bank accounts, email accounts, and other online accounts are vulnerable now that the Reddit user database might be in the wild.

NEVER USE THE SAME PASSWORD ON MULTIPLE SITES.

At worst share passwords among throw-away type sites like Reddit. Never share passwords between sites that actually matter.

Let's say that Reddit actually did hash the password -- debatable if it's necessary for that site, and I have advocated advanced techniques for doing this before -- why in the world would you trust the folks at Reddit with this secret (all the hashing in the world does nothing if the people who are doing the hashing have nefarious motives)? Why would you trust the people who man their data centers, or the people who share machines with them or handle their backup tapes or provide their internet services?

There is no credible reason why a shared password in the hands of Reddit alone -- even if they cross-their-heart promised to hash it --should give comfort to someone who reuse the same password on sites of value. That is absolute insanity, and it is a very dangerous practice.

It's far more disturbing to me that people worry about more than their Reddit account in this situation.

If you must "reuse" passwords, use one of the many utilities available to hash your name or email address with the target site domain on the client side, (for that particular one -- note that it's just one of hundreds available -- you can use their website, Firefox or IE 6 or 7 extensions) actually generating a unique password for each site while only having to remember one password on your end. There are many clever implementations, but the one linked here, for instance, allows you to preface passwords with @@ and it automatically does client-side, site-specific hashing, meaning that your shared secret isn't dangerously shared with the people at random internet sites.

Monday, December 18 2006

I frequently go scrounging around for niche development tools/libraries to solve one-off needs, and it's frightening how ubiquitous fake, 3D-rendered product box images are in the software tool industry.

These rectangular parallelepiped (aka "box") renderings generally feature some gangly, oversized text, coupled with awkward, gaudy graphics.

Further investigation has revealed that there's a whole industry of "3d box rendering" software vendors, all promising to pump up your sales if you put one of these travesties on your product page.

What's the point of these fake product box renderings? Why do so many ISVs insist upon leading their product page with them?

"It makes our product more real! Like something physical that you could hold in your hands."

No it doesn't. It makes it less real.

Seeing a clearly bogus box that no one would ever spend actual money printing doesn't fool anyone. Worse, it gives off the putrid stench of deception, implying that there's a real shelf-clogging box when many of these products provide nothing of the sort, most being e-delivery or at most a CD shipping in a flat pack.

It begins a relationship of mistrust.

"But Microsoft does it! Microsoft=successful, therefore it must be a good practice"

Apart from the weak (and admittedly strawman) rationalization, not only has Microsoft mostly abandoned the "picture of the box" graphics on their product pages, where they did use it they used the professionally produced graphics they printed on the real box -- the real box that you probably also saw at Circuit City or Future Shop or wherever.

The product page graphic made reference to real life. It didn't spin some fiction.

We as an industry need to stop this image on cuboid violence. If you must decorate a product page, it's less of a crime to use some of the standard "beautiful people looking happy" pictures, though even that should be avoided.

Thursday, December 21 2006
  1. [INDEX OUT OF BOUNDS EXCEPTION]
  2. [INDEX OUT OF BOUNDS EXCEPTION]
  3. [INDEX OUT OF BOUNDS EXCEPTION]
  4. [INDEX OUT OF BOUNDS EXCEPTION]
  5. [INDEX OUT OF BOUNDS EXCEPTION]
  6. [INDEX OUT OF BOUNDS EXCEPTION]
  7. [INDEX OUT OF BOUNDS EXCEPTION]
  8. [INDEX OUT OF BOUNDS EXCEPTION]
  9. [INDEX OUT OF BOUNDS EXCEPTION]
  10. [INDEX OUT OF BOUNDS EXCEPTION]
Saturday, December 23 2006

Did you know that your PC can be immediately available while consuming less than 1/25th the power? Read on!

(If your PC already is configured to go to the proper S3 sleep, look below for some power consumption numbers for the various levels)

Coinciding with the recent, widely-linked WSJ article on power usage around the home, I happened to be anxiously awaiting my own power meter's arrival in the mail. In my case I wanted it for a different purpose (a complete article on that topic will be published shortly. For now I just wanted to author this post -- on the topic of S1-S4 power modes -- so I could reference it from the other post, avoiding long parenthetical sidetracks like this), however in doing research for the other article, some rather surprizing analysis was completed that I thought worthwhile to share.

My primary development desktop PC is rather obsolete: Athlon XP 3200+ on an nforce2 motherboard, 1GB, two hard drive (totaling 200GB), network card, nvidia 6600GT 256MB AGP video card affair. From an energy usage perspective, it's very comparable to most current PCs.

I've been meaning to replace it with a shiny new Core 2 Duo for some time, but the hassle of setting up all of my software again, configuring everything the way I like it, and so on, is a huge disincentive. Add the fact that my current PC never leaves me really begging for more power, so there's not enough drive to upgrade.

On the bright side, while I've procrastinated about upgrading, the state of mainstream computing has continued to march forward, so what I'm upgrading improves with every passing quarter.

I use my laptop quite a bit, but the machine described above is really the workhorse, and I generally leave it on throughout the day. I leave it on partly because I sometimes need to remotely access it, but more so because I tend to have the need to jump onto it to do short tasks throughout the day. Booting up on a need basis isn't acceptable as even the fast boots of today are still too slow, not to mention having to start all of the various tools that I use.

I used to rely upon S4 (hibernation), but even there the time to restore is too time consuming for these quick-hit usages. Add to that the occasional failure of hibernation to actual recover from a hibernation, locking up when the load is complete, forcing me to dump the state and reboot from scratch -- makes me very wary of the feature as a whole, though I realize it's more than likely an issue with a specific driver or piece of hardware that I'm using.

So instead lately I've been resorting to standby -- auto initiated at a preset interval of non-usage, and manually triggered when I know I'm leaving the PC for a bit. I presumed this was a substantial power savings.

With delivered power meter in hand, I finally had a chance to prove it one way or other. First I measured the "base load", when the PC is sitting on and ready for use (note that actually doing something causes this number of spike, sometimes substantially, so I'm intentionally indicating the idle power usage). This measurement is only the mid-tower consumption, and does not include monitors, speakers, etc.

Idle Power Usage: 129 Watts.

Wow. 129W just sitting there idling. That's about $10 of electricity a month. An ignorable cost, but when you've had conservation beaten into you it really makes me feel quite guilty thinking that my PC was consuming upwards of 100kWh a month largely doing nothing. What a waste.

Next I measured standby mode. In this mode the machine takes just a second or two to recover, ready for use (which I do by either sending it a Wake-On-Lan packet from remote, or by clicking the mouse/keyboard).

Standby (S1) Power Usage: 112 Watts.

Still 112W? Admittedly I was quite shocked that a non-functioning, non-computing PC, with hard drives powered down, could consume this much power. Sure this was a fluke; I did the same test on several of PCs and found the same marginal power savings in standby (S1) mode. This finding was entirely contrary to many sources that claim much greater efficiency of S1 mode.

All of the above was in Windows 2003 (and is the same behaviour on several XP machines I tested). I rebooted the same machine into Vista, and was shocked to find the standby mode consuming far less power (not to mention that it was audibly different -- it actually shut down the fans and such).

Back in Windows 2003, I started investigating why the disparity exists, and why I'm not seeing the power savings I should.

It turns out that XP, and thus 2003, identifies the power support on first installation -- determining which of the S# modes your machine supports -- S1 is the weakest power mode, S3 is a "suspend to RAM" mode (where power is cut to basically everything but the RAM modules, retaining system state with very little power), and S4 is hibernation, where the state is saved to the hard drive. Not only will it not automatically accommodate later changes (for instance BIOS changes), but apparently it very frequently defaults to only configuring standby to S1, even where the machine fully supports more. Furthermore, some configurations of USB devices will cause it to revert to the hoggish S1 standby mode.

My search brought me to a little Microsoft utility called dumppo.exe. With this, I could imperatively force the operating system to start using S3 sleep (given that my PC already supported suspend to RAM. Every new motherboard supports it, although some default to it being disabled in the BIOS). After running dumppo admin minsleep=S3 and rebooted, I put the machine into sleep and checked my power meter.

Sleep (S3) Power Usage: 5 Watts.

(I've scaled all of the power consumption values relatively, and the above says "5 Watts")

Not only is the machine using the same power in S3 sleep as it uses when it's completely off -- yes it does use 5W powering the network card and other motherboard systems even when it's "off" -- but the availability is impressive, with it coming completely to life instantly, and thus far reliably.

Again I use Wake-On-LAN to tell the machine to come alive remotely, making it immediately accessible from afar, and it's far more available than hibernation. Win/Win/Win!

A little utility and suddenly this PC is using 1/20th the power than the prior standby mode. Given the average usage cycle, this will drop monthly energy consumption for this PC from 100kWh to approximately 35kWh a month. Imagine the conservation an entire office building would achieve (consider that many IT departments unnecessarily mandate that all PCs are on 24 hours a day to allow for automated off-hours patch deployments).

Not everyone will have this particular problem (with S3 not being recognized by the operating system), but some quick checks of PCs under my control and among friends and family has demonstrated this to be a ridiculously common scenario, so I thought it worth an entry.

Wednesday, December 27 2006

I've been interested in two-factor authentication on the cheap as a technique to improve systems security for some time. My interest is primarily in making this technology available for marginal cost, with limited scale-out fees. This is a task that should be easy given the widespread availability of powerful mobile computation devices -- namely cellphones with J2ME -- and the ease of adding secondary authentication to most platforms (e.g. GINA authentication in pre-Vista, and Credential Providers in Vista and similar options on other environments. Adding two-factor authentication to a web application is a breeze).

Two-factor authentication, for those who haven't come across it before, is the addition of a secondary identity "proof" above and beyond the normal one-factor password. The purpose being to limit risk when user passwords are surreptitiously gained by nefarious agents, which could happen due to keyboard sniffers (a simple USB interceptor at the back of the target's PC could log weeks worth of keystrokes), trojans, use of public or compromised computers, password reuse or selection weakness, and so on.

Passwords alone are a really weak technique for system security.

The most well-known implementation of two-factor authentication are RSA SecurID hardware tokens -- or similes --  usually built as little keychain units that display a frequently changing, time-based access number (unique per token, so the central system needs to know which key went to which user). To log onto secured systems, such as banks and corporate VPNs, the user needs to enter not only their normal password, but also the code displayed at that moment on their token. So user sally logs on with her password 5gromet4u and her authenticator token is displaying the value 10492838, so she enters that in the pertinent login box. On the server the authentication service validates her password, and that the token that she was assigned should be what was provided (on the server side it's usually generous, allowing the authenticator token for a few minutes in the past and future to accommodate minor time skews, or Sally being slow entering the values).

The cost of these simple devices is usually absurdly high, however. Worse, the service software is usually pompous, overbearing, overpriced systems that take an absurdly simple need and make it a colossal pain to deploy and manage. This is irritating because the core technology behind such systems is absurdly simple -- hash the current date (truncated to a certain level of precision -- say every minute) together with a private value on the hardware token and convert to a BASE10 value to a certain number of digits (e.g. the first 8 digits of the BASE10 encoded hash). On the server end it knows the private value associated with each hardware token, and can perform the same process, so if the time is synchronized on both ends the hash values match.

e.g. here's my 3 minute C# version of a minute-granularity token generating method.

private string GetTokenValue(string userSpecificPrivateKey, int tokenLength)
{
    System.Security.Cryptography.SHA1Managed hash = new System.Security.Cryptography.SHA1Managed();
    byte[] hashValue = hash.ComputeHash((new UTF8Encoding()).GetBytes(System.DateTime.UtcNow.ToString("yyyyMMddHHmm") + userSpecificPrivateKey));
    string hashString = String.Empty;
    for (int counter = 0; counter < (tokenLength>>1) && counter < hashValue.Length; counter++)
    {
        hashString += ((hashValue[counter] / 256.0) * 100.0).ToString("00");
    }
    return hashString.Substring(0,tokenLength);
}

So if I've been assigned the secret, user-specific value of AF8CAD55JK9 (a value that was securely communicated and configured, or burned directly, into the token device and of course configured on the server, but then never spoken aloud or communicated again), then at 1:00 pm UTC on December 28th, 2006, the method will return the token 5864947577 for a token length of 10 digits.

Cell Phone

So here we're all walking around with incredibly powerful cellphones featuring massive amounts of memory and computation power -- I still marvel to think that my cell phone has double the memory of my pimped-out 4MB Atari ST circa 1998 (which itself was 8x the stock memory), and features many, many magnitudes more computational power. At the time that Atari ST seemed infinitely capable -- and most cell phones promise tremendous flexibility and ease of expansion with the universal runtime of J2ME.

Surely to make such a token application for our cell phones should be enormously trivial. I've built GINA modules before, so that element is relatively trivial as well, and is only getting easier with Vista/Longhorn (Dear Microsoft - Where's my free Ferrari 5000 laptop for this Vista mention?)

So over the holidays I wanted a bit of a change from the norm, so I grabbed the NetBeans IDE and the mobility pack and then the pertinent Motorola SDK (all while deciphering a ridiculous array of often conflicting acronyms, and the version soup that is the Java world). All in all it was a remarkably simple and easy route to developing mobile applications, though I was a bit perplexed that I even needed a device specific SDK (I thought J2ME was sort of universal, beyond the basics like screen size/color support), meaning that widespread deployment would probably be far more difficult, with hundreds of possible builds being necessary.

Implementing the above into a J2ME application, displaying the current token value (changing every minute), along with some simple configuration options, was absurdly easy, despite Java not being a skill of mine. The slick little emulator worked wonderfully and displayed what the application would actually look like if it were running on my class of phone.

All in all the free J2ME development ecosystem has improved significantly since my last (abandoned) effort in this space, with some very slick tools and technologies.

Considering the enormous size of the market, it isn't surprizing.

Then I tried to deploy it to my cell phone. What a nightmare: Despite bluetooth and USB connectivity -- theoretically -- it was a complete no go.

The cell phone industry is a travesty. I went through this exact same exercise years ago, again giving up in disgust at all of the barriers that the cell phone industry in cahoots with the wireless providers put in one's way -- basically they do their damndest to limit the flexibility of the devices they provide (even though I bought this phone, and it is legally mine...not really sure how a provider has any right to put any constraints or locks on it. Of course I have the same thing with the Motorola PVR that I bought from my cable provider, again with various features and functions disabled by the cable company), trying to ensure that you're forced to do everything through their cost bloated network, buying from their grossly overpriced online stores, and application deployment has to happen through a system that ensures that they get a tremendous take even if their participation is entirely unnecessary.

To get this application on my phone I'd have to resort to a stack of basically warez-like software (from shady sources, spoken of in hushed voices) to circumvent all of the ridiculous constraints.

Once again I'm giving up at making the phone I carry around more useful. Maybe I'll look at one of those Windows Mobile phones (come on Microsoft! Where's my Acer Ferrari 5000!), which presumably offer more user control and flexibility.

Earlier EntriesLater Entries

Dennis Forbes