Dennis Forbes on Pragmatic Software Development
Subscribe to RSS
 
Wednesday, March 07 2007

Internet Usage Self-Assessment

How many times do you load sites like Slashdot, Netscape, Digg, and others during the day? Do you find all of the links colored a:visited? Do you wish they'd age-out quicker so you'd have more sites to visit and discuss, and to forward to peers and friends?

How many blogs do you read? How many RSS feeds do you subscribe to?

How much time do you spend browsing through Wikipedia entries?

How many futile online debates do you participate in (Microsoft versus Linux. Liberal versus Conservative. Pro- versus Anti-. Debates where nothing will ever be resolved, and no positions will ever be changed. No, seriously, no matter how convincing you think you are, and how enlightened the forum is, it is extraordinarily unlikely that you're ever going to change a single mind, much less the world).

How many facile blog entries are you wasting your time authoring, tickling the reader's ego by telling them that they surely must be among the cream of the crop reading such an enlightened entry (which should offend readers who can detect the obvious pandering, yet it's a tactic that is becoming common), desperately hoping that you'll get atop some momentary meme list, your 5-minutes of fame yielding nothing of real Earthly consequence?

How much time are you spending on all of the above in an average day?

Take a moment and answer honestly. Feel free to answer in the comments if you'd like, though this isn't meant to be a competition.

For many people the answer might be "very little or none at all" , but for quite a few I suspect the answer will lie somewhere between "a hella' lot" and "all day".

Squandered Opportunity

What are you gaining from that sunk time? Entertainment? Intellectual stimulation? An enlightened perspective? Work-related information (that's was my personal justification back when Slashdot was much more technology focused, and the discussions actually did expose me to new information about the industry and technologies, though at the time the turnover of stories was so low that a single short visit a day was sufficient)? Are you gathering some nebulous site-specific karma that's more of a badge of excessive free time, or on some sites a perspective that most correlates with the group think, than wizardly knowledge?

Most of the time the net benefit of a day spent on the meme sites is nothing more than entertainment, but few would admit to themselves that it was akin to watching an all-day episode of Survivor or Heroes.

What could you have achieved in that time?

Could you have spent that 3 hours building a Reddit clone (to waste someone else's time)? Launching your eTailoring venture? Learning Japanese? Learning Python? Taking up a new language? Implemented that awesome new piece of functionality? Gone to the gym and started the journey to fitness? Cooked an amazing, exotic meal?

Talking to and observing peers in the industry, I've noticed that this time suckage has become a widespread, massive squandering of opportunities, with so many talented developers, designers, and intellects basically wasting their days away. From a personal perspective, I know that occasionally I find myself "quickly" checking to see if there is anything on the feed sites before delving into a difficult problem, and some time later realizing that I'd completely burned away all of the personal project time that I had set aside.

It's for this reason that I'd previously talked about router blocks and Firefox rationing.

Honestly monitor where your time is going, and if you're being the best you that you can be.

Aside: Many will wonder if the "facile blog entries" bit was self-referential. These blog entries once served a commercial purpose -- yes, they were entirely selfish, as most human efforts are -- but as the associated ventures turned out to not need the publicity, and I no longer was looking for cold consulting clients, that overt selfish motivation evaporated. I've never really been motivated by appearing on the meme sites, as a quick look at the "peers" in the space tells me that it isn't really a great accomplishment to be on the front page alongside a "PICUTRE OF KITTEN [AMAZING!]". Now I mostly do this as a brain exercise, keeping entries very short, infrequent, and quick to compose, saving my time for my attempts to Try To Take Over The World!

Wednesday, March 07 2007

We've been using Microsoft's Team Foundation Server for version control and basic work item/requirements/bug tracking for about 9 months now. All in all it's a good tool, though really it still feels like a version 0.8 beta that got pushed out the door a little early.

Lining Up For The Landing

The Good

  • It's a world better than Visual SourceSafe
  • Superb integration with Visual Studio
  • Performance is great locally, but also over a low-speed pipe
  • Security is well-defined and adequate
  • Branching and merging works well
  • Multiple files are checked in as a changeset, rather than individually
  • The requirements/work items/bugs functionality is half-decent, albeit very unpolished and feature-poor

The Bad

  • Some basic functionalities, such as rollback, are missing (which is why I say that it seems like a pre-release beta in ways). There is a powertool hack that lets you rollback a changeset, in which case it checks out all of the affected files, does a get of the pre-checked out files, and then checks in the old file(s), leaving the "rolled back" version in the history.
  • burlington The application tier is incredibly fragile. If an enterprising team member decides to do some clean-up directly in the TfsVersionControl data-tier database (getting around missing functionality in the tools -- for instance there is no way to permanently delete, aka destroy, in the tools, remarkably, so if someone checks in a 500MB file and you want to remove it, you're forced to do it directly in the database), you will discover that a single missing related record -- the database doesn't define or enforce foreign-keys, so it isn't going to block the DELETE command there -- will cause the application tier to die a hundred deaths, excepting out on null values and other inanities. This is made far worse by the fact that the application tier caches a lot of lookup data, so check-ins/outs will work for a while after these related records were moved, making a simple database rollback impossible. Instead you need to go through every database manually rationalizing all of the data, determining where the application tier is dying.

    The worst part is that many of the unfound lookups should be complete non-events, either not displaying the unrelated records, or putting in placeholders while indicating that an error occurred. The idea that it takes down the entire version control system is completely unreasonable, and it really made me question the implementation.
  • Offline support is non-existent in the front-end tools, with it expecting a constant connection to the source control web services. There is a dubious powertool that manually works by removing the project from source control, and upon reconnection you tell it to reattach and then do a sync.

The Ugly

  • The application tier won't install on a domain controller, or on a 64-bit system, both limitation being entirely unreasonable and hindering for small shops.

All in all it's pretty decent, but I think they called it done a little early.

Tuesday, March 06 2007

For the stand-alone application developer targeting the Windows platform -- that increasingly endangered species --  when the need arose for a database backend, the default choice was historically the Microsoft Jet database engine, usually to interact with an mdb (e.g. Microsoft Access) database file (though Jet supported other, less-used options as well).

This goes all the way back to the terribly awkward DAO dll libraries callable from C, though interaction greatly improved through the years, to the ease and simplicity of today's ADO.NET.

When deployment time came, you could distribute with the freely redistributable Jet runtime (the only significant limitation being that you couldn't create a direct MS Access competitor, with table designers and the like), and while the client had no need for a Microsoft Access installation or license, if they did have it they could interact with the database directly where the need arose, or for extended functionality (e.g. reporting via Access). These mdb databases could also be accessed through other tools such as SQL Server linked servers, etc.

I write that in past-tense, as Microsoft has been beating Jet to death as of late.

The first punch was the complete lack of a 64-bit migration path -- existing or planned -- meaning that in a 64-bit instance of SQL Server you can't add linked Access databases, nor can you interact with Jet-supported databases from the 64-bit runtime of SSIS (though thankfully in that case you have the fallback of using the 32-bit runtime). And while Access 2007 runs atop a heavily modified version of Jet called ACE, the new library itself isn't redistributable being intended only for use from Access.

So what is the replacement? The primary replacement was the former MSDE (Microsoft Database Engine), called SQL Server Express in the latest iteration. Microsoft really wanted to push developers to the SQL Server platform from the smallest need to the largest need.

With a liberal, free redeployment, a very easy upgrade path to a "real" instance of SQL Server (and the easy integration and interoperation with other instances of SQL Server, taking part in functions such as replication), this is a compelling choice but for the fact that it is a resource-intensive overkill for many simple needs, with a multi-hundred megabyte install, a separate service which itself is far from slim (it is an actual instance of the same SQL Server database product that might be hosting the corporate HR databases, with only a couple of minor hardcoded limits differentiating it), and then the potential administration headaches in the future (the Slammer worm primarily infected MSDE installations that many users weren't even aware they had running. SQL Server's default configuration is far more intelligent now, such that by default it only listens on localhost, and it supports robust attaching/detaching of databases, but there still is an excessive surface-area for attack if only a basic database was desired).

If all you want is some basic table structures with simple indexing, the bulk of the database management system meant for large-scale corporate data warehouses just isn't reasonable.

Microsoft has another option now, albeit growing from a product path that has been around for a while, called SQL Server CE aka Compact Edition (confusingly you'll find it called Mobile Edition on many of the supporting documents, as that was a prior product name). Supporting a subset of T-SQL, and basic tables (no views, stored procedures, or triggers), it's an in-process, very lightweight option if you want a simple backend database in your application, and want it to easily interoperate with some other Microsoft technologies.

It's primarily unmanaged, but provides excellent .NET interoperability.

It isn't SQL Server, though. The code doesn't come from SQL Server. T-SQL is limited to a subset, the database format is completely different, and basic functionality like full-text search is missing. From an integration perspective, while it is manageable from SQL Server Management studio, the only real consistency is in the object explorer and the query analyzer, while index configuration and schema design occurs in completely new modules.

In its implementation some questionable decisions were made, such as the lack of non-unicode text types (there are a significant number of very legitimate uses for ASCII text. Going UCS-2 makes it more likely that surrogate keys get used where natural keys could have been the better choice at one-half the size, and is just a waste of space and performance if it isn't actually necessary).

So why am I talking about SQL Server CE at all?

The compelling feature is that it runs on the gamut of Windows targets, including mobile editions (e.g. smart cellphones and PDAs), including tools to sync between the devices. If you wanted to make a Getting Things Done task-tracking style app atop a reasonably robust, feature-rich-enough database, and you wanted it to be usable from a desktop and a PDA whether connected or not, it presents a very interesting option.

I'm going to play around with it a bit in the coming weeks when I need a distraction from more critical work, and this is just the intro piece to further analysis of this product. I may look at the unfortunately named VistaDB and other embedded, in-process options (I'd love to consider the embedded PostgreSQL -- itself a fabulous RDBMS -- however resources for it are few and far between).

Thursday, February 22 2007

In the pursuit of home improvement I've had to remove and reinstall several toilets over the past year, sometimes redoing the same toilet more than once due to mistakes made on the first outing: all of the preparation in the world sometimes isn't enough if you haven't practically experienced a task before.

I've gotten to enjoy the removal and installation of wax rings, the wonderful varieties of flanges, and the unfortunate results of using too thin of a wax ring when the newly installed flooring raise the toilet's height just a little too far.

Sure I could pay someone to do this, but sometimes I enjoy, err, getting my hands dirty and learning these things myself.

The thing I marvel at is how terribly hackish the interface between toilet and sewage system really is -- you have to align a very heavy toilet perfectly over two held-up-only-by-good-intentions bolts, often back-breakingly leaning over in very tight confines while setting it down just perfectly on the bolts; bolts that are barely contained by a flange that is the end piece of some fixed plumbing, meaning that if you overtighten then you're breaking a pipe that's a massive pain to repair.

It's close to impossible to gauge the difference between not tight enough and too tight with these sorts of things, but it's a costly mistake if you go too far one way or the other (too tight and you have a busted flange or toilet base, but if it's too loose you could have movement that breaks the seal, which causes floor warpage, which breaks the seal more, which causes more warpage, which pushes the unit up breaking the flange): Sort of like the oil change technicians who stripped the oil plug and didn't bother telling me until it popped off, creating an environmental disaster in my driveway, or the auto mechanics that over-cranked the spark plug, stripping the aluminum head.

Of course the toilet interface demonstratably works, given that there are billions of non-leaking, non-broken facilities in operation right now, but technically it is, in my opinion, a complete hackjob of an interface.

In many ways it's like HTML and browsers that will happily adapt to many transgressions of the already loose spec.

Our world has endless numbers of these "there has to be a better way" designs, but because of low expectations, the inertia of the status quo, lots of workarounds, and the training and experience of those who have to deal with it, it's fine.

I bring this up because software is endlessly compared to other disciplines, usually with the software industry cast in a poor light. 

"Why can't we build software like we build bridges?", we endlessly hear.

People say this because they have no idea how hacked out (and often over-engineered and costly -- have you looked at the cost of building a little road over a gap these days?) the business of building bridges, or airplanes, or toilets, often is.

It isn't as perfectly engineered as they probably imagine.

Sidenote - How in the world does an entirely ambiguous word like "biweekly" or "bimonthly" continue to be used? Biweekly is sometimes used to indicate every other week, other times used to indicate twice a week. Bimonthly has the same confusion, either meaning every two months, or twice a month.

Wednesday, February 21 2007

For years I've argued against ridiculous software patents: While I'm a fervent believer of the innovation fostering environment provided by IP protection and rights, the stream of trivial software patents has reached a torrential pace. We're now at a point where it's impossible to create any software solution or website without infringing upon the trivial "IP" of hundreds of patent holders, leading to the unintended consequence that innovation is suppressed because of the natural litigation risk presented by patent trolls.

One of the more recent abusers of the patent system is none other than Microsoft. They're even patenting methods they're taking from other products now (sure, Microsoft's patent wouldn't hold up during the course of a real lawsuit, but few opponents would have the resources to even bring it that far).

Such a comment inevitably summons the Microsoft-does-no-wrong defenders who repeatedly declare that Microsoft is different, and that these were defensive patents. Microsoft, we are told, is just building a patent shield in case a patent troll comes gunning for them (ignore the absurd foundation of the whole "defensive patent" argument -- patent trolls seldom have any interest in cross-licensing, and they seldom have published products that a target can counter-assault with dubious patents).

Maybe not.

When Microsoft's growth curve seemed limitless, they overlooked casual piracy (which is how they gained dominance), had limited invasive copy protection mechanisms, and played nicely in the software development community. As their hegemony has faced real competition, some of it coming from the open source world (given that capitalist competitors were easy to squash), and their growth has been stagnant and has the potential of reversing course in the coming years, Microsoft has changed for the worse. Now they're enlisting jackbooted squads of anti-piracy teams, and infesting their products with activation and "genuine advantage" bugs.

Now they're starting to threaten competitors, and worse the customers of competitors, with their patent trolls.

The risk presented by Microsoft's so-called "defensive" patents is exactly what is happening. They've been offensive patents all along.

Expect it to only get worse.

Wednesday, February 21 2007

Over the past 24 hours I've logged several hundred inbound visitors vectored over from http://www.dzone.com (I swear that I seldom check the stats -- honest to goodness cross my heart -- but I just happened to right after getting up this morning. It caught my eye as it wasn't one of the usual suspects). Fairly interesting programming-specific Digg/Reddit-like site, with a useful collection of links.

Just thought I'd mention it for those readers looking for something different. This isn't a reciprocity link, but rather that I truly found it a decent site with different content than the regulars.

Sidenote: One thing that I've always found fascinating about sites like that is the incredibly small percentage of users who actually "vote". I've been Digg'd, Slashdotted, Reddit'd, and now dzone'd, and I'd say that less than 5% of users who follow the links actually declare their feelings (in the case of Digg I would put the percentage even lower). While some of the sites don't allow for a negative vote, Reddit does but even there less than 1 in 10 link navigator expresses their feeling one way or the other.

Wednesday, February 21 2007

I recently finished the book Dreaming in Code (which I got for sort-of free, excluding income tax, courtesy of affiliate links just like that -- I'd forgotten that I even had any of those on here, but apparently some ancient article on SQL Server Hierarchies has a couple), grabbing a copy after it started appearing in all of the "customers who bought this book also bought"* sections for other books I was contemplating.

Wonderful book.

It documents the progress (or lack thereof) and missteps that occurred during the lengthy birthing of Chandler, an open source Outlook/Exchange "Killer".

Started with a lengthy, largely abstract wish list, and an ability to use whatever technologies and techniques that they wanted -- financed by the deep pockets of the undemanding Mitch Kapor -- Chandler was the sort of project that developers dreamily contemplate while analyzing the patterns on their cube wall, imagining how much better it would be than the tight confines and demanding deliverables of their current reality.

Chandler was the sort of project where the only apparent limitation seemed to be the collective imaginations of the contributors. The sky was the limit!

The result of such free reign and lack of demands can only be classified as a disaster: After a long period of development by a fairly large and expensive team, the releases thus far have been a rather dubious calendar product, in a very alpha state, and a server product that backtracks on many of the initial goals (for instance the "P2P solves everything" silver bullet).

It's difficult to categorize this book: It really isn't a how-to book at all. Nor is it really a book for a layman, as it frequently goes into a level of detail that could be daunting (sometimes the level of technical detail seems a bit like filler, not really playing a part in the telling of the story. Unnecessary detail for the story, but too little detail to be of interest for a technical analysis).

Really it's a how-not-to book: A reporting of many of the classic mistakes of software development happening in a real, visible project, and how they impacted the progress. The OSAF (Open Source Applications Foundation) team certainly isn't the first to make these mistakes, and they certainly won't be the last, but they had an uncommon honesty and openness to allow a reporter to document their every step.

I would imagine Mitch figured this would all turn out quite a bit differently, with Chandler rocketing to Firefox-like success, and this book documenting the new age approach to software creation.

This book came to mind yesterday when I hit publish on the "Most Destructive Metric In Software Development" entry. In that entry I mentioned that developers abhor using other people's code when they have the option of spinning something "original" themselves, and this materializes several times during the Chandler project. After eliminating the Zope ZODB (a Python object database) from consideration, for instance, and going off and going through several iterations of their own database, the individual behind the decision declared that they "never really looked at ZODB".

Classic.

* - Around four years ago we bought the great book "The Baby Name Wizard", and found it to be a helpful in choosing names for our first child. We're now a month away from welcoming our third child, and have begun the name contemplation game, and I just received a "People who bought the Baby Name Wizard also like..." email from Amazon. Their system has more inputs than I imagined.

Earlier EntriesLater Entries

Dennis Forbes - Dennis Forbes is a Toronto-based software architect and technology writer