I've always wanted to mention Quake 2 on here, so here goes.

One of the big advantages .NET brought to desktop applications, at least in regards to official Microsoft dogma, was XCopy deployments: Instead of long, convoluted setups installing dozens of components into the system, shared libraries into system folders, and registry settings in the registry (and maybe some win.ini settings for real historical fun), XCopy allows you to build applications that can be "installed" by simply copying a folder.

Everything old is new again. Years back this was the DOS way, and in many cases it's still the UNIX way.

Some time back my favourite game on the Windows platform -- I did, and though the time is extraordinarily rare still do, play "shooter" games -- was Quake2, in particular with the ActionQuake mod. This game lived in my computing ecosystem for quite a while, not only because it was great fun, but also because the application existed as a directory "island" of sorts: John Carmack and crew had disregarded the Windows developer guidelines, ignoring centralized libraries, components, and registry settings, and stored settings in little config files in the application directory, with mods, extensions and libraries appeared in that directory or subdirectories.

I could "install" the application on a new machine, including all of my settings and extensions, just by copying the Quake2 directory to the new PC. Similarly, if I installed a new harddrive I could move it there and it was fully functional immediately. No complaints about missing libraries, or ridiculous dependencies upon fixed drive letters or fixed paths.

It just worked, and happily adapted to wherever it found itself.

This seemingly trivial "feature" made this application live on my hard drive far longer than it would have otherwise. Without the hassle of reinstalling the app everytime I upgraded machines or reinstalled the OS --  a nuisance that led to many apps getting left behind -- which would also have meant reinstalling all of the patches and mods, and then laborious reconfiguring the settings to a close proximity of my tastes, it just migrated with me. It was always there waiting to provide a quick diversion during a time of thought.

With XCopy deployment, Microsoft has shown that they've seen the light, and have realized that the whole "tentacles throughout the system" approach has been a terrible mistake.

With IIS 7 we might finally see the same sort of benefit for web applications. As it is IIS is a bit of a mixed-up configuration mess, with many directory-specific settings being stored in the IIS metabase existing somewhere else (on Windows 2003 it's a convenient, well-documented XML file that you can find at %sysdir%\\inetsrv\\metabase.xml) -- mime types, directory and file security, where virtual directories/web apps start, cache settings, etc. With IIS you can't simply XCopy the app and have full configuration, but instead you have to have appropriate permissions (generally administratrive) to open the IIS administrative console and set, for instance, that your CSS files should be cacheable for a day but the frequently rewritten XML file shouldn't be cached all, which tells IIS to add the appropriate HTTP headers to each.

Compare this to Apache, which lets you configure the vast majority of directory and file specific settings via .htaccess files in each directory, saving the "system wide" httpd.conf for settings that are truly web server wide. Configuration is logical and single point, and an application can be migrated with virtually all related setup with tremendous ease. A remote developer with access to only his folder of the web app has the ability to configure things as they should be configured without ridiculously requiring administrative rights.

IIS 7 adds this sort of functionality, moving a lot of the virtual directory and folder/file settings into files that you put in your web app file structure (obviously it won't, or rather won't under ideal conditions, allow these files to be read by web users). No more hydra-setup where half the setup exists over there and half exists over here.

Everything old is new again.

I've received some great feedback regarding the entry on setting up a MediaWiki install on Windows. Many of the comments were kind words of thanks (which I really appreciate. Knowing that it helps people is my greatest motivation), and others helpfully suggested improvements to the instructions.

As an example of comment-driven improvements, my instructions have you installing the GNU diff utilities, in particular for the diff3.exe utility, however the MediaWiki setup scripts don't properly find it (e.g. as the instructions are currently written the GNU diff utilities are completely unused, although they can still be useful in your day-to-day travails). This is because a prior revision included fairly involved changes to the MediaWiki config/index.php script so it would properly locate diff3 on the Windows platform, as it is currently Unix-centric and doesn't look for the proper executable, not to mention that it parses the PATH environment variable incorrectly . After receiving two comments that those steps were a little too complex, however, I removed that section.

My goal was to get people experimenting with MediaWiki, or even just wikis in general, so diff3 functionality really wasn't critical. I pared the instructions accordingly. Similarly one early draft included the building and installation of a PHP memory cache to improve performance, but that too is unnecessary to simply try out the product.

Another line of comments involved asking:

• Why would I give instructions for Windows. People should just set it up on Linux and go with its native home.
• -or- Why would I recommend a wiki product that largely caters to the open source crowd. Instead I should be pushing Sharepoint, or something properly anointed by the Microsoft camp, enabled with all of the latest Microsoft buzzwords.

To answer this I really need to describe the philosophy of this blog, along with my resistance to "technology alliances".

In the byline of this blog I describe my philosophy as "pragmatic software development", and this really drives my recommendations. In this case there are a lot of development shops that are Windows-centric, with little or no UNIX/Linux experience, yet MediaWiki is one of the best, most featurer rich, "standard" wiki products out there. Choosing a solution that leveraged what shops already know with the best solution is a pragmatic approach.

Which brings me to my general philosophy towards Microsoft, as comments indicating that I'm either a Microsoft hater, or a Microsoft drone parroting the corporate line, have hit my inbox over the short history of this blog.

I am not subservient to Microsoft.

Unlike many Microsoft technology advocates (I truly love both SQL Server, and .NET, and I think they're remarkable solutions), I have no desire to ever work for Microsoft (Microsoft has some top notch, world-class talent, and I've met and worked with a lot of great talent from there, but they also have their share of both jerks and duds). I'm not going to praise their every move in hopes that I'll get noticed. yafla, my consulting/ISV company, has chosen to avoid any partnerships or tying to the Microsoft brand because we don't want to become another drone "consulting" company single-mindedly acting as a third-party sales force for Microsoft, desperately racking up Microsoft partner points by pushing less-than-optimal solutions on customers. We didn't choose to use .NET for our software because we're hoping to nestle into the Microsoft family -- we chose it on technical merit, and a pragmatic analysis of our current and prospective clients.

We work for our clients and ourselves, not Microsoft. This is a very important mantra for our services, and for the technology of our software, and if Microsoft wants their products to get recommended to our clients, and their technology to the foundation of our software, they need to make great products at competitive prices. No sales gladhanding, or sad career dreaming, is going to change that.

Am I saying that Microsoft solutions are second rate? Of course there are examples of Microsoft products that are terrible, and customers are being misled into buying buzzword-laden atrocities because a Microsoft partner is hoping to get invited to the next Microsoft dinner party. Yet there are also Microsoft solutions that are extraordinary. Windows 2003R2 is a superlative operating system, and where you need the breadth of its functionality, it can be well worth the money. Microsoft Small Business Server can be an amazing package of value for some small organizations, within the constraints of the product. Other times, however, if you have the appropriate skills, a Linux machine is the best choice, along with a stack of the many available free or close to free server products on that platform. Sometimes IIS 6 is the superior solution for a problem, while other times Apache would be your best bet. Sometimes PHP and MySQL is a great solution, and other times C#/ASP.NET with SQL Server is the perfect combo.

I don't blindly assume the Microsoft product to be the best, but neither do I automatically presume it to be second rate. Instead I evaluate on merit, and propose solutions based upon the customer and their needs.

To do otherwise would be just biased noise, and wouldn't be to the service of clients and peers.

Tagged: [Software Development], [Programming], [Software-Development]

Introduction

This article describes the wonder and curiosity that many developers start out with, whether it's when they entered their first Compute! type-in program on their Atari 400, picked up their first JavaScript in 1 Hour book, when they started toying with the gcc compiler for the first time, or when they began towards their first Computer Science degree in university.

It also describes how that natural enthusiasm can be crushed, and how it can hopefully be regained or maintained.

This is written for the developer, whether a new recruit or a veteran, motivated or unmotivated, spirited or crushed, yet it's also written for software development managers (who might identify how to make the workplace more enjoyable and more rewarding).

Like most entries of this genre (see also Optimal Software Development Processes and Practices) I selected a small list of widely applicable, but often overlooked, factors. This most certainly isn't exhaustive, but hopefully it leads to a bit of reflection.

The Enjoyable Profession of Software Development

Software development can be a tremendously rewarding, enjoyable career.

Few careers offer comparable opportunities to weave intricate, complex structures that, while virtual, have such a positive impact on the world around them. Few offer the freedom and creativity that software development does, or the very real potential for entrepreneurial riches.

Whether it's building a new peer-to-peer application, control software for a massive power generator, or improving the workflow of the corporate scorecard system, done right this can be a very fulfilling, enjoyable, challenging pursuit.

A Passion for Software Development

Does your mind race at all hours, abuzz with potential solutions for vexing software development challenges? Do you lie awake at night -- anxious like a preschooler on Christmas Eve -- eager for morning to arrive so you can implement the crafty coding structures you just thought up? Do you frequently find yourself powering up your system in the twilight hours to implement the fruits of an epiphany?

Or do you put in just enough face time and superficial effort that sacrifice makes up for undelivered results? Do you purge your mind of software development the moment the virtual end-of-day whistle goes off, sliding off your Aeron dinosaur satisfied that it's one day closer to the weekend? Do you dread Mondays, motivating yourself to keep going with the dream of a far off vacation?

Do you eagerly embrace new technologies, seeing it as a challenging opportunity to learn something new when a solution calls for a new skill? Would you voluntarily dive into the innards of the Firefox web browser if a solution demanded it and you'd never touched it before? Do you swim through documentation, thirstily absorbing new APIs, tools, and languages to expand your skill-set, eagerly embracing industry advances?

Or do you dread anything different, praying that you're tasked with challenges that require only the skills you've long held, allowing you to apply them in a mechanical, repetitious fashion? Do you hope every project is an echo of a prior project? Do you put off any task requiring research, and show disdain towards new languages, techniques and practices, hoping that they don't gain traction?

Are you really passionate about software development? Be honest with yourself.

A desire to outshine a teammate isn't passion. Nor is a motivation to impress the boss. Neither is a combination of the two worn as a magic defensive cloak against downsizing spells. These are second-rate, artificial passion substitutes: Mixed into the recipe, they yield sub par results, often leaving a nasty aftertaste of burnout and dissatisfaction.

Instead I'm talking about a bona fide interest and enjoyment of the craft and challenge of software development, even outside of career or job security issues (though it benefits the same). This isn't a job ad demanding that you're "passionate about business reports!", but rather is just a moment for sober reflection on whether you're over-clocking life, or running idle instructions in a tight loop.

If you're like many software developers in the industry today, a feeling of enthusiasm and enjoyment for the pursuit is just a distant memory (often during the happy days of university and your first job). Instead it has become a career, and is just something you do from 9-5 (or more when passion is replaced by sacrifice). Skills have likely stagnated, moving just enough to compete with coworkers, or to avoid obsolescence.

Of course there are those who've never enjoyed this career, and they probably will never enjoy it -- it just isn't their thing. The only advice I can offer to those people is a suggestion that life is too fleeting to spend so much time doing something you don't enjoy.

Many others, however, remember the passion, and sporadically get a fleeting taste of it again. For those people I propose some personal habits that, coupled with workplace practices (for managers, as well as people who rightfully manage up), will help recapture and maintain that passion.

Software developers who truly love what they are doing are the ones creating the most innovative code. They're the ones with productivity rates multiples of their peers. They're the ones that feel a little guilty getting paid to do something they enjoy so much.

The Top 5 Habits of Productive, Happy Software Developers

1. Be Marketable - Keep Up To Date Skills and Network Contacts

Most of us will work for over a dozen different firms over our careers.

We'll leave for better salaries and working conditions. We'll relocate to accommodate a spouse's career. We'll be laid off during corporate mergers and spin-offs, or even when the company goes bankrupt. We'll get turfed out because we're over-skilled, and thus overpaid, relative to the needs of the position. We'll be downsized because we aren't compatible with the new boss' empire building schemes. Maybe we'll get bored of a position and seek out something new.

This is the employment reality of most careers in the 21st century.

To some professionals this represents an exciting journey, and each transition is met with anticipation and enthusiasm. These people feel confident in their abilities, have a network of peers in the industry communicating interesting opportunities, and their skillset is up-to-date and marketable (they have the appropriate laundry list of abilities, credentials and certifications, and upgrade as needed), and while the possibility of their current employer closing shop tomorrow is something they'd prefer not happen, and they probably love the great group of people that they work with, it isn't something that they fear.

To less prepared professionals, however, the idea of losing their cushy job hangs over them like a black cloud. Their lack of apparent opportunities, and the feeling that they couldn't find an equivalent job, is enormously destructive of both motivation and job satisfaction. Paradoxically, job protectionism (such as making one "indispensable" through obscurity, by denigrating coworkers, and so on) often becomes a more likely activity of people in such positions than legitimate contributions.

This is incredibly destructive to morale, not just for the individual in question, but for everyone on their team: Often the malcontent, contagiously demotivated member of the team is the least employable, and it can be debated which condition led to the other.

SUMMARY:  No matter how much you love your current job, you should keep your CV current, and you should always keep up-to-date on industry opportunities. Know what skills are in demand, and try to gain experience in them (even if it means pursuing formal or self-training during your own time), and attain a level of comfort that you could transition to a different opportunity with minimal discomfort.

MANAGER SUMMARY: You should do everything in your power to make your group feel confident in their abilities -- ensure that everyone gets a chance with marketable technologies; encourage the pursuit of desirable certifications; and build skills through internal resources, workshops, and seminars. Unless you're running a sweatshop, this is unlikely to lead to a feared exodus of employees, but instead will empower and motivate your group to more openly contribute, and to demand more of each other.

2. Be The Master of Your Domain

The control we have over our environment can have a tremendous impact on our happiness. 

Something as simple as a sporadically malfunctioning key on our keyboard can ruin an entire day, for instance. Similarly, when you're nearing a deadline and your network connection starts flaking out, it can make an enjoyable jog to the finish line a frustrating exercise of physical restraint (in this case restraining yourself from tearing the wiring out of the wall). At least we have optical mice now, eliminating one of the primary causes of environmental control frustration.

Many times our work habits inevitably bring a feeling of "lack of control" into our work lives: By failing to fully read the documentation for our tools, investigating their behaviour, APIs, and nuances, we often create a situation where much of our development is basically crap-shoot trial and error, reacting as things don't work as planned.

I've witnessed development groups, not to mention that I've demonstrated this unsavoury trait myself, unhappily fighting with perceived technology deficiencies (usually as a deadline rapidly approaches), moaning and complaining about what seems to be faults in the language, technology, or platform, forever building workarounds under a fog of uncertainty, when in reality it was actually a fault in the understanding of the same. 

More often than not it's simply that they haven't spent the upfront time to understand the language (I remain amazed at the number of C# developers who have no idea what the using keyword is for, or why seemingly out-of-scope file objects are still locking files until some magical, indeterminate time in the future. Or the Delphi developers who needlessly nulled variables at the end of scope in a futile misguided attempt to fight mystery bugs), the technology, or the platform. Their frustration is created out of ignorance, and a small up-front investment would have sped up development, increasing the sense of control that the developers have over their domain.

SUMMARY: The next time something seems mysterious or unknown, take the time to properly investigate it. Classic lack-of-control approaches such as hacked workarounds or "reset the server daily" lead to a feeling of losing control, reducing job satisfaction and adding to the natural daily frustrations. And get your keyboard replaced if it starts malfunctioning.

MANAGER SUMMARY: Identify and investigate "easy-outs" proposed by your development team. While most software has faults, and products and technologies often work differently than we might imagine, many times such excuses are due to a lack of investigation and analysis. Even when things don't work as advertised, which is frequently the case, formally investigating and empirically determining behaviours is vastly superior to each developer endlessly fighting with and then hashing out strategies on a need basis. And make sure your developers have functioning keyboards.

3. Accommodate Your Financial Needs

I've worked in some great positions at the wrong times in my life, sapping my motivation until eventually I moved on. These positions were for great firms, with great working conditions and great coworkers and management, but it couldn't realistically adapt to accommodate my evolving financial needs. I invented dissatisfactions with the situation, turning an ideal situation into a daily torture.

After getting married and planning for our first child, for instance, the financial risk/reward that worked when I was living alone in a $600 apartment eating Ramen noodles was no longer satisfactory. Demands of owning a home, a car with infant carseats, education funds, daycare (for two children costing more than it would cost to lease two (2) BMW 750i's), and boxes and boxes of diapers, required more financial returns than I needed years before.

I moved on.

While the resulting role superficially wasn't as satisfactory, from a life perspective my mood brightened dramatically, and my day was much more enjoyable.

Of course this seems like cheap advice: Make more money! And Fast! Yet the reality is that developers often do make choices to the detriment of their financial condition, and if they go too far they will hate their job no matter how perfect it otherwise is. Working for equity of a start-up is great when you're just out of university, but it is destined for failure when you're more established.

SUMMARY: If your financials are out of balance, it will unavoidably sour your mood during the workday, making you resent your employer and your workplace. When life goals exceed the income of your position, immediately begin investigating alternatives (be it asking for a raise, looking for a more senior role in your organization, or seeking employment elsewhere). No motivational boost or cool company games room will overcome this basic life need.

MANAGEMENT SUMMARY: Be aware of the goals and needs of your group. Sometimes someone's needs grow beyond the possible return of a position, and it is important to appropriately communicate this (rather than giving vague hints of unseen raises and super-bonuses at some future point).

4. Have A Life Outside of Work

This is a rule that works for all professions -- having accomplishments providing satisfaction outside of work will smooth the inevitable downs of our professional lives, often providing one with a much better perspective. Without this, often minor workplace failures can explode into seemingly momentous events.

These accomplishments can even be in the same domain: A professional coder by day, and an open-source coder by night, for instance.

SUMMARY: There will be periods when everything seems to go wrong in the workplace. Having the cushion of achievements outside of work can avoid it spiraling into a workplace disaster, keeping spirits up through the tough times. Often non-work experiences benefit the workplace as well, whether it's techniques learned from nighttime projects, or delicious coffee courtesy of the nighttime barista classes.

MANAGER SUMMARY: There is a world outside of work.

5. Properly Manage Expectations

Developers, as a general rule, are terrible at managing expectations: Many of us are prone to overpromising deliverables, assuring stakeholders that we'll deliver these amazing results sooner than is reasonable. I've fallen victim to this syndrome myself, and I've seen it occur rampantly across the industry.

When D-day comes we convince ourselves into believing that the users built their own unrealistic expectations, and managers forced us into untenable timelines. While often that is the case, just as frequently the developers were the origin of misinformation. 

While there is a temporary sense of satisfaction wowing users and management with an exaggerated declaration of our abilities (we've likely even convinced ourselves), as time wears on this misinformation can be enormously destructive and debilitating. With every day closer to the deadline we get a little more desperate for a silver bullet, hoping that some magic technology or component will deliver us from damnation.

It seldom works out that way.

Users are unhappy. Management is dissatisfied. Employees are demoralized and devastated.

The best option is always to manage expectations, to ensure that we can reasonably deliver promised results without heroic effort.

SUMMARY: Plan for the long term, realizing that promises that aren't delivered on will cause you great workplace unhappiness later. Manage expectations to ensure that you can satisfy your "customers" with reasonable effort, and with a reasonably high probability of success.

MANAGER SUMMARY: Never demand unrealistic deadlines, and question employees when provided with the same. Encourage your troops to be more reasonable with their promises, especially to stakeholders outside of the group, and they'll have a much greater probability of meeting external expectations, leading to increased motivation for everyone.

Conclusion

This is an amazing, expansive career full of incredible innovation and endless opportunity. Ensure that you don't diminish your enjoyment through simple mistakes, such as pigeon-holing into a position, or endlessly setting up yourself for failure.

Control your destiny.

Tagged: [Software Development], [Programming], [Software-Development]

I've been doing this as a somewhat regularly updated blog for just over half a year now, and the results have been extremely satisfying: I get about ~2500 direct unique visitors on an average day (increasing 2-6x when something ends up being a meme-of-the-day on sites like Reddit or Digg, and of course many read via aggregators), search engine referrals are up to 200 or so a day, and viewing the "who's on" list is a laundry list of influential corporations and locations across the globe.

It does feed my ego a little bit seeing visitors from various governments, the CIA, nuclear research labs, just about every large financial company, and visitors from every end of the globe. My numbers aren't huge, but it's a perfect composite of influential and knowledgeable readers.

The most popular entries thus far are as follows (I'm providing the static version links where possible):

Effectively Integrating Into Software Development Teams
Optimal Software Development Processes and Practices
Spelling Matters
Everyone Is Above Average - The Overpopulated Top 2%

I've tried to minimize the number of entries (outside of the personal category, though this anniversary one being an exception) to keep the noise as low as possible -- if you're using a reader it won't constantly pretend there's new content when I'm just adding a peanut gallery comment about someone else's blog -- though on the flip side that means that I've delayed various .NET and SQL entries until they're "perfect". 

Perhaps I might have to find a compromise somewhere in between.

After fielding several wiki-related queries by clients and associates, along with numerous questions and comments online, it is evident that it's the Year of the Wiki. Wikipedia has proven the concept, and users have a growing awareness of the benefits that organic information growth could bring to their teams.

As such, I'm putting together a feature covering wiki options and alternatives, including specific instructions for configuring and using Wikis on Windows (as this is a particularly neglected area, and much of the information that exists is terribly out of date or quite simply non-functional). Of course yafla provides turnkey Wiki solutions and training as a service as well.

One more point: The consulting work has always overflowed purely from word-of-mouth and associate networking, so the business website has always been terrible (sort of the whole "the cobbler has the worst shoes" thing). As yafla is now entering a growth stage, wailing past the temporary manpower limit, I'm finally going to change the corporate website to properly reflect the services and capabilities of the organization, and actually allow options for prospective clients to engage our services. That should be up shortly, growing and improving rapidly over the coming weeks.

Three yafla resource "shout-outs":

yaflaColor - A dynamic web tool to select colors, including proper saturation and lightness varations of colors
pureJpeg - Remove extraneous JPEG blocks
High Performance SQL Server - Information to ensure your database designs and usage are optimal

Have a fantastic weekend!

I have been considering the possibility of yafla providing training services, developing and delivering programs here in the Greater Toronto Area (and globally where the monetary return makes it worthwhile), adding this service to the existing software development, outsource management, and consulting options. Not only is it an additional revenue stream, much more importantly it's another potential avenue for making contacts and getting involved as a vendor with new clients, creating opportunities to more easily offer our other services.

I've done a lot of group training in the corporate space, have been involved in quite a few tutorials and online training guides, and find it to be a very rewarding pursuit. Several of my associates have been heavily involved in the training industry during periods of their career. The possibility of hosting an "Advanced SQL Server" seminar or workshop, for example, seems very exciting.

We're very equipped to perform this task, and we certainly can do a better job than all of the trainers I've been exposed to. Our approach would never be to dedicate anyone to training alone (or even as a substantial period of their time), as continuing and up-to-date real-world experience would be absolutely critical to the program.

As such, I'd greatly appreciate any input anyone might have regarding the technical training industry, and how external training programs work at their organization. I know many mid- to large-sized firms have an "authorized training vendor", basically ensuring a universally weak level of training throughout their organization, and that could present a significant barrier to entry.

When Java first hit the development ecosystem, to many it wasn't just a method of doing efficient, high-level development, but rather it became a new religion: You couldn't only use Java as the glue between existing code, or even as the overwhelming bulk of your solution. A partial-Java solution simply wasn't good enough.

Instead your product had to be 100% Pure Java. The still sought-after eventual goal was a complete Java solution, from applications right down to the operating system, with only the smallest possible binary kernel, if even that. All of this would be running on a Java-aware processor, engineered specifically for Java.

Sun created a "100% Pure Java" campaign to push this philosophy, including banners and designations for appropriately certified software, and advocated it as a very desired moniker. Users were led to feel that mixed solutions were impure and somehow dirty: Are you some sort of nut running an impure solution, dirtied with some pointer munging, buffer overflow vulnerable C code? While there were (and remain) methods to call native code, they were discouraged.

Of course there is a lot of validity to this agenda. Primary being the fact that pure Java solutions are theoretically cross-platform, with no ties to external technologies. Compare this to a solution leveraging C libraries, which would require a rebuild or binary available for every distinct target platform. Additionally Java could only impose its sandbox and extensive security constraints if you stayed in the world of Java, and thus callouts to native code represented a risk.

In the real world, though, it often meant that developers were constantly solving long-conquered problems, redundantly reinventing solutions in Java that long existed elsewhere, or waiting until adequate libraries eventually appeared: Developers were pressured to use Java alone even when it was a hammer and the solution really needed a chisel.

Thankfully .NET hasn't been pushed in such a single-minded way (even if some of its champions have foolishly taken up such a misled cause, including some at Microsoft. Instead of a justified part of the solution, it becomes a religion. .NET! .NET! .NET! .NET!), and indeed Microsoft themselves has always facilitated, and even advocated, "impure" solutions. The majority of the .NET Framework, for example, is actually a very thin veneer over the existing Win32 facilities and libraries -- it was either that, or version 1.0 would have come with a much smaller, much less efficient library.

The "orchestration layer over native code" implementation is the reason .NET hasn't suffered the performance difficulties that Java has.

Microsoft chose to leverage what they'd already done, to maximize both performance, and to maximize the breadth of the library. 

This advantage isn't limited to Microsoft, though, and the developer can utilitize this functionality as well. .NET offers very simple COM and P/Invoke functionality to leverage "legacy" code (or even new code developed in a best-solution, non-.NET technology), allowing you to easily use your existing DLLs and/or COM libraries as first class partners in your .NET solutions. Even if they're created in "dirty" languages.

I take advantage of this functionality regularly, utilizing existing best-solution libraries and functions, regardless of whether they're pure .NET or not. For instance in creating the static version of the "best of" blog entries, I quickly -- maybe 2 hours -- wrote a quick transformation tool that basically imported the "best of" RSS feed (it isn't included in the normal category lists), then doing some XSL transformations (using extension objects in the XSL given that XSLT alone wasn't adequate for some special purposes -- for instance HTMLDecoding the description block of the RSS) to the resulting XHTML, as well as creating an index page.

One goal when creating this solution is that the resulting pages are all fully XHTML compliant, and they pass the W3C validity checks. While I could easily see how the pages rendered in Mozilla/Firefox/IE/Opera, and of course they all rendered fine, technically there were a couple of deviations from the spec. Some of these errors and warnings were caused by unavoidable transformation issues, while others were caused by minor mark-up errors in the original blog entries (both because of my own errors when doing it by hand, but also because of Radio Userland's "helpful" auto-"cleanup" of HTML. It is remarkable how often auto-formatting is detrimental).

HTML Tidy to the rescue.

I had several options with HTML Tidy, the easiest of which would be to ShellExecute out to the EXE, telling it to process an existing file. I could have taken more time and tried to make a managed C++ version of Tidy, but I really didn't want to spend that much time.

I decided to have a bit more fun, not to mention building a more integrated, higher performance solution, and use the Tidy dll from the micro-.NET utility. I grabbed the Tidy source code (Tortoise CVS is a great solution for this, in this case using :pserver:anonymous@cvs.sourceforge.net:/cvsroot/tidy), updated the included MSVC projects to Visual Studio 2005, and added them to the transformation utility solution. I set the Tidy dll project output to the build directory of my .NET utility (in this case $(SolutionDir)\blogStatic\bin\$(ConfigurationName)). The MSVC build worked perfectly right away, which is amazing given that Win32 isn't an officially supported build.

To reference the Tidy dll methods, of course I had to add the DLL import signatures, in this case adding only the ones I had a need for.

  [StructLayout(LayoutKind.Sequential)]
  struct TidyBuffer
  {
    public IntPtr bp;           /**< Pointer to bytes */
    public uint size;         /**< # bytes currently in use */
    public uint allocated;    /**< # bytes allocated */
    public uint next;         /**< Offset of current input position */
  };

  class FileClean
  {
    [DllImport("libtidy.dll")]
    public static extern IntPtr tidyCreate();

    [DllImport("libtidy.dll")]
    public static extern int tidyParseFile(IntPtr tidyPointer, [MarshalAs(UnmanagedType.LPStr)]string fileName);

    [DllImport("libtidy.dll")]
    public static extern int tidyParseBuffer(IntPtr tidyPointer, ref TidyBuffer tidyBuffer);

    [DllImport("libtidy.dll")]
    public static extern int tidyCleanAndRepair(IntPtr tidyPointer);

    [DllImport("libtidy.dll")]
    public static extern int tidySaveFile(IntPtr tidyPointer, [MarshalAs(UnmanagedType.LPStr)]string outFileName);

    [DllImport("libtidy.dll")]
    public static extern int tidyRelease(IntPtr tidyPointer);

    [DllImport("libtidy.dll")]
    public static extern int tidySetCharEncoding(IntPtr tidyPointer, [MarshalAs(UnmanagedType.LPStr)]string encoding);

    [DllImport("libtidy.dll")]
    public static extern int tidyOptSetBool(IntPtr tidyPointer, int value, int Bool);

    public static bool CleanFile(System.String outputfileName, System.IO.MemoryStream docDataStream)
    {

      int result = -1;

      IntPtr tidyPointer = tidyCreate();
      try
      {
        // We want the resulting file to be UTF8 encoded
        tidySetCharEncoding(tidyPointer, "utf8");

        byte[] docDataArray = docDataStream.ToArray();

        TidyBuffer tidyBuffer;
        tidyBuffer.size = (uint)docDataArray.Length;
        tidyBuffer.allocated = (uint)docDataArray.Length;
        tidyBuffer.next = 0;

        GCHandle pinHandle = GCHandle.Alloc(docDataArray, GCHandleType.Pinned);
        try
        {
          tidyBuffer.bp = Marshal.UnsafeAddrOfPinnedArrayElement(docDataArray, 0);

          if (tidyParseBuffer(tidyPointer, ref tidyBuffer) >= 0)
          {
            tidyOptSetBool(tidyPointer, 29, 1);
            tidyOptSetBool(tidyPointer, 23, 1);
            if (tidyCleanAndRepair(tidyPointer) >= 0)
            {
              result = tidySaveFile(tidyPointer, outputfileName);
            }
          }
        }
        finally
        {
          pinHandle.Free();
        }
      }
      finally
      {
        tidyRelease(tidyPointer);
      }

      return (result == 0);
    }
  }

Most of this should be self-evident, however the two tidyOptSetBool calls may be a little cryptic. For the sake of brevity I haven't used the constants, but 29 is the TidyMakeClean value of TidyOptionId enum (see tidyenum.h), and 23 is the TidyXhtmlOut value. Together these indicate that I want to clean the documenting, converting it to XHTML. Note that I've also set the encoding to UTF8.

Voila, after transforming the RSS to the memory stream as quasi-conformant HTML, I passed the stream to this function, along with the desired output filename, and out went a cleaned-up, valid XHTML document. Pedants everywhere were thwarted from pointing out minor deviances from the standard. I could have processed to another buffer, and then done follow-up processing in .NET as well, but this was sufficient.

This is a trivial example, but it really exemplifies the great value of the easy interoperation of .NET. With it I could instantly leverage existing code, without having to search out bastardized ported versions, and instead could go right to the source.