Dennis Forbes on Pragmatic Software Development
Subscribe to RSS
 
Saturday, June 10 2006

While it's evident that Microsoft is staffed with a lot of top-notch people, history has empirically demonstrated that they have quite a few dregs as well: Just recall how disastrously the whole .NET thing was handled circa-2000.

For those who forgot, suddenly every product (including those finished or on the verge of being finished) became a part of the .NET vision, even if they had absolutely no interaction with the .NET technology stack: Windows Server.NET, Exchange.NET, Messenger.NET, SQL Server -- all a part of the .NET generation -- just as Microsoft declared everything in the generation before a part of the DNA vision (I still hear developers talking about "Microsoft DNA", not really sure what they're talking about).

As a developer who was heavily involved with the betas of what we call .NET today - a runtime and a framework, and the associated tools, for building next generation solutions - I really had no idea what .NET was in Microsoft parlance. Just as ActiveX got muddled into a meaningless term, .NET was being hijacked to basically mean "buy whatever is new or coming out soon".

Eventually that insanity stopped, and .NET collapsed down to a sortof virtual-machine runtime, a framework, and a set of tools. .NET 1.0 was one runtime, one framework, and Visual Studio.NET 2002. .NET 1.1 was a new runtime, a new framework, and Visual Studio.NET 2003. .NET 2.0 was a new runtime, an expanded framework, and Visual Studio 2005 (note the dropping of .NET on the naming, given that Visual Studio, as always, also makes non-.NET applications). There are countless assemblies and extension libraries available targeting each of them, and of course I can make libraries tomorrow that target .NET 1.0, .NET 1.1, or .NET 2.0, and it doesn't magically evolve them into .NET 3.0.

Well it looks like Microsoft is at it again. They've decided that Vista's technology platform, WinFX (which will be partially backported), is so great that it can't be just a set of assemblies or systems that the .NET runtime interacts with. No, it must be .NET 3.0! So now if you have the .NET 2.0 runtime, the .NET 2.0 Framework, targeting it with Visual Studio 2005, and you add in the WinFX framework...voila, you have .NET 3.0.

Insanity. Absolute, unbelievable insanity. Perhaps there's some amazing explanation -- for instance that their April Fools project ran a little long, and they just got the output out -- but I suspect it is just more of the same that we saw circa-2000. Some short-term euphoria over a gonna-be-released-soon project has them screwing with the terminology yet again.

Already the boards are full of "So....does this mean WinFX comes with LINQ?" (LINQ is one of the technologies promised for the next real wave of .NET)

Thursday, June 08 2006

[The static location of this piece can be found at this address]

FxCop As a Code Quality Tool

For the past while I've been using Visual Studio Team Edition for Software Developers, one of its benefits over the Professional Edition being the inclusion of static code analysis functionality right in the IDE.

This functionality comes via the FxCop codeset, which is an excellent -- albeit unpolished -- freely available tool for analyzing the probable code quality of Intermediate Language assemblies, testing code to ensure compliance with naming standards, best practices, and highlighting areas of code that are suspect. While it's less than pleasant starting FxCop analysis from scratch on long existing project -- to be met with hundreds upon hundreds of error messages -- it's a painless process if you add it to your quality checks early on.

The standalone FxCop is largely the same as the VSTE version, and in some ways is superior. For instance that it retains the ability to actually pass configuration settings to rules, rather than accepting whatever the defaults for the rule are.

Cyclomatic Complexity

One of the few differences between the standalone application and the VSTE-included version are the addition of several new maintenance checks in the Team Edition code, one of the most useful being the cyclomatic complexity checks. Cyclomatic complexity, for those who haven't come across it before, is often used to roughly gauge the complexity of a piece of code, to determine likely candidates for refactoring, and to identify what will likely become a maintenance problem in the future. Finding the most complex pieces of code often brings you to the buggiest code as well.

Given that I still use FxCop, both the .NET 1.1 and .NET 2.0 versions (not least because the integrated version offers no ability to configure settings for rules, instead only allowing you to wholesale enable or disable. This eliminates the ability to set thresholds for tests such as the cyclomatic complexity rules), the lack of consistency between the two versions was an annoying gap.

Introducing Cyclomatic Complexity Analysis For FxCop

So I implemented a simple cyclomatic counting rule for the standalone FxCop. While in there, I added checks for statement count (the number of intermediate language "statements", which can be indicative of overly complex methods), and callout count (e.g. callouts to other methods, again which can be an indicator of overly complex/convoluted methods).

As one added benefit, I added the ability to log all of these metrics to an SQL-capable OleDB destination (e.g. SQL Server, Access, etc). If you configured an OLEDB connection string, as detailed below, you can do data analysis after a run to create pretty reports of the complexity distributions of your projects, and so on. 

Download Links

yafla FxCop Rules for .NET 1.1 (e.g. FxCop 1.32)
yafla FxCop Rules for .NET 2.0 (e.g. FxCop 1.35)

Caveats

Like any tool of this type, there is only a moderate correlation between the metrics measured and actual code quality or maintainability: It is entirely possible that the optimal implementation is a highly-complex, lengthy method. This tool only provides guidance, helping to determine which code should get a complexity analysis, however from there experience and good judgement have to be applied to determine if it's really a fault. If you're using the .NET 2.0 version of FxCop, make use of the SuppressMessage attribute on methods that are necessarily highly complex.

Instructions

Drop yaflaRules.dll in your FxCop Rules subdirectory (e.g. C:\\program files\\Microsoft FxCop 1.32\\Rules).

If you want more advanced settings, configure FxCop with your targets and selected rules and then save the project file. Open the newly created .FxCop file in an editor (for instance notepad) and find the <Settings /> element. Expand it to an opening and closing tag (e.g. <Settings></Settings>), and between it add

<Rule TypeName="MethodComplexity"></Rule>

Between the Rule element add any of the following entries as Name attributes of an Entry element (as exampled following) -

Connection String - an OleDb connection string determining where it will log metrics. e.g. Provider=SQLNCLI;Server=(local);Database=Analysis;Trusted_Connection=yes;
Target Table - The target table for metric logging. Default - MethodComplexity
Cyclomatic Critical Error - Level at which a critical error is triggered. Default - 60
Cyclomatic Error - Level at which an error is triggered. Default - 50
Cyclomatic Critical Warning - Level at which a critical warning is triggered. Default - 45
Cyclomatic Warning - Level at which a warning is triggered. Default - 40
Cyclomatic Information - Level at which an infromation event is triggered. Default - 20
Cyclomatic Recommended - Recommended level. Default - 20
Statements Critical Error - Statement count at which a critical error is triggered. Default - 500
Statements Error - Statement count at which an error is triggered. Default - 350
Statements Critical Warning - Statement count at which a critical warning is triggered. Default - 250
Statements Warning - Statement count at which a warning is triggered. Default - 200
Statements Information - Statement count at which an information event is triggered. Default - 150
Statements Recommended - Recommended maximum statement count per method. Default - 100
Callouts Critical Error - Callout count at which a critical error is triggered. Default - 100
Callouts Error - Callout count at which an error is triggered. Default - 75
Callouts Critical Warning - Callout count at which a critical warning is triggered. Default - 50
Callouts Warning - Callout count at which a warning is triggered. Default - 40
Callouts Information - Callout count at which an information event is triggered. Default - 30
Callouts Recommended - Recommended maximum callout count per method. Default - 30

For instance, you might end up with a <Settings> element that looks like the following:

<Settings><Rule TypeName="MethodComplexity"><Entry Name="Connection String">Provider=SQLNCLI;Server=(local);Database=Analysis;Trusted_Connection=yes;</Entry><Entry Name="Callouts Warning">100</Entry><Entry Name="Cyclomatic Critical Warning">500</Entry></Rule></Settings>

If you opt to take advantage of metrics logging, the destination table (which will be default will be MethodComplexity, unless overridden with the Target Table name entry) requires the following columns:

ContainingType - text (e.g. nvarchar(255))
MethodName - text (e.g. nvarchar(255))
Cyclomatic - int
Statements - int
Callouts - int

e.g.
CREATE TABLE [dbo].[MethodComplexity](
 [ContainingType] [nvarchar](255) COLLATE SQL_Latin1_General_CP1_CI_AS NOT NULL,
 [MethodName] [nvarchar](255) COLLATE SQL_Latin1_General_CP1_CI_AS NOT NULL,
 [Cyclomatic] [int] NOT NULL,
 [Statements] [int] NOT NULL,
 [Callouts] [int] NOT NULL
) ON [PRIMARY]


Hopefully someone finds this interesting. It scratched my itch.

Sunday, May 28 2006

I've been playing with Team Foundation Server, Whidbey (Visual Studio 2005), and Yukon (SQL Server 2005) since early in the beta cycles. All three of them are remarkable products, with enormous advances over their predecessors (in the case of TFS, I'm spuriously considering Visual SourceSafe the predecessor, although TFS is a elephant compared to the mouse of VSS), and all of them should be critical components for anyone developing in the Microsoft camp.

All three of them also happen to be a little unpolished, with odd little quirks and errata, hilariously incomplete documentation, and a tendency towards resource hoggishness.

One thing I've found remarkable, however, given that the three of them have been in final form for anywhere from two months to over half a year, is how little real information and first-hand accounts are available online. I'm continually hitting roadblocks where there are marginal functions or incomplete documentation, and it's surprizing to find zero references to the same problems or questions on any of the normal forums (e.g. Google Groups, online searches, etc). Among the development community, outside of the desperate-to-get-anointed-free-support-MVP crowd, they just don't have the aura of excitement they probably deserve.

Given that there are literally millions of developers and technology hobbyists out there, it's usually the case that any problems one faces are well trodden, and a quick search on the newsgroup usually yields exactly the answer one needs, so this dearth of time-travel support really is disconcerting.

The only conclusion I can draw is that there simply aren't that many developers seriously using these technologies. Visual Studio 2005 is of course seeing some use, but there are still huge armies of developers sticking with 2003 (given the break between .NET 1.1 and 2.0). A lot of SQL shops are still taking a wait-and-see approach with 2005. Team Foundation Server, primarily because of the cost of the Team editions, and the cost of a TFS Server license if you grow past a 5-user team, seems to be fairly rare.

Thursday, March 16 2006

Before I go about possibly reinventing the wheel, I thought it worthwhile to ask: Could anyone point me to .NET / Windows server modules for SXIP 2.0 and/or OpenID? They're both fairly trivial identity solutions, so if I can't find one I'll implement one or both. Not only for personal needs, but because I can see some uses for them in client projects.

Thank you kindly.

Tuesday, March 07 2006

ASP.NET has improved dramatically with v2.0, to the point of making ASP.NET v1.x look like a bit of a hack job. One of the great improvements covered in this entry is the addition of Master Pages.

Master Pages allows you to define a template layer (and coupled back-end code) to be used on content pages using that master page. For instance a master page might define all linked scripts, CSS, and script blocks, along with a navigation header and footer that exist on all pages on the site (or at least those pages using the master page). An ugly example sits a subdirectory away -- at the root pages for yafla, where the navigation header and footer exist in a master page.

ex. MasterPage.Master

<%@ Master Language="C#" AutoEventWireup="true" CodeFile="MasterPage.master.cs" Inherits="MasterPage" %>

<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN"
"http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
<html xmlns="http://www.w3.org/1999/xhtml" >
<head runat="server">
    <title>My Master Page - This Title Will Be Overridden
       By the Title Element
In Content Pages</title>
    <link rel=stylesheet type="text/css" href="stylesheet_in_all_pages.css" />
</head>
<body>
    <h3>This is my universal header!</h3>
    <form id="form1" runat="server">
    <div>
        <asp:contentplaceholder id="ContentPlaceHolder1" runat="server">
        </asp:contentplaceholder>
    </div>
    </form>
    <h3>&copy; 2015 Robot Inc.</h3>
</body>
</html>

You can also define code for the various events in the master page, which will run on the pertinent content pages.

Content pages then define what will fill the content block (or multiple content blocks as the case may be), and of course implement their own back-end code.

<%@ Page Language="C#" MasterPageFile="~/MasterPage.master"
AutoEventWireup="true" CodeFile="Default2.aspx.cs" Inherits="Default2"
Title="This is my overridden title" %>
<asp:Content ID="Content1" ContentPlaceHolderID="ContentPlaceHolder1" Runat="Server">
This is my content for this content page
</asp:Content>

Very trivial.

Of course this isn't the only way that this result could be achieved -- I could derive from a page object that imperatively creates all of the common elements, or I could use multiple user controls that defined the basics, but neither of those solutions, or similar workarounds, seem as elegant as master pages to me. There are equal or superior solutions in other platforms, however I'm sticking to the topic of ASP.NET in this entry so they are irrelevant.

The one hiccup I faced in the use of master pages was my desire to have meta keywords (which exist in the header) vary by page, despite the fact that the meta keyword is basically dead. I want the keywords to vary, similar to the way I can declaratively override the title in content pages. Unfortunately this required some code workarounds, which in my case included adding a public property on the master page, MetaKeywords, with a default keyword list, which I then added to the header in the PreRender stage of the master page (the following example is simplified for demonstrations sake, however a real implementation would scan the headers to ensure that the pertinent header doesn't already exist before adding it).

    public string MetaKeywords = "default keywords";

    protected void Page_PreRender(object sender, EventArgs e)
    { 
      SetMetaValues(this.Page.Header, "keywords", MetaKeywords);
    }

    public static void SetMetaValues(System.Web.UI.HtmlControls.HtmlHead head, string name, string content)
    {
      HtmlMeta metaValue = null;

      metaValue = new HtmlMeta();
      metaValue.Attributes.Add("name", name);
      metaValue.Attributes.Add("content", content);
      head.Controls.Add(metaValue);

      return true;
    }

Any content page could access its Master property to set the property, and the meta keywords would be appropriately set when the page was rendered. By using the MasterType directive the Master property of the page automatically resolves to the proper type.

Unfortunate that a declarative mechanism wasn't added for arbitrary header elements in the content pages.

The goal of master pages, of course, is to avoid the scourge of copy/paste coding: Unnecessarily having a single line of code in multiple places is an evil in software development, yet it's often the easy, thoughtless solution, yielding volumes of redundant code that invariably diverges and causes maintenance problems for years to come, reducing the quality and agility of the codebase.

I despise copy/pasted code. It truly is a peeve of mine.

When analyzing the quality of code bases, one of the first checks I usually perform is to use one of the automated code duplication checkers (available for most languages). There is a remarkable correlation between code duplication rates and code quality.

The benefit of master pages isn't limited to a single master template, however, but instead you can actually layer multiple master pages. For instance on the yafla site the services category pages use the Services master page, adding additional service specific back-end code and layout, while it uses the web site wide master page. It mirrors the templated way in which many websites are developed.

The downside of layered master pages is that the GUI team apparently didn't have time to build multiple level parsing into the web designer -- wherever you're working on content pages that have more than one level of master pages above them, you are limited to the source view. To attempt otherwise yields a "Design view does not support creating or editing nested master pages. To create or edit nested master pages, use the Source view." Unfortunate, but not deadly.

As an aside, one of the big improvements with ASP.NET v2 is better support of per-page development, similar to classic ASP and competitors such as PHP. This solves one of the primary problems many had with ASP.NET, which is that they didn't prefer to work within the "web site as a monolithic application" model that ASP.NET v1 pretty much enforced. Strangely the improvements bringing these benefits has been met with little fanfare, and few are even aware of it. I do plan on doing a feature on it shortly.

Tagged: [], [], []

  .NET 
Friday, March 03 2006

I've received a couple of fantastic comments about troubles that people have faced adding items from here to their del.icio.us bookmarks, namely because Radio Userland uses a constant title for all entries (and del.icio.us automatically uses the title, so three different entries get the same title if you fail to manually override its choice). The common title problem was one of the reasons I created the notables static listing, though of course that listing is just a subsection of entries.

To help with this issue, I've added quicklinks below each entry to add it to your del.icio.us bookmarks, furl bookmarks, to Digg it or to Reddit it (which will link to an existing entry if one is already on there), and to check for Technorati links (there are seldom Technorati links because most of the readers here aren't bloggers, or they aren't the sort of bloggers that comment on every site they visit. I'd get a big boost in the Technorati rankings if I started pandering to the incestuous blogging community). I've mirrored these items to the static section as well.

  .NET   Blogging   IT   Software Development   SQL 
Monday, February 27 2006

Many users have a small number of passwords -- often only one -- that they use everywhere. For their corporate domain account, their blog, their photo site, their email, their banking and PayPal accounts, and their discussion groups, one key opens them all. Despite the incredible risks involved with this practice, it is more prevalent than ever.

Password reuse is often the habit of jaded users who've been bitten by the "lost password" bug a few too many times, especially against sites that they seldom visit (or even frequently visited sites in cases where they've relied upon their browser or password utility to remember all of the variations...a hard drive crash or system migration leaves them helplessly flailing about, unable to access dozens of sites).

It's so much easier to remember one password than it is to remember dozens.

Of course, few will actually admit to recycling passwords like this, and instead it's the exceptions using unique 20 character random sequences that are most likely to speak out. Yet impromptu prodding of acquaintances, clients, and contacts, along with the results of several recent security surveys, has me convinced that these security best-practice aficionados are the exception, and a large number of users, perhaps even a majority, are dangerously reusing the same password prolifically.

If someone discovers your password on site A, there's a very good probability they can use it to access site B, and C, and D, and E, and so on. The security of your account relies upon the faith of a lot of people who you shouldn't have faith in, not to mention that it depends upon the weakest link before it all potentially unravels.

Despite all of the safeguards that I've put in place in the architecture and design of 360notes, I've tried to minimize the potential damage if an exploit ever did happen by eliminating any unnecessary information where possible (reducing the surface attack area). Following this philosophy, not only do I not want to store your password -- of course I only store a hash and not your original password, which should be a universal practice even among "low value" sites -- but I never want your probably-reused password ever hitting the site in the first place.

Instead of sending your password to be hashed on the server, I want it hashed on the client end, before it even gets sent down the wire.

This wouldn't be a possibility without JavaScript, however the functionality makes JavaScript pretty much mandatory, so in this case it's reasonable to require it for even the basic functionality of the site. As such, the account creation and logon system incorporates functionality that hashes a combination of your username, password, and the domain on the client end, passing through the hash to the server as your "password". As a secondary benefit, the server can generate single-use variants (salts of sorts) which it provides with the form. If such a variant is provided, after the client script has created the hash, it then hashes the first hash with the variant, which the server can do as well, offering basic line encryption as well presuming that the server is tracking the variants, and ensuring they are server provided and not reused (it doesn't replace SSL, so there is still avenues for man-in-the-middle attacks and untrusted remote servers masquerading as the official site, however it's a step in the right direction where SSL can't or won't be used).

The SHA1 algorithm is well known, and in this case I decided to go with the excellent SHA1 implementation by Paul Johnston. Implementing it was trivial, and a simple example demonstrating how to use it for this purpose follows.

<script language="JavaScript" src="sha1.js" type="text/javascript"></script>
...
<form name="loginForm" id="loginForm">
<input id="passwordHash" type="hidden" value="">
Email Address: <input id="emailAddress" type="text" size="20"><br/>
Password: <input id="password" type="password" size="20"><br/>
<input type=button onclick="DoLogon();" value="Logon" />
<input type=hidden id="variant" value="" />
</form>
<script language="JavaScript">

function DoLogon()
{
  var domain = "
www.360notes.com";
  var username = document.getElementById("emailAddress").value;
  var password = document.getElementById("password").value;

  var hashString = domain + "|" + username + "|" + password;
  var hash = hex_sha1(hashString);
 
  /* Variant - trivial "encryption" if the server has provided
     a tracked, single use pseudo-salt. */
  var variant = document.getElementById("variant").value;
  if (variant.length > 0)
  {
    hash = hex_sha1(hash + "|" + variant);
  }

  document.getElementById("passwordHash").value = hash;
 
  /* Remove the password element from the form before submitting */
  document.getElementById("loginForm").removeChild(document.getElementById("password"));
   
  /* Submit the form. */
  document.loginForm.submit();
}

Voila. Now I never know that you use the password 4muppet8 on every site, and instead I only ever see a unique hash specific for this domain.

Of course this scheme still suffers a critical weakness: If, somehow, a nefarious agent could replace the server side scripts, and somehow my remote server validation scripts failed, they could simply alter it to pass through the original password. While that scenario is far more remote and unlikely than the already remote and unlikely database delving or line monitoring, it does demonstrate why the optimal situation would be intrinsic browser support: Instead of creating a site-specific custom script to secure and individualize the password for a specific domain, which allows users to reuse passwords without actually giving the password to any specific site, the browser should support this functionality directly, and it should be evident in the UI.

In addition to the password input type, there should be a secure password type (with obvious, non-spoofable graphical indicators that it is a secure pasword box) as a basic HTML element, automatically incorporating this sort of enhancement. HTTP already supports digest autentication, which is similar, but unfortunately it is incompatible with the form logon approach commonly used, not to mention that it has its own failings.

Earlier EntriesLater Entries

Dennis Forbes - Dennis Forbes is a Toronto-based software architect and technology writer