This article describes the wonder and curiosity that many developers start out with, whether it's when they entered their first Compute! type-in program on their Atari 400, picked up their first JavaScript in 1 Hour book, when they started toying with the gcc compiler for the first time, or when they began towards their first Computer Science degree in university.
It also describes how that natural enthusiasm can be crushed, and how it can hopefully be regained or maintained.
This is written for the developer, whether a new recruit or a veteran, motivated or unmotivated, spirited or crushed, yet it's also written for software development managers (who might identify how to make the workplace more enjoyable and more rewarding).
Like most entries of this genre (see also Optimal Software Development Processes and Practices) I selected a small list of widely applicable, but often overlooked, factors. This most certainly isn't exhaustive, but hopefully it leads to a bit of reflection.
Software development can be a tremendously rewarding, enjoyable career.
Few careers offer comparable opportunities to weave intricate, complex structures that, while virtual, have such a positive impact on the world around them. Few offer the freedom and creativity that software development does, or the very real potential for entrepreneurial riches.
Whether it's building a new peer-to-peer application, control software for a massive power generator, or improving the workflow of the corporate scorecard system, done right this can be a very fulfilling, enjoyable, challenging pursuit.
Does your mind race at all hours, abuzz with potential solutions for vexing software development challenges? Do you lie awake at night -- anxious like a preschooler on Christmas Eve -- eager for morning to arrive so you can implement the crafty coding structures you just thought up? Do you frequently find yourself powering up your system in the twilight hours to implement the fruits of an epiphany?
Or do you put in just enough face time and superficial effort that sacrifice makes up for undelivered results? Do you purge your mind of software development the moment the virtual end-of-day whistle goes off, sliding off your Aeron dinosaur satisfied that it's one day closer to the weekend? Do you dread Mondays, motivating yourself to keep going with the dream of a far off vacation?
Do you eagerly embrace new technologies, seeing it as a challenging opportunity to learn something new when a solution calls for a new skill? Would you voluntarily dive into the innards of the Firefox web browser if a solution demanded it and you'd never touched it before? Do you swim through documentation, thirstily absorbing new APIs, tools, and languages to expand your skill-set, eagerly embracing industry advances?
Or do you dread anything different, praying that you're tasked with challenges that require only the skills you've long held, allowing you to apply them in a mechanical, repetitious fashion? Do you hope every project is an echo of a prior project? Do you put off any task requiring research, and show disdain towards new languages, techniques and practices, hoping that they don't gain traction?
Are you really passionate about software development? Be honest with yourself.
A desire to outshine a teammate isn't passion. Nor is a motivation to impress the boss. Neither is a combination of the two worn as a magic defensive cloak against downsizing spells. These are second-rate, artificial passion substitutes: Mixed into the recipe, they yield sub par results, often leaving a nasty aftertaste of burnout and dissatisfaction.
Instead I'm talking about a bona fide interest and enjoyment of the craft and challenge of software development, even outside of career or job security issues (though it benefits the same). This isn't a job ad demanding that you're "passionate about business reports!", but rather is just a moment for sober reflection on whether you're over-clocking life, or running idle instructions in a tight loop.
If you're like many software developers in the industry today, a feeling of enthusiasm and enjoyment for the pursuit is just a distant memory (often during the happy days of university and your first job). Instead it has become a career, and is just something you do from 9-5 (or more when passion is replaced by sacrifice). Skills have likely stagnated, moving just enough to compete with coworkers, or to avoid obsolescence.
Of course there are those who've never enjoyed this career, and they probably will never enjoy it -- it just isn't their thing. The only advice I can offer to those people is a suggestion that life is too fleeting to spend so much time doing something you don't enjoy.
Many others, however, remember the passion, and sporadically get a fleeting taste of it again. For those people I propose some personal habits that, coupled with workplace practices (for managers, as well as people who rightfully manage up), will help recapture and maintain that passion.
Software developers who truly love what they are doing are the ones creating the most innovative code. They're the ones with productivity rates multiples of their peers. They're the ones that feel a little guilty getting paid to do something they enjoy so much.
The Top 5 Habits of Productive, Happy Software Developers
Most of us will work for over a dozen different firms over our careers.
We'll leave for better salaries and working conditions. We'll relocate to accommodate a spouse's career. We'll be laid off during corporate mergers and spin-offs, or even when the company goes bankrupt. We'll get turfed out because we're over-skilled, and thus overpaid, relative to the needs of the position. We'll be downsized because we aren't compatible with the new boss' empire building schemes. Maybe we'll get bored of a position and seek out something new.
This is the employment reality of most careers in the 21st century.
To some professionals this represents an exciting journey, and each transition is met with anticipation and enthusiasm. These people feel confident in their abilities, have a network of peers in the industry communicating interesting opportunities, and their skillset is up-to-date and marketable (they have the appropriate laundry list of abilities, credentials and certifications, and upgrade as needed), and while the possibility of their current employer closing shop tomorrow is something they'd prefer not happen, and they probably love the great group of people that they work with, it isn't something that they fear.
To less prepared professionals, however, the idea of losing their cushy job hangs over them like a black cloud. Their lack of apparent opportunities, and the feeling that they couldn't find an equivalent job, is enormously destructive of both motivation and job satisfaction. Paradoxically, job protectionism (such as making one "indispensable" through obscurity, by denigrating coworkers, and so on) often becomes a more likely activity of people in such positions than legitimate contributions.
This is incredibly destructive to morale, not just for the individual in question, but for everyone on their team: Often the malcontent, contagiously demotivated member of the team is the least employable, and it can be debated which condition led to the other.
SUMMARY: No matter how much you love your current job, you should keep your CV current, and you should always keep up-to-date on industry opportunities. Know what skills are in demand, and try to gain experience in them (even if it means pursuing formal or self-training during your own time), and attain a level of comfort that you could transition to a different opportunity with minimal discomfort.
MANAGER SUMMARY: You should do everything in your power to make your group feel confident in their abilities -- ensure that everyone gets a chance with marketable technologies; encourage the pursuit of desirable certifications; and build skills through internal resources, workshops, and seminars. Unless you're running a sweatshop, this is unlikely to lead to a feared exodus of employees, but instead will empower and motivate your group to more openly contribute, and to demand more of each other.
The control we have over our environment can have a tremendous impact on our happiness.
Something as simple as a sporadically malfunctioning key on our keyboard can ruin an entire day, for instance. Similarly, when you're nearing a deadline and your network connection starts flaking out, it can make an enjoyable jog to the finish line a frustrating exercise of physical restraint (in this case restraining yourself from tearing the wiring out of the wall). At least we have optical mice now, eliminating one of the primary causes of environmental control frustration.
Many times our work habits inevitably bring a feeling of "lack of control" into our work lives: By failing to fully read the documentation for our tools, investigating their behaviour, APIs, and nuances, we often create a situation where much of our development is basically crap-shoot trial and error, reacting as things don't work as planned.
I've witnessed development groups, not to mention that I've demonstrated this unsavoury trait myself, unhappily fighting with perceived technology deficiencies (usually as a deadline rapidly approaches), moaning and complaining about what seems to be faults in the language, technology, or platform, forever building workarounds under a fog of uncertainty, when in reality it was actually a fault in the understanding of the same.
More often than not it's simply that they haven't spent the upfront time to understand the language (I remain amazed at the number of C# developers who have no idea what the using keyword is for, or why seemingly out-of-scope file objects are still locking files until some magical, indeterminate time in the future. Or the Delphi developers who needlessly nulled variables at the end of scope in a futile misguided attempt to fight mystery bugs), the technology, or the platform. Their frustration is created out of ignorance, and a small up-front investment would have sped up development, increasing the sense of control that the developers have over their domain.
SUMMARY: The next time something seems mysterious or unknown, take the time to properly investigate it. Classic lack-of-control approaches such as hacked workarounds or "reset the server daily" lead to a feeling of losing control, reducing job satisfaction and adding to the natural daily frustrations. And get your keyboard replaced if it starts malfunctioning.
MANAGER SUMMARY: Identify and investigate "easy-outs" proposed by your development team. While most software has faults, and products and technologies often work differently than we might imagine, many times such excuses are due to a lack of investigation and analysis. Even when things don't work as advertised, which is frequently the case, formally investigating and empirically determining behaviours is vastly superior to each developer endlessly fighting with and then hashing out strategies on a need basis. And make sure your developers have functioning keyboards.
I've worked in some great positions at the wrong times in my life, sapping my motivation until eventually I moved on. These positions were for great firms, with great working conditions and great coworkers and management, but it couldn't realistically adapt to accommodate my evolving financial needs. I invented dissatisfactions with the situation, turning an ideal situation into a daily torture.
After getting married and planning for our first child, for instance, the financial risk/reward that worked when I was living alone in a $600 apartment eating Ramen noodles was no longer satisfactory. Demands of owning a home, a car with infant carseats, education funds, daycare (for two children costing more than it would cost to lease two (2) BMW 750i's), and boxes and boxes of diapers, required more financial returns than I needed years before.
I moved on.
While the resulting role superficially wasn't as satisfactory, from a life perspective my mood brightened dramatically, and my day was much more enjoyable.
Of course this seems like cheap advice: Make more money! And Fast! Yet the reality is that developers often do make choices to the detriment of their financial condition, and if they go too far they will hate their job no matter how perfect it otherwise is. Working for equity of a start-up is great when you're just out of university, but it is destined for failure when you're more established.
SUMMARY: If your financials are out of balance, it will unavoidably sour your mood during the workday, making you resent your employer and your workplace. When life goals exceed the income of your position, immediately begin investigating alternatives (be it asking for a raise, looking for a more senior role in your organization, or seeking employment elsewhere). No motivational boost or cool company games room will overcome this basic life need.
MANAGEMENT SUMMARY: Be aware of the goals and needs of your group. Sometimes someone's needs grow beyond the possible return of a position, and it is important to appropriately communicate this (rather than giving vague hints of unseen raises and super-bonuses at some future point).
This is a rule that works for all professions -- having accomplishments providing satisfaction outside of work will smooth the inevitable downs of our professional lives, often providing one with a much better perspective. Without this, often minor workplace failures can explode into seemingly momentous events.
These accomplishments can even be in the same domain: A professional coder by day, and an open-source coder by night, for instance.
SUMMARY: There will be periods when everything seems to go wrong in the workplace. Having the cushion of achievements outside of work can avoid it spiraling into a workplace disaster, keeping spirits up through the tough times. Often non-work experiences benefit the workplace as well, whether it's techniques learned from nighttime projects, or delicious coffee courtesy of the nighttime barista classes.
MANAGER SUMMARY: There is a world outside of work.
Developers, as a general rule, are terrible at managing expectations: Many of us are prone to overpromising deliverables, assuring stakeholders that we'll deliver these amazing results sooner than is reasonable. I've fallen victim to this syndrome myself, and I've seen it occur rampantly across the industry.
When D-day comes we convince ourselves into believing that the users built their own unrealistic expectations, and managers forced us into untenable timelines. While often that is the case, just as frequently the developers were the origin of misinformation.
While there is a temporary sense of satisfaction wowing users and management with an exaggerated declaration of our abilities (we've likely even convinced ourselves), as time wears on this misinformation can be enormously destructive and debilitating. With every day closer to the deadline we get a little more desperate for a silver bullet, hoping that some magic technology or component will deliver us from damnation.
It seldom works out that way.
Users are unhappy. Management is dissatisfied. Employees are demoralized and devastated.
The best option is always to manage expectations, to ensure that we can reasonably deliver promised results without heroic effort.
SUMMARY: Plan for the long term, realizing that promises that aren't delivered on will cause you great workplace unhappiness later. Manage expectations to ensure that you can satisfy your "customers" with reasonable effort, and with a reasonably high probability of success.
MANAGER SUMMARY: Never demand unrealistic deadlines, and question employees when provided with the same. Encourage your troops to be more reasonable with their promises, especially to stakeholders outside of the group, and they'll have a much greater probability of meeting external expectations, leading to increased motivation for everyone.
This is an amazing, expansive career full of incredible innovation and endless opportunity. Ensure that you don't diminish your enjoyment through simple mistakes, such as pigeon-holing into a position, or endlessly setting up yourself for failure.
Control your destiny.
Tagged: [Software Development], [Programming], [Software-Development]
I've been doing this as a somewhat regularly updated blog for just over half a year now, and the results have been extremely satisfying: I get about ~2500 direct unique visitors on an average day (increasing 2-6x when something ends up being a meme-of-the-day on sites like Reddit or Digg, and of course many read via aggregators), search engine referrals are up to 200 or so a day, and viewing the "who's on" list is a laundry list of influential corporations and locations across the globe.
It does feed my ego a little bit seeing visitors from various governments, the CIA, nuclear research labs, just about every large financial company, and visitors from every end of the globe. My numbers aren't huge, but it's a perfect composite of influential and knowledgeable readers.
The most popular entries thus far are as follows (I'm providing the static version links where possible):
Effectively Integrating Into Software Development Teams
Optimal Software Development Processes and Practices
Spelling Matters
Everyone Is Above Average - The Overpopulated Top 2%
I've tried to minimize the number of entries (outside of the personal category, though this anniversary one being an exception) to keep the noise as low as possible -- if you're using a reader it won't constantly pretend there's new content when I'm just adding a peanut gallery comment about someone else's blog -- though on the flip side that means that I've delayed various .NET and SQL entries until they're "perfect".
Perhaps I might have to find a compromise somewhere in between.
After fielding several wiki-related queries by clients and associates, along with numerous questions and comments online, it is evident that it's the Year of the Wiki. Wikipedia has proven the concept, and users have a growing awareness of the benefits that organic information growth could bring to their teams.
As such, I'm putting together a feature covering wiki options and alternatives, including specific instructions for configuring and using Wikis on Windows (as this is a particularly neglected area, and much of the information that exists is terribly out of date or quite simply non-functional). Of course yafla provides turnkey Wiki solutions and training as a service as well.
One more point: The consulting work has always overflowed purely from word-of-mouth and associate networking, so the business website has always been terrible (sort of the whole "the cobbler has the worst shoes" thing). As yafla is now entering a growth stage, wailing past the temporary manpower limit, I'm finally going to change the corporate website to properly reflect the services and capabilities of the organization, and actually allow options for prospective clients to engage our services. That should be up shortly, growing and improving rapidly over the coming weeks.
Three yafla resource "shout-outs":
yaflaColor
- A dynamic web tool to select colors, including proper saturation
and lightness varations of colors
pureJpeg - Remove extraneous JPEG blocks
High Performance SQL Server - Information to ensure your
database designs and usage are optimal
Have a fantastic weekend!
I have been considering the possibility of yafla providing training services, developing and delivering programs here in the Greater Toronto Area (and globally where the monetary return makes it worthwhile), adding this service to the existing software development, outsource management, and consulting options. Not only is it an additional revenue stream, much more importantly it's another potential avenue for making contacts and getting involved as a vendor with new clients, creating opportunities to more easily offer our other services.
I've done a lot of group training in the corporate space, have been involved in quite a few tutorials and online training guides, and find it to be a very rewarding pursuit. Several of my associates have been heavily involved in the training industry during periods of their career. The possibility of hosting an "Advanced SQL Server" seminar or workshop, for example, seems very exciting.
We're very equipped to perform this task, and we certainly can do a better job than all of the trainers I've been exposed to. Our approach would never be to dedicate anyone to training alone (or even as a substantial period of their time), as continuing and up-to-date real-world experience would be absolutely critical to the program.
As such, I'd greatly appreciate any input anyone might have regarding the technical training industry, and how external training programs work at their organization. I know many mid- to large-sized firms have an "authorized training vendor", basically ensuring a universally weak level of training throughout their organization, and that could present a significant barrier to entry.
When Java first hit the development ecosystem, to many it wasn't just a method of doing efficient, high-level development, but rather it became a new religion: You couldn't only use Java as the glue between existing code, or even as the overwhelming bulk of your solution. A partial-Java solution simply wasn't good enough.
Instead your product had to be 100% Pure Java. The still sought-after eventual goal was a complete Java solution, from applications right down to the operating system, with only the smallest possible binary kernel, if even that. All of this would be running on a Java-aware processor, engineered specifically for Java.
Sun created a "100% Pure Java" campaign to push this philosophy, including banners and designations for appropriately certified software, and advocated it as a very desired moniker. Users were led to feel that mixed solutions were impure and somehow dirty: Are you some sort of nut running an impure solution, dirtied with some pointer munging, buffer overflow vulnerable C code? While there were (and remain) methods to call native code, they were discouraged.
Of course there is a lot of validity to this agenda. Primary being the fact that pure Java solutions are theoretically cross-platform, with no ties to external technologies. Compare this to a solution leveraging C libraries, which would require a rebuild or binary available for every distinct target platform. Additionally Java could only impose its sandbox and extensive security constraints if you stayed in the world of Java, and thus callouts to native code represented a risk.
In the real world, though, it often meant that developers were constantly solving long-conquered problems, redundantly reinventing solutions in Java that long existed elsewhere, or waiting until adequate libraries eventually appeared: Developers were pressured to use Java alone even when it was a hammer and the solution really needed a chisel.
Thankfully .NET hasn't been pushed in such a single-minded way (even if some of its champions have foolishly taken up such a misled cause, including some at Microsoft. Instead of a justified part of the solution, it becomes a religion. .NET! .NET! .NET! .NET!), and indeed Microsoft themselves has always facilitated, and even advocated, "impure" solutions. The majority of the .NET Framework, for example, is actually a very thin veneer over the existing Win32 facilities and libraries -- it was either that, or version 1.0 would have come with a much smaller, much less efficient library.
The "orchestration layer over native code" implementation is the reason .NET hasn't suffered the performance difficulties that Java has.
Microsoft chose to leverage what
they'd already done, to maximize both performance, and to maximize
the breadth of the library.
This advantage isn't limited to Microsoft, though, and the developer can utilitize this functionality as well. .NET offers very simple COM and P/Invoke functionality to leverage "legacy" code (or even new code developed in a best-solution, non-.NET technology), allowing you to easily use your existing DLLs and/or COM libraries as first class partners in your .NET solutions. Even if they're created in "dirty" languages.
I take advantage of this functionality regularly, utilizing existing best-solution libraries and functions, regardless of whether they're pure .NET or not. For instance in creating the static version of the "best of" blog entries, I quickly -- maybe 2 hours -- wrote a quick transformation tool that basically imported the "best of" RSS feed (it isn't included in the normal category lists), then doing some XSL transformations (using extension objects in the XSL given that XSLT alone wasn't adequate for some special purposes -- for instance HTMLDecoding the description block of the RSS) to the resulting XHTML, as well as creating an index page.
One goal when creating this solution is that the resulting pages are all fully XHTML compliant, and they pass the W3C validity checks. While I could easily see how the pages rendered in Mozilla/Firefox/IE/Opera, and of course they all rendered fine, technically there were a couple of deviations from the spec. Some of these errors and warnings were caused by unavoidable transformation issues, while others were caused by minor mark-up errors in the original blog entries (both because of my own errors when doing it by hand, but also because of Radio Userland's "helpful" auto-"cleanup" of HTML. It is remarkable how often auto-formatting is detrimental).
HTML Tidy to the rescue.
I had several options with HTML Tidy, the easiest of which would be to ShellExecute out to the EXE, telling it to process an existing file. I could have taken more time and tried to make a managed C++ version of Tidy, but I really didn't want to spend that much time.
I decided to have a bit more fun, not to mention building a more integrated, higher performance solution, and use the Tidy dll from the micro-.NET utility. I grabbed the Tidy source code (Tortoise CVS is a great solution for this, in this case using :pserver:anonymous@cvs.sourceforge.net:/cvsroot/tidy), updated the included MSVC projects to Visual Studio 2005, and added them to the transformation utility solution. I set the Tidy dll project output to the build directory of my .NET utility (in this case $(SolutionDir)\blogStatic\bin\$(ConfigurationName)). The MSVC build worked perfectly right away, which is amazing given that Win32 isn't an officially supported build.
To reference the Tidy dll methods, of course I had to add the DLL import signatures, in this case adding only the ones I had a need for.
[StructLayout(LayoutKind.Sequential)]
struct TidyBuffer
{
public IntPtr
bp;
/**< Pointer to bytes */
public uint
size; /**< #
bytes currently in use */
public uint allocated; /**<
# bytes allocated */
public uint
next; /**<
Offset of current input position */
};
class FileClean
{
[DllImport("libtidy.dll")]
public static extern IntPtr
tidyCreate();
[DllImport("libtidy.dll")]
public static extern int tidyParseFile(IntPtr
tidyPointer, [MarshalAs(UnmanagedType.LPStr)]string
fileName);
[DllImport("libtidy.dll")]
public static extern int tidyParseBuffer(IntPtr
tidyPointer, ref TidyBuffer tidyBuffer);
[DllImport("libtidy.dll")]
public static extern int
tidyCleanAndRepair(IntPtr tidyPointer);
[DllImport("libtidy.dll")]
public static extern int tidySaveFile(IntPtr
tidyPointer, [MarshalAs(UnmanagedType.LPStr)]string
outFileName);
[DllImport("libtidy.dll")]
public static extern int tidyRelease(IntPtr
tidyPointer);
[DllImport("libtidy.dll")]
public static extern int
tidySetCharEncoding(IntPtr tidyPointer,
[MarshalAs(UnmanagedType.LPStr)]string encoding);
[DllImport("libtidy.dll")]
public static extern int tidyOptSetBool(IntPtr
tidyPointer, int value, int Bool);
public static
bool CleanFile(System.String outputfileName, System.IO.MemoryStream
docDataStream)
{
int result = -1;
IntPtr tidyPointer = tidyCreate();
try
{
// We want the resulting
file to be UTF8 encoded
tidySetCharEncoding(tidyPointer, "utf8");
byte[] docDataArray = docDataStream.ToArray();
TidyBuffer tidyBuffer;
tidyBuffer.size =
(uint)docDataArray.Length;
tidyBuffer.allocated =
(uint)docDataArray.Length;
tidyBuffer.next =
0;
GCHandle pinHandle = GCHandle.Alloc(docDataArray,
GCHandleType.Pinned);
try
{
tidyBuffer.bp =
Marshal.UnsafeAddrOfPinnedArrayElement(docDataArray, 0);
if (tidyParseBuffer(tidyPointer, ref tidyBuffer) >= 0)
{
tidyOptSetBool(tidyPointer, 29, 1);
tidyOptSetBool(tidyPointer, 23, 1);
if (tidyCleanAndRepair(tidyPointer) >= 0)
{
result = tidySaveFile(tidyPointer, outputfileName);
}
}
}
finally
{
pinHandle.Free();
}
}
finally
{
tidyRelease(tidyPointer);
}
return (result == 0);
}
}
Most of this should be self-evident, however the two tidyOptSetBool calls may be a little cryptic. For the sake of brevity I haven't used the constants, but 29 is the TidyMakeClean value of TidyOptionId enum (see tidyenum.h), and 23 is the TidyXhtmlOut value. Together these indicate that I want to clean the documenting, converting it to XHTML. Note that I've also set the encoding to UTF8.
Voila, after transforming the RSS to the memory stream as quasi-conformant HTML, I passed the stream to this function, along with the desired output filename, and out went a cleaned-up, valid XHTML document. Pedants everywhere were thwarted from pointing out minor deviances from the standard. I could have processed to another buffer, and then done follow-up processing in .NET as well, but this was sufficient.
This is a trivial example, but it really exemplifies the great value of the easy interoperation of .NET. With it I could instantly leverage existing code, without having to search out bastardized ported versions, and instead could go right to the source.
[The static location of this piece can be found at this address]
Work habits and conditions vary dramatically between software development groups.
At one extreme of the spectrum are productive, tuned development
teams delivering solutions on schedule and on budget, staffed
with passionate experts happily building their skills and
careers while providing valuable cutting-edge solutions. At
the other extreme lie dysfunctional, under-utilized teams. The
latter group endlessly catapults magnitudes past estimates of
time and money, with a 
revolving door roster of
contemptuously-treated employees.
Most shops fall somewhere in-between -- imperfect, but continually working towards better, more efficient practices and process, all while trying to provide a rewarding, fulfilling, and career-building experience for team members.
This entry is targeted at that continually-optimizing audience, and is written largely for development group managers and team/group leaders, though it's still informative for the general development community (lateral and upward management is critically important, and is the force that drives most process change).
These observations are gleaned from my experience with teams of varying sizes and types, acting in a variety of roles including management, team lead, developer, process consultant, mentor, mentee, and technology consultant. I have directly participated in, or observed, the development process in small, tightly-knit engineering groups stocked with professional electrical engineers, to mega corporations with walking-dead teams awaiting the next executive shuffle to mete out some doom on their orphaned projects.
I've learned from the successes, but I learned even more from the failures.
I've focused on best-practices that are optimal in virtually any team, and which are frequently evident problems, while avoiding those that are more likely to vary significantly between teams and organizations. Work environment, for example, is very subjective and context specific: Some developers and teams work optimally in private, quiet offices with comfy chairs and scheduled communications, while others thrive in hectic, noisy open offices, interfacing directly with the "customer" throughout the day, eagerly setting up development kiosks on whatever can serve as a seat. Some organizations need armies of BAs and BSAs and QAs and UAs, with layers upon layers of heavy process and checks built atop a traditional waterfall development model, while other shops work best with a highly agile, frequent delivery model, completed by multi-role development resources. It isn't possible to declare universal best-practices within those domains.
I also appreciate that most real-world teams have budgets, and that feel-good lists of .COM-style excess aren't realistic or helpful in the current IT climate. Every developer doesn't need every tool available, or every electronic toy on the market, to do their job effectively. I haven't tried to pander to the development community, blowing smoke about how entitled and empowered they should be, beyond the level that is actually beneficial to their career and the role.
On to the list!
Select processes, practices, and standards based upon your particular type of organization, type of project, and workforce, and then customize them for your specific needs.
Of course this suggestion seems ridiculously obvious, yet this is an industry rife with groups foolishly mimicking the process of those who appear to be successful.
Whether it's the previously agile vertical-solution shop trying to act like they're a bank, overloaded with process and CMM levels far beyond necessity because that looks like what the "big boys" do, or the banking group trying to adopt XP practices and frequent deliveries because it's the hip new trend that all the blogs are talking about, when their product only requires bi-yearly deliveries under a heavy layer of checks and balances: What works for one type of organization doesn't necessarily work for another, and adopting those processes under the wrong environment will bring defeat rather than success.
Don't act like you're developing life-critical flight control software when you make a P2P app, just as you shouldn't do the opposite. Don't pretend that you're an ISV, or assume that much of what's right for a retail software vendor is transferrable, if you're actually an in-house IT development shop.
This sort of cargo-cult mentality exists at a lower level as well. During the .COM boom, success apparently came from stocking up on Aeron chairs -- Get the chairs, and you'll do well. After the collapse the idiocy continued, with the same chairs now being demonized: Now they could only bring doom, and had to be removed post haste. It's a chair people. In the grand scheme of things, it really isn't going to make that much of a difference, positive or negative.
The same could be said about foosball tables; catered lunches; big, corporate-style meetings; having big cozy offices for everyone, or tiny open plan offices for everyone. Someone seemingly did it to their success, so therefore it was perceived as an applicable practice for everyone.
Just because someone else benefited from it doesn't mean that it would benefit your organization. It could very well hurt your organization.
Source control is the foundation of good coding practices.
Not only does source control keep your code centralized (facilitating organized access for developers, along with easy backups and automated integrity assurances), with the proper use of branching and labels it can greatly increase the agility of your team: Having the comfort of making major changes in a code branch, while retaining the ability to revert to a labeled release or trunk, is liberating, reducing developer paranoia and ameliorating the risk of change.
Whether you're using Microsoft Visual Sourcesafe, Perforce, CVS, or one of the countless other available source control products, it is a critical component in your toolset. It's critical even if you're a single developer working alone in your one-man software development shop.
Source control is also invaluable for change management and auditing, historically tracking how much has changed, and where, over a given period of time, which is a benefit that often pays off handsomely years later. For example auditing the code history to determine when an errant algorithm was introduced to gauge how long it's been screwing up the calculations.
Adequately trained quality assurance teams can even use source control to determine what has changed between releases to tightly focus their testing efforts without relying on the often incomplete word of the developers.
The technique to use source control properly depends upon the tool that you're using, and the standards and needs of the group: Some groups use source control only to check-in production releases, while other groups check code in frequently, with every developer checking in multiple times per day, merging branches at convergence checkpoints. Ensure that you know the feature-set of your source control package inside and out, and that you're following a rational, best-benefit standard of use for your scenario.
If your usage is non-optimal because of limitations of your SCM tool, consider a product change. It really is that important.
Tip: Frequently perform project builds on a clean machine with limited network access by following development environment configuration instructions, and then getting and building from source control. During this process many teams discover hard coded paths, missing dependencies, and solution defects. It's better to detect it early than for development to grind to a halt the day before a big release when the corporate IT department decommissions that old server that supposedly wasn't in use for the past year.
Standards can be extraordinarily detailed, or as simple as noting which industry standards a team subscribes to (e.g. "All .NET coding will be done in conformance with the Microsoft .NET Design Guidelines"). It could even be the formalization of the lack of specific guidelines in a certain realm (e.g. "Put the braces wherever you'd like, and either tabs or spaces are fine"). Documented standards should cover all aspects of development, including development environment, tools, accepted languages, best-practices, check-in behaviour, naming guidelines, and so on.
Of course standards should be living documents, with occasional reassessments leading to the removal and addition of new points.
If, on the other hand, your standards are word-of-mouth, or "by example" (usually with many conflicting examples throughout the organization), you don't really have standards -- more likely it's acting as a demotivating, inefficient confusion for newer developers to the team, and a constant political skirmish between the veterans of the group: While good developers will happily adapt and conform to the idiosyncrasies and preferences of a group or project, it is frustrating and unproductive when they have to constantly go back and rename tables and rework classes because of unwritten "standards" that exist only in the minds of a few. When developers lose confidence in their code because of uncertainty over trivia such as spacing standards, something is seriously wrong, and people are focusing their care and concern on something that should be thoughtless.
As an added benefit, the process of documenting standards is often a time for rational debate, when "that's the way we've always done it" standards have to be legitimately rationalized and justified, or rightfully discarded. Many teams have improved their process, finally jettisoning unjustified legacy standards, during just such an exercise.
Having concise, documented standards (which can be the documented absence of standards in a particular realm) saves everyone from having doubts about whether they're conforming, eases new developers into the fold, and increases the percentage of good standards while removing the excess baggage. It also facilitates external assistance more smoothly.
Every project, and every team, should have a daily or bi-weekly status updates by every member to their team leader or manager, or better still to every other member of the team (including from the manager to the team). This is regardless of your development methodology.
This can be a scrum, or a simple daily email stating what work the person is planning for the day, and what they achieved during the past day/period. This practice not only keeps everyone focused on the task at hand, with better time accountability (just like everyone else, and possibly worse as a group, developers are procrastinators), but it also gives a transparency of development that let's the manager, leads, team members, and stakeholders see where impediments exist, providing them an opportunity to help, or to recognize and respond to schedule slip early on, before it becomes a crisis. This transparency is one of the fundamental goals of the CMM process, and the benefits can be largely gained just by frequent, scheduled, mandatory and accountable communications.
Optimally this information will also exist in an archived, centralized, widely accessible location, such as an audio recording of a scrum, or a centralized collection of update notes.
It is critically important not only that this information is conveyed, but that it's actually consumed and retained as well -- management and the rest of the team must pay it proper heed, and they shouldn't wallow over during the day to ask what was previously stated. That undermines the entire process when it turns into a wasted effort that no-one pays attention to.
Having lots of specifications, standards, code libraries, discussions, solutions, and examples strewn across the topology of the network is of little use to anyone, and can be detrimental. Outdated documents will continually reappear, and misinformation takes on an unstoppable life of its own.
Centralize and standardize, ensuring that everyone knows exactly where to find all pertinent information on a moment's notice. Everyone should know exactly where they can find, and thus where they can store, specific types of information.
As it is, many teams start off with the "dumping bin"-style repository when they begin a project, planning on sorting it out later, however soon enough there are GBs of files -- many of which have little or nothing to do with the product or project -- confusingly cluttering a massive mess of a directory structure. This puts a serious damper on productivity.
Documents can't be found. People have to constantly interrupt each other to ask where to find something or where to store something. Eventually an expensive initiative will have to be undertaken to separate the wheat from the chaff, at a time when most people have forgotten why all of the various unrelated information is there in the first place. Undoubtedly good data will be removed, and bad data will be retained.
Before you create the first project or product file, decide exactly how you're going to organize the information, planning for the long term. Who will have access to it, and what permissions will they have? How they will access it? What will be contained where?
Tip: Documents such as specifications yield the same benefits of source control.
The first step of any project shouldn't be to hit the IDE, or even to start high level design work. The first step should be to research and understand all competing or similar products, and any and all libraries and tools that could be leveraged to empower your solution.
As I write this, countless teams are spinning their wheels reinventing the Microsoft Application Blocks, or they're developing a data transformation service that could be integrated in Biztalk or SQL Server Integration Services less expensively (and much more flexibly). Perhaps they're developing a web portal that's just an incomplete, second-rate clone of SharePoint, Community Server or Zope.
There are even teams duplicating functionality already existing in the .NET Framework or standard template library because they never spent the time to look.
For virtually any need there are libraries and existing solutions that could either replace the need for the project altogether, or more likely could serve as a foundation of a more robust, more capable solution. If you're in the solution business, the desire to propose and implement the best solution should trump any desire to code for the heck of it.
Developers can't complete that sort of competitive research in an afternoon, or as a footnote, nor should you rely upon off-the-cuff commentary provided in meetings by individuals vaguely aware of the options.
While it varies by project and industry, such research usually requires a significant upfront investment of time before coding begins, allowing the stakeholders and development group to all feel entirely confident that the project is justified and will yield the best possible results. Document every finding and decision to save endlessly having to explain why one decided to develop what seems to be a clone of Biztalk.
One of the primary reasons why many shops continue to grossly under-estimate the time required for projects is that they have limited, or fictional, historical real-world time tracking to base future projects upon -- they might have half-a-dozen developers working full-time, but the best they can account for at the end of the week is that a couple of small app issues got worked on.
This is sadly very typical.
Where there is some form of time tracking system in place, often it is of little value: Developers, as a general observation, will try to find ways to under-report the amount of time they spent on a particular development problem or project, as it's a bit of a badge to claim to have "thrown together" complex solutions in a "couple of hours". Furthermore few shops properly partition times (e.g. by project stage and task), further blurring the value.
Implement a time tracking system, and coach your team to accurately and regularly post times (daily, if not more frequently. Any less frequent and accuracy suffers dramatically (as someone with the bad habit of filling out timesheets at the end of the month, anxiously looking at file timestamps to figure out what I worked on when, I say this from personal experience).
Ensure that there is enough granularity that the time reporting is usable. For instance that you can see that competitor research actually took two weeks, instead of the foolishly anticipated 2 hours.
Avoid tacitly encouraging false timesheet entries: Don't act concerned when real world values start appearing, and most tasks are much more time-intensive than originally thought. Adapt and accept that reality differs greatly from the mirage that most people imagine that they see.
You have your team of six hyper-intelligent and eager software developers, overloaded creating the next great scrum-tracking application. Based upon some user feedback, you realize that a small percentage of your users may want to interact with the system in a minimal way using J2ME equipped cell-phones, but your team is entirely staffed with .NET experts. Do you a) add a J2ME developer to the team, and just try to expand the J2ME product line, b) let one of your existing developers try their hand at J2ME, c) outsource that non-core component. Most shops would pick b without hesitation, incorrectly justifying it under the guise that it's "free" because it's covered by the payroll.
If you decide to try your hand at it in-house -- despite the considerable domain difference -- the results will almost certainly be a second-rate solution, delivered more slowly and more expensively than just outsourcing the need. This is worsened in that a resource originally dedicated to the core product has been diverted to a non-core need, and will come back to the core product with a need to spend time reacclimating.
This isn't to say that this is always a bad solution: Sometimes it entirely makes sense to build such skills in the core team (if this is a component that will need regular modification, or if it's an expanding customer base), not to mention that the work diversion can prove entertaining and rewarding for a developer.
Undertake non-core competencies -- be it building a one-off data bridge to a partner, putting up a corporate website, designing logos and icons, or countless other non-core needs -- with eyes wide open, realistically assessing the cost to do it in-house. Estimation of the "knowns" is already notoriously bad in software development, but the situation is far worse for non-core competencies, where developers are prone to extraordinary underestimations.
Realistically consider leveraging external solution providers (or even other teams within the organization) where it can keep your group focused -- with appropriate communications, teams will understand that it enhances their value and effectiveness, rather than presenting a risk to their employment. [Note: My organization provides outsourced consulting and development, so I do have a conflict of interests. However we always go in with the intention of empowering and furthering the existing team, and never to supplant them]
Getting the team working on nights and weekends means little if it saps their passion to produce, expensively inflates the bug count, and increases turnover. Yet sadly a lot of managers work under the delusion that effort and sacrifice are worthy substitutes for results. Indeed, many management techniques are built entirely around deriving maximal effort and demanding complete sacrifice, inventing artificial deadlines and manufactured crises to prod the troops.
The results are invariable destructive and counter-productive.
This is rooted in basic human psychology. Just look at a famous cough syrup with a heavy advertising presence: Surely it must work better because of the sacrifice required to ingest it? Contrast that with the presumption of self-destruction that most attribute to activities that people find enjoyable.
If it's fun or requires little sacrifice, then it can't be beneficial. If it is unpleasant, then it must be doing good. These foolish and unfounded notions have sold a lot of snake oils over the years, and they've supported a lot of human folly.
Savvy Machiavellian developers know how to exploit this management myopia, under-delivering until crunch time, and then putting in some superficial extra effort to get the project done, yielding kudos all around for going "above and beyond". Just by finally delivering long overdue results that they could have achieved in the normal workday. This is especially the case in shops with limited time reporting, and no regular status updates: Show up early in the morning (sending emails as validation), grunt and groan and gripe, shuffle around regularly, and then leave late (again sending emails to validate how committed one is), maybe VPNing in during the middle of the night to again demonstrate great sacrifice.
In some shops that alone is a substitute for any results at all.
On the flip-side, the "slacker" that comes in at 10am, leaves at 4pm, and takes extended lunches somehow keeps generating the bulk of the product design and code, but his apparent lack of effort and sacrifice will be dealt with at the next performance review.
Many heavy and onerous processes are undertaken, and then maintained, not because they've empirically proven themselves to be useful, but rather because the effort involved gives a mentally cheap illusion of achievement. From volumes of documents that no one ever references or validates, to arduous signature gathering exercises. Most of the time they're the mindless completion of a task with no real benefit to the project or the process, but it goes unquestioned as a part of the flow.
Discard all processes that are mechanically completed with little actual benefit. Tweak those that require more effort than necessary, maximizing the results while minimizing the effort.
Effort alone doesn't make your quality better, software better, or team smarter. Focus on results, and welcome and encourage results that come with minimal apparent effort or sacrifice.
Building a top-notch, effective software development team is a lot of hard work, coupled with a bit of luck, but hopefully some of these points have given a bit of food for thought that might encourage some development in the right direction. This list most certainly isn't comprehensive, and I've stayed away from domain-specific practices, but it is a start.
Tagged: [Software Development], [Programming], [Software-Development]
Getting back into the flow after the holiday break.
In addition to branded software, yafla also provides consulting and custom software development (in the Greater Toronto Area), which has kept me extremely busy over the past while. That's an area of the business that I haven't really written about (primarily because I keep client project and platform details strictly confidential), however I have a case study where the client is interested in getting their name out there, and the need - and from it the solution - are really interesting and noteworthy, so I might start working it in.
As an aside - both the survey component and yaflaColor are currently broken (as of this writing). Both have been updated to .NET 2.0, and for some reason my third-party host has an issue where helper assemblies are being locked by an outside process, possibly due to synchronization issues with the NAS. This should be resolved shortly. I've let them sit broken for the day simply because they're non-critical and it lets the 3rd party solve the problems on their end.