A decade+ ago, most "online" comments were conceived and birthed in feature-rich, fat-client applications. These were tools that generally offered a rich gamut of functionality: spell-checking, automatic intelligent threading, offline composition, selective content blocks (such as plonking unliked trolls, censoring expletives), automatic notification of certain keywords or topics, alongside a wide breadth of additional capabilities.
You could read and participate in conversations on a massive array of topics, from law and order, to product support forums for a particular vendor's database product, to the seedier side of the alt hierarchy. All using the same client application that you were comfortable with, configured just the way you liked it.
After authoring your brilliant, convincing argument (or your question about what video card to buy or how to call a certain API function) and hitting send, the application would queue it up much like an outgoing email, and when the opportunity arose (when you dialed up to your local BBS), it would send it to your local server via a standard protocol, where it would be shared with a decentralized universe of servers.
Usually your brilliant literary gem would be immediately visible to the world -- limited only by the rate of propagation -- though a small number of newsgroups had post moderation that requiring each new addition to first be approved.
This standardized protocol, message format, and distribution mechanism allowed for rich client functionality without reinventing it for every single newsgroup. Imagine how absurd it would have been if you used a different set of tools, with a different set of functions, to interact with comp.sys.ibm.pc.hardware.video than you did with comp.sys.ibm.pc.hardware.sound?
Just as importantly, the standard message format and transport protocol allowed for very easy indexing and archiving -- easily searchable across time and space by whichever search vendor did the best job. This is how we got the incredible functionality of DejaNews (which was later purchased by Google and rebranded as Google Groups), which managed to reach its indexing fingers back a decade earlier than it was even imagined.
If you do software development, you've probably found newsgroups to be by far the most useful resource to search when looking for answers: While a normal web search will yield thousands of noise responses and pay sites begging for money to see the answer (that they usually ripped from a usenet newsgroup), a quick tab over to the groups will usually immediately find the archive of someone who faced a similar question or problem, and the helpful replies.
Of course Usenet is still around and very much alive, and some sites still use NNTP. Unfortunately the quantity of useful answers has been declining, or at least that's my impression, as more and more conversations are being siphoned off into poorly structured, often unindexed islands of information.
Why is every new web app creating yet another terrible reinvention of a container for discussions? Why are we functionally stepping back 20 years for every single new forum (see Digg, YouTube, Reddit, and others for examples of colosally broken discussion systems that people interact with despite their enormous failures, having no alternatives. There are a few, Slashdot for instance, that are moderately evolved, but it took half a decade to achieve a somewhat usable system, and even then the failings are numerous)?
Worse still, why are so many sites storing conversations and threads in isolated silos of data, stored and communicated in completely non-standardized ways. I can easily find and reference threads that I reminisce reading on a usenet newsgroups 14 years ago (usually for "I told you so!" purposes), yet it's often impossible to find a thread or comment on a modern web forum even if I remember seeing it a month ago.
This isn't an argument for a return to the days of yore, and I'm candy-coated the history and usability of Usenet, but it does seem like a lot of people are continually reinventing the wheel, ignoring the lessons of the past.
It does seem like the value of each additional piece being added to the global solution set is being diminished or completely lost: Where once we had clearly defined domains of information, clearly deliminated and indexed by topic, with a clear threading organization and meta-data structure (author, date, what other comment entry it's a reply to, and so on) that could easily be interpreted by anyone who understood the NNTP spec, now we're at the point where search engines have to try to interpret a million variations of rendering engines, inevitably losing most context and metadata, and that's only if they happen to even crawl across the conversations in the first place.
Somehow we need to find a medium, taking from the past while incorporating modern technology. Perhaps a new embedded commenting structure should be an addition to Firefox 3.
How many times do you load sites like Slashdot, Netscape, Digg, and others during the day? Do you find all of the links colored a:visited? Do you wish they'd age-out quicker so you'd have more sites to visit and discuss, and to forward to peers and friends?
How many blogs do you read? How many RSS feeds do you subscribe to?
How much time do you spend browsing through Wikipedia entries?
How many futile online debates do you participate in (Microsoft versus Linux. Liberal versus Conservative. Pro- versus Anti-. Debates where nothing will ever be resolved, and no positions will ever be changed. No, seriously, no matter how convincing you think you are, and how enlightened the forum is, it is extraordinarily unlikely that you're ever going to change a single mind, much less the world).
How many facile blog entries are you wasting your time authoring, tickling the reader's ego by telling them that they surely must be among the cream of the crop reading such an enlightened entry (which should offend readers who can detect the obvious pandering, yet it's a tactic that is becoming common), desperately hoping that you'll get atop some momentary meme list, your 5-minutes of fame yielding nothing of real Earthly consequence?
How much time are you spending on all of the above in an average day?
Take a moment and answer honestly. Feel free to answer in the comments if you'd like, though this isn't meant to be a competition.
For many people the answer might be "very little or none at all" , but for quite a few I suspect the answer will lie somewhere between "a hella' lot" and "all day".
What are you gaining from that sunk time? Entertainment? Intellectual stimulation? An enlightened perspective? Work-related information (that's was my personal justification back when Slashdot was much more technology focused, and the discussions actually did expose me to new information about the industry and technologies, though at the time the turnover of stories was so low that a single short visit a day was sufficient)? Are you gathering some nebulous site-specific karma that's more of a badge of excessive free time, or on some sites a perspective that most correlates with the group think, than wizardly knowledge?
Most of the time the net benefit of a day spent on the meme sites is nothing more than entertainment, but few would admit to themselves that it was akin to watching an all-day episode of Survivor or Heroes.
What could you have achieved in that time?
Could you have spent that 3 hours building a Reddit clone (to waste someone else's time)? Launching your eTailoring venture? Learning Japanese? Learning Python? Taking up a new language? Implemented that awesome new piece of functionality? Gone to the gym and started the journey to fitness? Cooked an amazing, exotic meal?
Talking to and observing peers in the industry, I've noticed that this time suckage has become a widespread, massive squandering of opportunities, with so many talented developers, designers, and intellects basically wasting their days away. From a personal perspective, I know that occasionally I find myself "quickly" checking to see if there is anything on the feed sites before delving into a difficult problem, and some time later realizing that I'd completely burned away all of the personal project time that I had set aside.
It's for this reason that I'd previously talked about router blocks and Firefox rationing.
Honestly monitor where your time is going, and if you're being the best you that you can be.
Aside: Many will wonder if the "facile blog entries" bit was self-referential. These blog entries once served a commercial purpose -- yes, they were entirely selfish, as most human efforts are -- but as the associated ventures turned out to not need the publicity, and I no longer was looking for cold consulting clients, that overt selfish motivation evaporated. I've never really been motivated by appearing on the meme sites, as a quick look at the "peers" in the space tells me that it isn't really a great accomplishment to be on the front page alongside a "PICUTRE OF KITTEN [AMAZING!]". Now I mostly do this as a brain exercise, keeping entries very short, infrequent, and quick to compose, saving my time for my attempts to Try To Take Over The World!
We've been using Microsoft's Team Foundation Server for version control and basic work item/requirements/bug tracking for about 9 months now. All in all it's a good tool, though really it still feels like a version 0.8 beta that got pushed out the door a little early.
The Good
The Bad
The
application tier is incredibly fragile. If an
enterprising team member decides to do some clean-up directly in
the TfsVersionControl data-tier database (getting around missing
functionality in the tools -- for instance there is no way to
permanently delete, aka destroy, in the tools, remarkably,
so if someone checks in a 500MB file and you want to remove it,
you're forced to do it directly in the database), you will discover
that a single missing related record -- the database doesn't define
or enforce foreign-keys, so it isn't going to block the DELETE
command there -- will cause the application tier to die a
hundred deaths, excepting out on null values and other inanities.
This is made far worse by the fact that the application tier caches
a lot of lookup data, so check-ins/outs will work for a while after
these related records were moved, making a simple database rollback
impossible. Instead you need to go through every database manually
rationalizing all of the data, determining where the application
tier is dying.The Ugly
All in all it's pretty decent, but I think they called it done a little early.
For the stand-alone application developer targeting the Windows platform -- that increasingly endangered species -- when the need arose for a database backend, the default choice was historically the Microsoft Jet database engine, usually to interact with an mdb (e.g. Microsoft Access) database file (though Jet supported other, less-used options as well).
This goes all the way back to the terribly awkward DAO dll libraries callable from C, though interaction greatly improved through the years, to the ease and simplicity of today's ADO.NET.
When deployment time came, you could distribute with the freely redistributable Jet runtime (the only significant limitation being that you couldn't create a direct MS Access competitor, with table designers and the like), and while the client had no need for a Microsoft Access installation or license, if they did have it they could interact with the database directly where the need arose, or for extended functionality (e.g. reporting via Access). These mdb databases could also be accessed through other tools such as SQL Server linked servers, etc.
I write that in past-tense, as Microsoft has been beating Jet to death as of late.
The first punch was the complete lack of a 64-bit migration path -- existing or planned -- meaning that in a 64-bit instance of SQL Server you can't add linked Access databases, nor can you interact with Jet-supported databases from the 64-bit runtime of SSIS (though thankfully in that case you have the fallback of using the 32-bit runtime). And while Access 2007 runs atop a heavily modified version of Jet called ACE, the new library itself isn't redistributable being intended only for use from Access.
So what is the replacement? The primary replacement was the former MSDE (Microsoft Database Engine), called SQL Server Express in the latest iteration. Microsoft really wanted to push developers to the SQL Server platform from the smallest need to the largest need.
With a liberal, free redeployment, a very easy upgrade path to a "real" instance of SQL Server (and the easy integration and interoperation with other instances of SQL Server, taking part in functions such as replication), this is a compelling choice but for the fact that it is a resource-intensive overkill for many simple needs, with a multi-hundred megabyte install, a separate service which itself is far from slim (it is an actual instance of the same SQL Server database product that might be hosting the corporate HR databases, with only a couple of minor hardcoded limits differentiating it), and then the potential administration headaches in the future (the Slammer worm primarily infected MSDE installations that many users weren't even aware they had running. SQL Server's default configuration is far more intelligent now, such that by default it only listens on localhost, and it supports robust attaching/detaching of databases, but there still is an excessive surface-area for attack if only a basic database was desired).
If all you want is some basic table structures with simple indexing, the bulk of the database management system meant for large-scale corporate data warehouses just isn't reasonable.
Microsoft has another option now, albeit growing from a product path that has been around for a while, called SQL Server CE aka Compact Edition (confusingly you'll find it called Mobile Edition on many of the supporting documents, as that was a prior product name). Supporting a subset of T-SQL, and basic tables (no views, stored procedures, or triggers), it's an in-process, very lightweight option if you want a simple backend database in your application, and want it to easily interoperate with some other Microsoft technologies.
It's primarily unmanaged, but provides excellent .NET interoperability.
It isn't SQL Server, though. The code doesn't come from SQL Server. T-SQL is limited to a subset, the database format is completely different, and basic functionality like full-text search is missing. From an integration perspective, while it is manageable from SQL Server Management studio, the only real consistency is in the object explorer and the query analyzer, while index configuration and schema design occurs in completely new modules.
In its implementation some questionable decisions were made, such as the lack of non-unicode text types (there are a significant number of very legitimate uses for ASCII text. Going UCS-2 makes it more likely that surrogate keys get used where natural keys could have been the better choice at one-half the size, and is just a waste of space and performance if it isn't actually necessary).
So why am I talking about SQL Server CE at all?
The compelling feature is that it runs on the gamut of Windows targets, including mobile editions (e.g. smart cellphones and PDAs), including tools to sync between the devices. If you wanted to make a Getting Things Done task-tracking style app atop a reasonably robust, feature-rich-enough database, and you wanted it to be usable from a desktop and a PDA whether connected or not, it presents a very interesting option.
I'm going to play around with it a bit in the coming weeks when
I need a distraction from more critical work, and this is just the
intro piece to further analysis of this product. I may look at the
unfortunately named VistaDB
and other embedded, in-process options (I'd love to consider the
embedded PostgreSQL -- itself a fabulous RDBMS -- however
resources for it are few and far between).
Browsing through the meme-linked sites a few days ago, I came across this article on Vanity Fair. In that entertaining read, the author details his many failed attempts, and then ultimate success, trying to track down the location of the Windows XP "Autumn" background picture. Remarkably the photographer didn't even realize that he was the source of a photo seen by hundreds of millions.
The Vanity Fair author was mesmerized by the tranquility and beauty of the picture, and became convinced that he had to find the location, and possibly even visit it. When we see only a small part of a scene, we fill in the surroundings, and he envisioned an idyllic countryside.
Eventually he gets a strange email from a Microsoft source that, while factually a bit incorrect, remarkably leads him to a historian in Burlington with an amazing memory of the area.
The article was a bit surprizing to me as this is my stomping grounds: I live in Northern Burlington -- a part of the Greater Toronto Area -- and every spring, summer and fall, many weekends see us taking drives through the various roads in North Burlington. It isn't really good for the environment and the conservation of oil, but it's remarkably effective getting young children to sleep.
The particular road where the picture was taken is one that we've been down countless times (the summer past we happened upon a Ferrari get-together very close to this location, with several dozen very high-end Ferraris sharing the road with us).
I've taken many fall pictures in the area, as the colours are often spectacular.
Anyways, we were off to go tobogganing today, so I just had to drive past the source of a previously-anonymous picture that so many had seen. As "proof", I took the picture above, trying to emulate the zoom and orientation of the XP picture. The old fence is gone, as is the large tree on the left, but otherwise it's the same.
And to satiate the author's quest for information, let me say that Northern Burlington/Halton is stunningly gorgeous. With multi-million dollar homes intermixed with smaller existing housing (which will eventually get replaced by mansions I'm sure), bordered by the Bruce Trail and the Niagara Escarpment, it's an area of low-intensity, almost-hobbyist type farming , often with nature completely untouched. There are large areas of incredibly dense hardwood trees.
Some areas just a bit further out make you feel like you're in the centre of Northern Ontario, with homes shrouded in vegetation, the tree canopy making it fairly dark on the brightest summer day. All just a short commute from downtown Toronto.
In the pursuit of home improvement I've had to remove and reinstall several toilets over the past year, sometimes redoing the same toilet more than once due to mistakes made on the first outing: all of the preparation in the world sometimes isn't enough if you haven't practically experienced a task before.
I've gotten to enjoy the removal and installation of wax rings, the wonderful varieties of flanges, and the unfortunate results of using too thin of a wax ring when the newly installed flooring raise the toilet's height just a little too far.
Sure I could pay someone to do this, but sometimes I enjoy, err, getting my hands dirty and learning these things myself.
The thing I marvel at is how terribly hackish the interface between toilet and sewage system really is -- you have to align a very heavy toilet perfectly over two held-up-only-by-good-intentions bolts, often back-breakingly leaning over in very tight confines while setting it down just perfectly on the bolts; bolts that are barely contained by a flange that is the end piece of some fixed plumbing, meaning that if you overtighten then you're breaking a pipe that's a massive pain to repair.
It's close to impossible to gauge the difference between not tight enough and too tight with these sorts of things, but it's a costly mistake if you go too far one way or the other (too tight and you have a busted flange or toilet base, but if it's too loose you could have movement that breaks the seal, which causes floor warpage, which breaks the seal more, which causes more warpage, which pushes the unit up breaking the flange): Sort of like the oil change technicians who stripped the oil plug and didn't bother telling me until it popped off, creating an environmental disaster in my driveway, or the auto mechanics that over-cranked the spark plug, stripping the aluminum head.
Of course the toilet interface demonstratably works, given that there are billions of non-leaking, non-broken facilities in operation right now, but technically it is, in my opinion, a complete hackjob of an interface.
In many ways it's like HTML and browsers that will happily adapt to many transgressions of the already loose spec.
Our world has endless numbers of these "there has to be a better way" designs, but because of low expectations, the inertia of the status quo, lots of workarounds, and the training and experience of those who have to deal with it, it's fine.
I bring this up because software is endlessly compared to other disciplines, usually with the software industry cast in a poor light.
"Why can't we build software like we build bridges?", we endlessly hear.
People say this because they have no idea how hacked out (and often over-engineered and costly -- have you looked at the cost of building a little road over a gap these days?) the business of building bridges, or airplanes, or toilets, often is.
It isn't as perfectly engineered as they probably imagine.
Sidenote - How in the world does an entirely ambiguous word like "biweekly" or "bimonthly" continue to be used? Biweekly is sometimes used to indicate every other week, other times used to indicate twice a week. Bimonthly has the same confusion, either meaning every two months, or twice a month.
For years I've argued against ridiculous software patents: While I'm a fervent believer of the innovation fostering environment provided by IP protection and rights, the stream of trivial software patents has reached a torrential pace. We're now at a point where it's impossible to create any software solution or website without infringing upon the trivial "IP" of hundreds of patent holders, leading to the unintended consequence that innovation is suppressed because of the natural litigation risk presented by patent trolls.
One of the more recent abusers of the patent system is none other than Microsoft. They're even patenting methods they're taking from other products now (sure, Microsoft's patent wouldn't hold up during the course of a real lawsuit, but few opponents would have the resources to even bring it that far).
Such a comment inevitably summons the Microsoft-does-no-wrong defenders who repeatedly declare that Microsoft is different, and that these were defensive patents. Microsoft, we are told, is just building a patent shield in case a patent troll comes gunning for them (ignore the absurd foundation of the whole "defensive patent" argument -- patent trolls seldom have any interest in cross-licensing, and they seldom have published products that a target can counter-assault with dubious patents).
When Microsoft's growth curve seemed limitless, they overlooked casual piracy (which is how they gained dominance), had limited invasive copy protection mechanisms, and played nicely in the software development community. As their hegemony has faced real competition, some of it coming from the open source world (given that capitalist competitors were easy to squash), and their growth has been stagnant and has the potential of reversing course in the coming years, Microsoft has changed for the worse. Now they're enlisting jackbooted squads of anti-piracy teams, and infesting their products with activation and "genuine advantage" bugs.
Now they're starting to threaten competitors, and worse the customers of competitors, with their patent trolls.
The risk presented by Microsoft's so-called "defensive" patents is exactly what is happening. They've been offensive patents all along.
Expect it to only get worse.
