Dennis Forbes on Pragmatic Software Development
Subscribe to RSS
 
Thursday, March 08 2007

The Usenet Legacy

A decade+ ago, most "online" comments were conceived and birthed in feature-rich, fat-client applications. These were tools that generally offered a rich gamut of functionality: spell-checking, automatic intelligent threading, offline composition, selective content blocks (such as plonking unliked trolls, censoring expletives), automatic notification of certain keywords or topics, alongside a wide breadth of additional capabilities.

You could read and participate in conversations on a massive array of topics, from law and order, to product support forums for a particular vendor's database product, to the seedier side of the alt hierarchy. All using the same client application that you were comfortable with, configured just the way you liked it.

After authoring your brilliant, convincing argument (or your question about what video card to buy or how to call a certain API function) and hitting send, the application would queue it up much like an outgoing email, and when the opportunity arose (when you dialed up to your local BBS), it would send it to your local server via a standard protocol, where it would be shared with a decentralized universe of servers.

Usually your brilliant literary gem would be immediately visible to the world -- limited only by the rate of propagation -- though a small number of newsgroups had post moderation that requiring each new addition to first be approved.

The Advantage of Standards

This standardized protocol, message format, and distribution mechanism allowed for rich client functionality without reinventing it for every single newsgroup. Imagine how absurd it would have been if you used a different set of tools, with a different set of functions, to interact with comp.sys.ibm.pc.hardware.video than you did with comp.sys.ibm.pc.hardware.sound?

Just as importantly, the standard message format and transport protocol allowed for very easy indexing and archiving -- easily searchable across time and space by whichever search vendor did the best job. This is how we got the incredible functionality of DejaNews (which was later purchased by Google and rebranded as Google Groups), which managed to reach its indexing fingers back a decade earlier than it was even imagined.

If you do software development, you've probably found newsgroups to be by far the most useful resource to search when looking for answers: While a normal web search will yield thousands of noise responses and pay sites begging for money to see the answer (that they usually ripped from a usenet newsgroup), a quick tab over to the groups will usually immediately find the archive of someone who faced a similar question or problem, and the helpful replies.

Of course Usenet is still around and very much alive, and some sites still use NNTP. Unfortunately the quantity of useful answers has been declining, or at least that's my impression, as more and more conversations are being siphoned off into poorly structured, often unindexed islands of information.

Why is every new web app creating yet another terrible reinvention of a container for discussions? Why are we functionally stepping back 20 years for every single new forum (see Digg, YouTube, Reddit, and others for examples of colosally broken discussion systems that people interact with despite their enormous failures, having no alternatives. There are a few, Slashdot for instance, that are moderately evolved, but it took half a decade to achieve a somewhat usable system, and even then the failings are numerous)?

Worse still, why are so many sites storing conversations and threads in isolated silos of data, stored and communicated in completely non-standardized ways. I can easily find and reference threads that I reminisce reading on a usenet newsgroups 14 years ago (usually for "I told you so!" purposes), yet it's often impossible to find a thread or comment on a modern web forum even if I remember seeing it a month ago.

This isn't an argument for a return to the days of yore, and I'm candy-coated the history and usability of Usenet, but it does seem like a lot of people are continually reinventing the wheel, ignoring the lessons of the past. 

It does seem like the value of each additional piece being added to the global solution set is being diminished or completely lost: Where once we had clearly defined domains of information, clearly deliminated and indexed by topic, with a clear threading organization and meta-data structure (author, date, what other comment entry it's a reply to, and so on) that could easily be interpreted by anyone who understood the NNTP spec, now we're at the point where search engines have to try to interpret a million variations of rendering engines, inevitably losing most context and metadata, and that's only if they happen to even crawl across the conversations in the first place.

Somehow we need to find a medium, taking from the past while incorporating modern technology. Perhaps a new embedded commenting structure should be an addition to Firefox 3.

The Ultimate Goal

  • Standard message structure and accessibility for archiving and indexing. Deja News provided an incredible example of the value this brought to the table.
  • Standardized authoring tools and structure - a threaded discussion forum has almost exactly the same needs as every other threaded discussion forum. Users spend so much time authoring comments that it is remarkable that we haven't long had a rich <comment></comment> HTML element as a supersized TEXTAREA, supporting all of the nuances and features shared by virtually every conversation site.

Reader Comments

Down the line, I think that http://microformats.org/ will be able to address some of these concerns. I don't see one specifically for comments/messages/discussion, though.
Leo Petr @ 3/9/2007 7:53:58 AM
Great Post.
I was wondering the same myself.
With all the different types of forums it makes everything harder to track and find, even using RSS feeds for forums doesn't perform as the "OLD" standard NNTP apps had.
I sure hope a standard will emerge.
Maayan Porat @ 3/9/2007 12:37:15 PM
Interesting take on this. And I completely understand your point. I'd never thought of it that way.
I'd always seen it as the opposite. And perhaps most "ordinary users" do too. I did use Usenet way back in the early days of the Web (94, 95); but I found it confusing having a separate application (a "newsreader") that talked a completely different protocol (NNTP) to access a completely different concept (a "newsgroup").
With online groups, it's just another web page. You use the same application ("web browser"), same protocol (http) to access the same concept ("web pages") as you do for everything else. So from one direction it looks like a lack of standards, but from another direction it is really a settling on one standard (http, HTML and web browsers for everything).
You have a point about search. In practise I don't think it makes that much difference. Often it's best to find one good forum that concentrates on the topic at hand, and go from there.
Actually, I remember the reason I stopped using Usenet. Most of it was unmoderated garbage! Because online forums were privately hosted and usually strictly moderated, there was a lot less spam and less retarded flamewars than on Usenet.
Andreas Mross @ 3/15/2007 5:47:43 PM

Add Comment

Name *:

Email Address:

(your email address is not displayed)
Website:

Comment *:

Earlier EntriesLater Entries

Dennis Forbes - Dennis Forbes is a Toronto-based software architect and technology writer