If you're running Firefox 3.5 or Safari 4, take a look at the "benchmark"/technology demo I just put up. [Safari 4 compatibility added based upon the great comment submitted by Oliver]

It's a modified variant of the SunSpider benchmark that I've written about before (in less than flattering terms), which I heavily altered to utilize the remarkable new Web Worker functionality you can now explore in Firefox 3.5. If you're really analyzing performance, be sure to disable Firebug as it significantly impacts the results.

Web Workers, a standardization of a feature of Google Gears, are a remarkably simple method of multi-threading JavaScript, not just to get it out of the UI thread — where it can be very detrimental to the user experience as the interface freezes while a script runs — but also to scale across multiple CPUs and cores on modern PCs, which while seemingly a ridiculous notion ("but it's just JavaScript! Multithreading?") is becoming a real concern as the JavaScript engines continue to advance and the usage and scope of the language and related technologies continue to expand.

Through a simple, synchronized message passing system and a minimalist API, the Web Workers model lends itself to robust, elegant code that isn't as prone to classic multi-threading pitfalls. While not a part of the current instantiations, in a theoretical implementation there is no reason why web workers couldn't be located on entirely different machines, given that each worker is essentially an isolated runtime, sharing very little (the navigator properties and some basic security info for things like enforcing XmlHttp restrictions), communicating via serialized messages.

Understanding the Benchmark

The benchmark/technology demo is operational in Chrome, Opera, and Internet Explorer, but only if you change Web Workers to 0. In that case it is sequentially running the set of tests in the main thread, as JavaScript has traditionally been run. I didn't intend for this to be used for cross-browser comparisons, even if I resort to presenting just such a comparison at the end of this entry, and instead the focus is really on the technology, so the real "power" is seen once you start to turn up the web worker dial, all the way to 11.

Web worker multithreading isn't limited to Firefox 3.5. Oliver left a comment pointing to a Safari-ready variant he threw up, so I modified the test accordingly (the difference being that when Safari implemented it, it didn't intrinsically include JSON encoding, so your caller and receiver had to do that themselves). I didn't realize that Safari had covered this ground, though it isn't shocking given how rapidly that browser has been advancing.

With one web worker, the UI remains fully responsive to user interaction, which is an experience quite unlike what was seen at 0 (where the browser essentially locks up during the run), and the actual run itself suffers little for the isolation. On a quad-core CPU, the CPU usage during the duration of the test cycle fluctuates around approximately 25%.

At two web workers, the individual tests take slightly longer to run, however the actual completion and pace of the tests in the whole is greatly improved. Not quite a halving of the runtime, but not too far off. Two cores are saturated during the duration of the test.

At three web workers, three of the cores are filled with work, and the total elapsed time improves somewhat, albeit not by the ratio that correlates with the 50% increase in computation power.

At four web workers, we've tapped out the parallelism and despite all four cores being saturated for most of the duration, the total runtime actually suffers slightly. Going above four doesn't cost much, but it also brings no real gain (beyond possible algorithm gain isolated various parts of the application).

You can also run a mode where instead of running a modified js directly in the worker thread, the code is passed as a string parameter, eval'd into a function reference, and the function is run. There are some interesting observations to be observed by this test, such as the lack of tracemonkey loop optimizations on eval'd code (see bitwise-and in particular. It suffers dramatically when run as an eval'd function relative to running as literal JavaScript). This surprised me as the eval merely instantiates a function in the current context, but doesn't run it, yet the performance penalty remains because it was sourced from an eval.

Here are some results for 1-8 threads, running 10 cycles of each test, gathering the total elapsed time in Safari 4 and Firefox 3.5 RC2. This was run on a quad-core Q9400 machine, and of course your mileage will vary. While it is evident that Firefox 3.5 is using more of the available processing power as you move past 1 thread, with it increasing from 25%, 50%, 75%, to 100% at 1, 2, 3, and 4 threads respectively, it doesn't fully benefit from the additional resources, yielding a greatly diminished rate of return. Safari, on the other hand, already started with a considerable lead, and it pulled away with each thread up to the optimal 4, really hitting its stride.

I'll add some charts and the like to this entry later, but just thought I'd drop a line on that demo of a very promising technology that will soon see fairly robust deploymet (one huge benefit of Firefox -- shared by Chrome and Opera -- is that the uptake rate for new versions is extremely high).

Firefox 3.5 – You Should be Trying It

Have you tried out Firefox 3.5 yet? It’ll happily install side-by-side, so there’s very little risk giving it a spin.

While currently a Release Candidate, it is already supported by most popular add-ins, such as Firebug and Adblock Plus, and brings a bounty of new functionality, more comprehensive emerging standards support, all while improving performance.

Performance has never been Firefox’s strong suit. While some incorrectly believe that Firefox has just recently been overtaken by upstarts like Webkit and its bastard child Chrome – the lore being that Firefox’s creaky “old” legacy code is not as nimble as the newer projects – the truth is that Firefox has always been a performance laggard compared to all but Internet Explorer.

It has always used more memory, done things more slowly, with benchmarking and resource consumption metrics that seldom earned it praise. The history of Firefox (aka Mozilla Suite, Phoenix and Firebird) is a tortured legacy littered with complaints about slow start-up times, a slow page layout engine, massive memory consumption, and middling JavaScript performance.

These are the costs of building on the XUL abstraction, along with some expensive add-on scaffolding. The ability for an add-on to dramatically change the behaviour of the browser (all from within a world of JavaScript and chrome) isn’t just an accidental happenchance, but instead is the result of considered, pervasive design choices.

The EU Fines Opera for Their Monopolistic Browser Stranglehold

If a small footprint and snappy response were the keys to success, Opera would have taken the market by storm given that it’s been evangelizing and focusing on that pursuit, to a great degree of technical success and leadership, for well over a decade now. Yet most users care more about a slightly better bookmark manager than they do about even magnitude differences in JavaScript performance.

But performance does matter. As web applications grow richer and more elaborate, the weakest of the contenders will get pushed out of the race. Already web applications like Slashdot and Facebook are getting unpleasant with a few of the current contenders, yet some of the new features of standards like HTML 5 promise dramatic new functionality that will completely blow the top off of the domain restrictions of the browser world.

So with recent releases the Firefox crew has reduced the toll you have to pay for these architectural choices, and through a lot of hard work it’s getting to be a memory lightweight, as far as web browsers go, and performance has improved across the board, sometimes dramatically.

Your Browser Is a Remarkably Complex Software Product

And to sidetrack for a moment about a personal peeve, let me kick down the frequent whiners’ complaint that it’s “just a web browser”. Every time someone make such an ignorant, misinformed statement -- usually during some entitled complaint about resource consumption on their obsolete PC -- I think a universe blinks out of existence in some alternate reality.

There is very likely nothing that you do on your PC that is as rich and complex as the things that you’re doing right now just browsing the web. It’s isn’t “just a web browser”, it should be “I can’t believe how much is happening for me to loll around on the web for a bit!”.

Taking a couple of paragraphs to reminisce, I signed up for a Slashdot account not long after the site opened for business. Following my normal tradition at the time, I suffixed my newly created account with the two-digit year of creation, in that case 98. It was an easy way to choose account names that were less likely to be rejected as already in use.

I didn't imagine that I'd be using the same account over a decade later, so now I have a nick that makes me look like I’m pretending to be an eleven year old (if you ignore the 4-digit user ID, it is conceivable that an eleven year old might visit Slashdot).

I didn’t join the ranks of Slashdot to cheerlead Linux and open source, despite it being the overwhelming direction of the site at the time.

My intentions were much less idealistic.

I actually signed up to troll a co-worker: a Linux-loving long-haired hippy named Warren Postma. The guy was always going on about the greatness of open source software and the related initiatives, about the evils of Microsoft and closed-source, and so on, and of course he was an early fan of Slashdot.

[Over time I came to appreciate that much of what he had to say was right on the money, and in reality he was one of the best developers I’ve had the pleasure of working with.

And he wasn’t really long-haired or a hippy, though maybe my perception imagined him that way at the time, what with all the free-software rhetoric]

That was an era when being up on the latest meme was a lot more difficult, so a site like Slashdot was a welcome source, allowing you to knowingly declare that you'd already seen the dancing baby when a less-connected co-worker gets it emailed to them and comes over acting all connected-like. Though to be fair Slashdot came a little late to give a heads up of the dancing baby craze, but it was there for All Your Base Natalie Portman hot grits, though along the way to enlightment came some less appealing memes involving tubs and goats.

Slashdot no longer fills the industry-dominating position that it once did (where being "Slashdotted" was a term used in the mainstream press) — I suspect it has more to do with its competitors pandering to the common, all degrading eventually to funny pictures, "facepalms" and non-stop political posturing and claims of oppression, with sites like Digg and later Reddit escaping from the technology ghetto, while Slashdot remained chained to its original niche — but it often still has interesting and elucidating discussions throughout its comments, even if the UI of the site continues to grow worse with every passing month. Seriously, guys, the endless page thing was demonstrastably lame on DZone, so why did you think it was a good idea?

So during a moment of mental relaxation yesterday I thought I’d see what was up in the world of /., where the top story at the time detailed a congressman’s bill to “Ban ISP caps”. I found the submission hard to believe, so I RTFA to find that it had been very poorly interpreted.

What the congressman really proposed is more of a fee oversight body similar to those that many utilities have to work within.

Here in Canada the monopoly phone companies, for instance, have to apply to the CRTC every time they want to change fees (whether to increase or decrease) or service levels, justifying why they need to make those changes. Of course they’ve always been grotesquely profitable, and seldom get denied their desires (except, humorously, when it came to reducing fees to compete better against new upstarts), but just pretend that such bureaucracies make them more competitive or accountable or economical or something.

So I posted a comment saying that the summary was inaccurate. I also tossed in a little aside about utilities, drawing from the congressman and the submission’s analogy comparing ISPs with utilities, speculating that soon ISPs might just switch to a pure consumption model more analogous with the utilities that they are being compared to, saying "I don't get carte blanche from the electric company to use it all for free, complaining that "they provide 20A to the house so I should be able to use 20A around the clock for free!"."

I touched a sore spot, which I have to confess to doing intentionally. Sometimes I can’t help but troll a bit, and given that it’s a pretty common position on sites like Slashdot that it is a basic human right to torrent a pipe full around the clock, I thought I’d pick that scab a bit.

I got the typical “the bits are free and the man is just keeping us down.” variety of responses, including my favourite.

Anyone who is comparing ISPs and bandwidth to other utilities such as electricity, water, or anything tangible should simply stop posting… immediately. The Internet is not a series of tubes, your comparisons are invalid, and you are ignorant.

Please cease in the proliferation of these ludicrous analogies.

Ignoring that the utility analogy came directly from the congressman himself, this comment has made me personally invested in this spurious analogy, so now I must embrace it with vigour.

And, for that matter, the internet is effectively a series of tubes, as fun as it might have been to point and laugh at Mr. Stevens. Those tubes have a finite capacity, and if you want to push more through, you have to lay more or “wider” tubes.

Conceptually it isn’t all that different from a power or water network, where many of the same circuit principals apply.

And those “tubes” didn’t magically appear from nowhere. They aren’t public infrastructure. The internet isn’t free. Bandwidth isn’t free. Throughput isn’t free. All of it costs money to lay and service, and as demands grow, there is the capital required to increase capacity.

So let’s go back to the electricity analogy (even though I had no intention to draw it out so literally, I’m invested now).

Back to the responses again, a less stupid one said:

The total amount of power you use in a month directly affects the amount of fuel a power utility has to burn, or the amount of water you consumer affects how much water the utility has to treat. Bits on a connection aren't like that. If you don't use a bit on their fibre link to the backbone, that doesn't leave them with an extra bit, and if you use a bit, the next one is coming at the same time and same cost anyway.

Fair point. When you use electricity, the variable costs increase. When you use water they have to buy a little more chlorine and pay the electric company to run the pumps.

Yet here in Ontario most of my electricity comes from either hydro-electric (our power company was historically called “Ontario Hydro”, causing confusion for decades as children tried to understand why the power company was effectively named Ontario Water), or nuclear.

The variable costs with either are a small percentage of the total cost to operate those services, and instead the vast majority of the cost comes in the form of capital expenditure – what they had to build and maintain to handle the current load, and to handle tomorrow’s load.

Most of the “conservation” efforts – incremental and accelerated fees for heavy consumers, and rebates for power saving appliances and technologies — have been aimed at avoiding exceeding the gross capacity of the system right now, because the next step will cost billions, and it’s not to buy a few more fuel rods.

Nuclear power plants and giant dam projects don’t come cheap; nor does building a water purification system to service a city of 200,000. It’s one of those perverse situations where if demand dropped significantly, they’d actually have to increase costs.

To wring every last bit of ridiculousness out of this analogy, imagine that you live in a village of 100 people and @Tesla just twittered about a cool new tech called electricity.

Hyped about the latest fad, you get a bank loan and put up a wind generator, connecting it to anyone willing to pay an even split of the costs (a loan you amortized to get paid off just as the turbine needs replacement, plus basic maintenance and upkeep).

Your new turbine is spinning around the clock (it’s a very windy valley near the ocean), putting out a constant 100kW, and everyone is enjoying their 1kW space heater, all paying their 1/100^th cost.

Every now and then someone turns on a hairdryer, but it balances out because other people sometimes turn off their space heater, and in the end the load can be handled due to its natural distribution.

But then someone decides that they’d also like to run their 5kW back massager around the clock – you did say that electricity could be used around the clock, and the wind is free, right? — so they crank it up and your system starts to suffer brownouts and everyone suffers.

You face some tough choices: You can either put a 1kW limit on everyone (“dedicated” capacity), which would be unfortunate given that most of your customers benefit from sporadically spiking their load while they use their motorized hair curler, or you can start charging or limiting only the outliers, maybe targeting the growing trend of back massagers specifically, putting limiters on that specific use of your power.

Or you can add capacity.

So you decide to do the latter in the most cost-effective way, arranging a power sharing agreement with a neighbouring village: When you have left over capacity you send it to them, and they do the same in return.

It all balances out, and everything is fine in the world again, and that one customer gets to enjoy his back massager, essentially freeloading on the standard customer load.

But then another user decides that he, too, might as well get a back massager. And then another. And then another.

The neighbouring village starts rethinking the power sharing agreement because it is no longer serving both partners equally. They demand either payment for their inconvenience, or they’re going to sever the line.

So now, despite the fact that there is “no cost” for each incremental kW, the village is faced with a massive capital (or loan servicing) cost of putting up another wind turbine.

This is grossly simplified, as analogies often are (and there are countless similar scenarios, from toll roads to bridges to movie rentals…this is like beginner economics, but so many are so blissfully ignorant of it), but there are a lot of parallels with internet backbones, where end-users are getting 10Mbps connections to a grossly oversold network at a heavily discounted price. Any delusion that there’s some 10Mbps dedicated channel available for every subscriber is absurd. Go and buy a commercial dedicated T1 at 1/6^th the speed and see how the pricing stacks up.

Consider that my little suburban town has about 50,000 high speed customers, all with 10Mbps service. If we all used our throughput, this single town would be a 400Gbps load on the network. Try laying a 400Gbps city wide network, much less province wide, much less country wide, much less continent wide, much less…that is an enormously expensive proposition.

This is the reality of the internet today. Maybe the backbones should be a public bit of infrastructure, with massive, limitless pipes from coast to coast, but that isn’t currently the case. If people want to argue for that – I think a next generation, coast-to-coast Canada backbone would be a great project for some of those stimulus billions — then they should argue for that.

Things are getting better. There are billions of dollars constantly being spent improving the backbone of the net (YouTube and Hulu today alone today account for more traffic than the entire backbone could handle in 2000), and I’m sure in a few years we’ll all being using our 1Gbps connections to play a live-streamed 1080p never-obsolete video game system.

I have no desire to defend ISPs, or to support some of the various tactics they have pursued (traffic shaping, for instance, is not at all justifiable, nor should a carrier care or restrict which sites you visit, trying to monetize the fact that you visit Hulu, or trying to extort Google. Restrict or charge based on the throughput, not specific uses of it that should be none of your concern), but some of the rhetoric on this topic is just so absurdly ridiculous, and can only be drawn from the delusions of self-serving fantasy.

The denial of the reality of overselling and fair use is how we end up with traffic shaping, or stupid constraints like cellular carriers refusing to allow uses like tethering (which is because they rely upon most of their customers not using their "alloted" throughput, and tethering would completely throw off that assumption. I'd rather they base all of it around actual usage and cease trying to indirectly massage usage through unnecessary secondary restrictions and throttles.

The standard tripe about how one’s usage somehow only ever uses excess capacity (is there some “only if available” NoQuality-of-Service bit?), where people assume that their actions somehow exist in a vacuum of inconsequence, unbound by the limits or constraints of anyone else, offends common sense.

Eight year old Victoria Stafford went missing from the nearby town of Woodstock on April 8th. She had left her school at the end of the day, starting the short walk home alone, but purportedly never made it.

Video from a high school up the street (see the enhanced version as well), discovered the next day, shows Victoria walking with a thus far unidentified female (a sketch, purportedly based upon a witness account, has been released. Given the sudden appearance of this supposed witness account two weeks later, it seems to be an attempt by the police to put a little pressure on a suspect to see if they break or make a panicky misstep).

The town in question, Woodstock, Ontario, sits on a major highway that goes from Windsor to Toronto and Montreal and beyond, with a nearby branch going to the Buffalo area. It seems noteworthy that Victoria’s public school is a very short distance from an onramp.

I find this case distressing. These things generally don’t turn out well.

Whether the abduction is real or not (there’s a general cynicism about cases like this because of vile, murderous sociopaths creating a cry wolf situation, leading many to automatically disbelieve), it usually forebodes very bad things.

Of all of the video cameras that blanket our society, the best they’ve got — at least that they’ve publicly announced — is the single grainy video from the high school (see the graphic I made of approximately the zone covered by this video, as determined by correlating landmarks with satellite imagery.) There were lots of people around, but history has shown time and time again that people are really, really terrible witnesses of anything, and that seemed to have held true in this case.

Who is the woman in the video? Apparently no one knows.

This has me thinking about pervasive, distributed monitoring. Where in time of need – like an incident like this (yes, “think of the children”) – swarm media and electronic capture can be combined to zero in on the truth. Preferrably in a way that utilizes the enormous talent and load distribution of the public.

High definition video capture is becoming dirt cheap. Solid state storage is rapidly evolving.

I’m not talking about the Big Brother 1984-style of central government monitoring, with the endless pitfalls and abuses that entails, but rather a situation where almost incidentally most everything is recorded by the public using a distributed array of devices.

It seems inevitable, for instance, that in just a few years every car on the road will have forward and rear facing cameras. The former is already in place on most police vehicles (helping to keep police in line as much as to capture public malfeasance), and the latter is making inroads on large vehicles that are prone to backing over people and things.

Video retention will inevitably come next, under the auspices of road safety (similar to how your car is ready and willing to rat on you for speeding if you get in an accident). Soon it won’t be an option, but will be a legal requirement for using the roads.

“Sorry, bub, but your car’s video system clearly shows you blowing through that red light. The other guy’s video system shows that he was in the right.”

99.999% of the time the video loops over and is erased and inconsequential and irrelevant, but every now and then it serves an important purpose in getting to the truth.

Imagine, for instance, that they could put out a call for anyone who drove down that street or that neighbourhood in the time period in question (presuming they don't already know from telemetrics via systems like OnStar, again soon to be the legal norm as every road becomes a toll road with vehicle self-reporting), from which they got a number of different time and position videos.

Videos of vehicles parked in the vicinity. Of the woman in the white coat waiting.

Add the video capture on private buildings of all sorts (homes, businesses, parking lots, etc). Merge it all together into a exhaustively documented, fact-based accounting of what happened.

Cellphones of course play a part as well.

Someone premeditating a crime would likely leave their cell phone at home (though I would wager that the woman in the white coat probably had one on her), or disable it in some way, but it would nonetheless allow the net of discovery to close in on the truth. The lack of certain data is often indicative.

Say to contact a guy in the area (via text message of course) and find out that he was in the neighbourhood taking pictures of doors, and wouldn’t you know it but he happened to have one that has a suspect matching the description walking towards the crime scene 9 minutes before the crime.

That’s if you even need to contact him. Maybe he uploaded his GPS-and-time-tagged photos to Flickr, and a police investigation entails a photo search, again drawing from the enormous distributed capture that happens every day.

It is neither a utopia nor a dystopia, but I think pervasive, distributed, decentralized recording and archiving will be a good thing for society.

Should we bother paying out big dollars for arrays of magnetic drives, or should we push the envelope and go with an array of SSDs? Should we wait a while? Will our vendors and the storage systems support this technology? Will existing products make optimal use of it? Will the SSDs burn out under our usage models?

Dell, for instance, still has nary a mention of SSDs in their servers and storage products site. Their reps still telling you that SSDs are unsupported.

Yet the evidence is obvious that in the year ahead SSDs are going to absolutely annihilate the existing field of storage vendors. Suddenly "outsiders" like Intel (now really known for storage products) and Fusion-Io are the leaders, making the existing market look like a bunch of chumps. Paying big dollars for a large array of magnetic drives seems like a choice that will certainly yield some serious buyer's regret a few months down the line.

SSDs change everything.

In a similar way, the extraordinary advances in JavaScript over the past year have completely changed the scope of what a "web application" could entail, and we as developers still haven't fully come to realize what this means. Opera, Webkit, Tracemonkey, and now, jumping to a big lead, the supercharged V8 engine of Chrome 2 (I have some serious misgivings about Google's browser given that it's the product of an ad company, but it is uncontestably becoming a real contender. I will warn that on the Chrome download page linked before this parenthetical aside, they put the agreement to send...cough...anonymous statistics in exactly the position where people have habitually learned to click to agree to the ToS), the advances have been truly spectacular.

Exciting times ahead. These innovations aren't simply evolutionary, but change the scope and rules of the game.

I have my work machines. My wife has her laptop. My kids have their PCs.

Family and friends that comes to visit connect their iPhones and laptops to our 802.11g/n.

We have shared media that we play on devices like the Xbox 360 and PCs over protocols like uPnP, DAAP and SMB. We have shared files, such as financial info, that we need to access from any machine, all while confident that they remain as secure as we want them to be.

We do VPNing from remote locations like coffee shops to access files and media. We have a shared printer.

To optimize this situation without leaving my power guzzling pig of a PC on around the clock (I’ve setup and torn down a number of home servers over the years, all the ways back to a FreeBSD-based device in the mid-90s that also doubled as a firewall), a while back I bought a Synology DS-106j NAS unit and dropped it on the network, deciding on it after looking at some Windows Home Server products, finding them overpriced and uncompelling (not least because of concern about licensing and DRM BS.)

One of the selling points of this ~$200 unit (to which you add whatever 3.5” hard drive fits your fancy) was the gigabit network port. To me that implied that it must be capable of at least keeping a 100Mbps pipe saturated with goodness.

I also liked that it gently sipped power (13W or so with the hard drive), allaying any guilty about leaving it on around the clock.

The Synology unit is fantastic in many ways.

It has a fantastic interface. It is rock solid (zero downtime with it) and is very feature rich, with several free “firmware” updates since I purchased it adding a number of new features and interface improvements. It is easy to setup and use, and to a superficial degree secure. It comes with great desktop software for common tasks like backing up (and versioning) files on your Windows machines. You can offload torrent downloading to it if you’re into that sort of thing.

But the hardware seems to be underpowered. The 200Mhz MPC8241 CPU that powers it can’t deliver on the extensive feature list, in my opinion.

The 1Gbps network port turned out to be brave talk that it couldn’t back up with action. Normal SMB/CIFS (the protocol that Windows utilizes to access network file shares) averaged about 2.5MB/second throughput with the device, which factoring in overhead is around 22Mbps, leaving plenty of headroom before it hit the limits of even a lowly 100Mbps network.

I got it up to 8MB/second or so by turning off all of the features like media sharing, and then disabling all security on the share and enabling guest access – it seems that access control security significantly adds to the computational burden – but that isn’t a viable day-to-day option so I had to revert and it was back to 2.5MB/second. FTP was a little better, but was still way below expectations given that the same hard drive got a rough 40MB/second from a direct-attached USB 2 enclosure.

Given that I am now downloading from the web at large at 1.1MB/second, this seemed silly. As we’ve started to amass monstrous AVCHD video files of a GB or more from the home video camera (the latest GB or so features my youngest son herding some chickens that had escaped from their pen at a local farmhouse), it was getting unsustainable.

Synology is now selling a purportedly faster unit, but given that I saw much lower real-world speeds than what they graph for the old unit, and their new unit really doesn’t do that much better anyways, it didn’t compel me to upgrade. Many competitive low end home NAS devices have reportedly similar performance in the wild.

And it isn’t just waiting while moving large files or having SyncBack jobs run. Navigating the music directory from a uPnP client made it obvious that the slowness impacted all activities, with basic operations having intolerable delays. Attaching to its network shares from client PCs inexplicably had multi-second waits before responding.

Then again, I’m the guy who thinks that the biggest crisis in the electronics world is the excessive lag before DVD players will eject a disc, so maybe I’m an outlier, but it seemed like a pretty big weakness of what would otherwise be a great product.

It really bothers me when units doing tens or hundreds or thousands of millions of operations per second take perversely long times to do simple things.

On the bright side, the small dimensions of the enclosure made it easy to find it a home, but it also had the S.M.A.R.T. monitoring of the hard drive complaining about borderline heat incessantly. And the printer sharing never worked properly, but given the endless variety of printers I won’t blame that on the NAS unit.

So I decided that it’s time to replace the device, with the following criteria for its successor.

• Relatively cheap, because I’ll probably just be replacing it soon enough anyways
• CIFS and FTP at a minimum
  
• uPnP would be nice. DLNA wouldn’t be bad. DAAP would be gravy
  
• Some method to backup its own files to a remote location, or at worst to an attached USB drive, with security options like encryption
• Good performance. On a switched 1Gbps network I would hope to see read throughput times of 20MB/second or better, and writes not much worse, with a very low latency on activities: Network shares should enumerate close to instantly
• Small enough that it can be physically secured such that the alarm responders would have arrived before a thief could have gotten to and removed it
• Low enough power usage that I don’t have squirrels giving me a beatdown
• A bit of fun. I look for these things to be educational and challenging to a degree. The easiest solution doesn’t provide me the entertainment value, so turnkey is a negative. At the same time, I have a very finite amount of time to deal with things like this, so it can’t require too much coddling

Local RAID isn’t important to me, as availability isn’t a primary consideration (and multiple drives go contrary to power conservation / heat reduction goals, and generally increase maintenance.) Every file that is on the unit always exists on at least one other drive on a separate system, so if the drive died I could quickly rebuild and repopulate and life would be grand again. And it is vastly more likely that two drives in one unit would catastrophically fail than it is that two drives in physically separated devices would fail, especially when the separation between can be thousands of kilometres.

Nor do I want this to function as a media station, which I consider an entirely different function, with an entirely different requirements list. I don’t need or want HDMI output, blu-ray decoding computational power, 3D offloading, a MAME box, or anything of that sort. That’s a different project, with a different set of goals.

After looking at the available market options of targeted NAS devices, and hardware solutions like Shuttle computers, I decided to have a little fun and build my own (in the minimalist stick-some-lego-like pieces together way that is "building" a PC), so I grabbed a couple of components.

• $102.30 - An Intel D945GCLF2 mini-ITX motherboard, featuring an on-board passive cooled Atom 330 dual-core processor. The Atom is not a powerhouse processor by any measure of the imagination, but for this purpose it should be fine.
  
  My only real hesitation with this board is that it uses a relatively power hungry chipset, leading to the mixed-up situation where the chipset has an active fan, yet the CPU has just a heatsink. I’d prefer a lower power board with zero moving parts, but didn’t want to risk going too low on processing power  
• $24.75 - A 2GB DDR2-667 RAM module. Even 2GB is excessive for the targeted purpose
• $55.22 - An Apex MI-008 MINI-ITX Case. The case is small enough that it’s stashable, but big enough that it won’t be a hard drive cooker. Like the motherboard, the case isn't a paragon of efficiency, instead coming with a less efficient power supply that is capable of a grossly excessive 250W, so that isn’t optimal, but I’ll worry about that at a later point
• $126.90 – Western Digital WD10EADS Caviar Green 1TB drive. It is intentionally a bit slower of a drive, putting energy efficiency (and with that reduced heat) ahead of raw speed, but it’s still very fast. This would be a great place to use an SSD drive, so maybe I'll swap to one of those as they mature and become more cost effective for this amount of storage

<$200 for the “NAS enclosure” part, then $127 for a good hard drive with room to grow. Pretty inexpensive, and would probably even make a decent internet appliance, which presumably is the intended use of the motherboard/CPU combo.

Putting it together was a snap, with dutiful oversight from my children, followed by a quick flash to the latest BIOS release.

I installed Ubuntu 8.10 Server from a USB stick (I considered installing FreeNAS, but decided that I wanted something a bit more custom), after which I discovered that the hard drive wasn’t set to bootable and the system wouldn’t boot without the USB key, so I had to boot with the key and use parted to manually set the sda device to the boot device, after which the USB key was no longer necessary.

Installation complete, I stashed it away, sans keyboard or mouse or monitor connection (which it makes no complaints about, happily existing without head, keys or mouse), and access since has been via putty. A magical source and destination of media and files, out of sight and hopefully usually out of mind.

I installed Samba (during the OS install by choosing the file server option), created the users and groups and perms, added some shares, and started accessing it from my workstation. It was all very easy and straightforward, including even setting up AppArmor to enforce mode against the Samba services, adding a small additional guarantee of security.

I started the first file move, ready to witness some low-cost awesomeness.

11MB / second to write to it, averaged out over the copy of a GB sized video file. I got about the same speed reading back from it.

Pretty good compared to 2.5MB/second (and now I had much better security granularity than I had before, on a much more versatile platform), but not quite what I was expecting. Awesomeness not witnessed.

The hard drive isn’t the fastest of the bunch, but it’s still very decent.

I’m far from an expert when it comes to Linux (or more generally Unix) systems, seemingly achieving some small level of localized expertise at intervals in my career, but then the solutions simply work and I don’t have to touch them again, so the knowledge rusts and each time I return I feel like I'm a Linux virgin (who would have ever thought that Linux and virgin would used in the same sentence?) It is similar to my proficiencies with Perl or the equally obtuse Powershell, where I put together very decent solutions, but then several months later look in disbelief that I actually wrote the code that now looks foreign.

Nonetheless, I quickly determined that the network adapter was auto-negotiating itself incorrectly to 100Mbps, and 11MB/second is about the max you could achieve over such a link.

After a quick bit of searching, I added ethtool -s eth0 speed 1000 duplex full autoneg off to my interfaces file (after validating that it worked at the command line). It was now at full 1Gbps glory.

So I performed the file test again.

47MB / second to write to it. 55MB / second to read back from it.

Sweet! That isn’t that far off of local storage, and is entirely with the acceptable zone. It was achieved on a $200 host device. File sharing is hardly the most demanding application nowadays, but I was happy to get something much more usable.

Just to take a side trail here for a moment, I just wanted to mention that Linux, and the evolution and progress of it and related projects, really is amazing. It is astounding that such a solution evolved the way it did. Even people who don’t think they benefit from it regularly use devices run on the OS. The amount of functionality and technology available, including the source so you can roll your own, truly is remarkable. Linux didn’t start this movement, and projects like FreeBSD, sendmail and prior initiatives blazed a successful trail before Linux really took off, but it is the most evident element of that development model.

Back to setting up the homebrew NAS, virtually every step of the way I had to consult the wizard, Mr. Google, but at least the answer was usually only a page 1 or 2 hit away (though Google Groups has taken a perilous dive into close-to-uselessness, and proved of little use.)

To this point I still haven’t had luck finding a good uPnP server for the device. uShare simply didn’t work. Mediatomb inexplicably doesn’t work with the xbox360, and even if it did the fact that its anonymous web user interface has the default behaviour of exposing your entire file system makes me question the wisdom of the developers.

Various other projects didn’t fit the bill or didn’t work for me, and most seemed to have been abandoned.

I thought I’d try the MythTV backend, as it is purported to double as an uPnP AV mediaserver, to find that it bizarrely needs an X server running and an X client on the other end just to perform the setup. I decided to purge it from the system to find it wouldn’t uninstall due to /dev/.static/dev being mounted read only. I found a workaround involving stopping and starting udev, which remounted it as read/write, but at that point the damage was apparently done, and it refused to gracefully leave the system.

So I went MythTV huntin’, committing a moronic error in the process (it tried hard to warn me, but I persisted in my foolishness.) I’ll let the putty log speak to it.

Installing everything once was so much fun that I decided to do it again!

So I reinstalled again. The second time around it was a very quick process (even with a nuked /usr/bin I still had a chance to backup customized config files to a client box before reinstalling, so I really didn’t lose much.)

Power wise, I’ve gone from ~13W for the DS-106j, to 35W to 40W for the new device (measured with a Kill-A-Watt. Handy device.) This represents a pretty big move in the wrong direction, but it wasn’t unexpected. 4-8W or so could likely be saved going to a more efficient, fanless power supply. That’ll be a future experiment. The Caviar Green, at 3-4W, isn’t much more power hungry than an SSD, so that wouldn’t make a huge difference. The motherboard and its chipset is the real piggy of this farmyard.

Alas, in return for the extra power consumption I now have a vastly more interesting network device, serving files at a much more usable pace.

So my first adventure is beginning the work on a .NET-based uPnP server, ensuring that it works on Mono along the way. Thus far it has been nothing but remarkable success, and it still astounds me seeing fairly complex .NET applications running well on a Linux box. The thing really, really works!

Still, you owe it to yourself to read the article about his exploits in January's Inc. It is a fascinating story of internet success against the odds, and the site that is serving up 1.6 billion pages per month on an inexpensive modicum of hardware. That article references Markus' blog entry from 2006 where he explains how his extraordinary success story began.

[Imagine that I insert some drawn-out blowhard "lesson" to be learned from Mr. Frind's success here, allowing me to justify making an entry that is basically nothing more than a link, all while pretending that if you follow these simple steps you too can achieve the same results]

It's always a danger — in the nerdly get-some-unkind-emails way — to question it. It has quite an army of loyal fans who, I presume, have had their ego carefully stroked over the years into loyal defensiveness ("Yes you are a top notch programmer! Yes you are!"): Any prior time I've disagreed with Jeff on here has resulted in a flurry of emails that are the text equivalent of the infamous Chris Crocker video.

Yet Jeff's latest entry has me unable to contain myself.

In that post, Jeff opines that Windows 7 might just be worth a look because, he says, in the latest outing Microsoft has taken to changing the visible parts of the OS, instead of, I guess, just improving the underlying awesomeness. The example Jeff draws from is that the calculator has visually changed, whereas before it was just the underlying mechanics of calculation that saw awe-inspiring improvements, all while gaggles of ungrateful goons continued to hurl insults at Microsoft, unaware of the great gift they had been handed. That Microsoft has decided to enlighten us to the great improvements they've made by visually making change apparent, instead of just doing their magic in secret.

As a newsgroup troll might say, errrrr...wut?!?!?!

What planet has Jeff been living on? What spaceship did he just hop off of, interstellar cruise of the Outer Gamagia quadrant completed, that leads him to be so completely out of touch with reality?

Here on planet Earth, Vista was seen as largely being about changing the UI — much like XP before it — and many of the complaints were that the actual utility of the OS suffered (even basic operations like moving files seemed to have missed being QA'd, slowing to a paralyzing crawl under completely ordinary uses). Functionality got lost under layers of paint, and interfaces seemed to be changed for the sake of change.

To many, Vista was 99% visual changes and 1% detrimental functional changes. But at least it brought the unwashed masses a calc.exe that had shaded buttons and a translucent title bar!

Conversely, a lot of the excitement about Windows 7, relative to Vista, is that it fixes stuff "under the hood" (better, strong, faster.)

But I'm no Vista basher, and actually believe that much of the anti-Vista vitriol is undeserved and unfounded. While I was on the record saying that it would be a product failure because it was wrongly focused and had little that compelled people to desire it, I'm actually somewhat of a fan of the OS, insofar as the comparison is with XP. Vista even has some very cool features under the hood, such as TxF, though that's the sort of structural change that isn't really useful until applications start using it, but they won't use it until it is available in a good percentage of deployed PCs.

Back to Jeff's entry, the ridiculous example of the accessory calculator being an example of...anything...really strikes me as absurd, and it seems to be the sort of "try to draw some big observation from some small example" space filler you end up resorting to when you're trying to hit a schedule.

To add to the march of absurdity, Jeff links to a ridiculous post by the occasionally interesting Raymond Chen.

In Raymond's post we hear about how tough it is for poor Microsoft (sidenote: what's with the bizarre victim complex that many Microsoft employees develop?) You see, prior to Windows 2000 someone at Microsoft made the choice that when you use a calculator in Windows, you really want to enjoy IEEE floating point rounding errors in all of your results, because the people that developed calc.exe — which would literally be a 8 hour project for an intern..you don't even need to make an installer — decided to take the laziest route possible, implementing it in the most naive way available. Raymond goes on the defensive, telling us that those critics just don't understanding floating-point. Not really, Raymond. They just don't understand how Microsoft could have ever thought that it was a reasonable decision for a calculator app to use and suffer from, versus the decimal math of virtually every other calculator app.

So Microsoft swapped out the embarrassing calculation "engine" of calc.exe (Jeff got the timing of the change seriously wrong. It wasn't between XP and Vista. It was before Windows 2000), put in the bignum-style implementation that should have been there from day one, and people were supposed to send them flowers or something? You ungrateful sons of...

Anyways, Windows 7 will invariably make a big impact, so I do plan on taking a look at it soon. But I'm certainly not motivated because calc.exe got some minor changes.

Does it really matter though? Ultimately what really matters is if the site renders as close to as expected as possible in the major browsers, and most of them happily overlook even egregious errors (Internet Explorer was criticized early on for being so forgiving, but given its dominance the other browsers really had no choice but to allow the same sloppiness. Most web publishers weren't about to re-engineer their site just to ensure that it displayed correctly in Opera, for instance.)

Out of curiousity I decided to check some other sites to see how many ensure that their (X)HTML is clean. The following are the results as they stand at this moment, though of course as content is added or removed the state will change (though a clean site is often a clean site with intention, and new content is automatically filtered to ensure that it is pure).

• FAIL - http://www.reddit.com - 36 errors as XHTML 1.0 Transitional. EDIT: Rechecked Reddit, and now it's a PASS
• FAIL - http://www.slashdot.org - 167 errors as HTML 4.01 Strict
• FAIL - http://www.digg.com - 32 errors as XHTML 1.0 Transitional
• FAIL - http://www.cnn.com - 40 errors as HTML 4.01 Transitional (inferred as no doctype was specified)
• FAIL - http://www.microsoft.com - 193 errors as XHTML 1.0 Transitional
• FAIL - http://www.google.com - 58 errors as HTML 4.01 Transitional
• FAIL - http://www.flickr.com - 34 errors as HTML 4.01 Transitional
• FAIL - http://ca.yahoo.com - 276 errors as HTML 4.01 Strict
• FAIL - http://www.sourceforge.net - 65 errors as XHTML 1.0 Transitional
• FAIL - http://www.joelonsoftware.com - 33 errors as XHTML 1.0 Strict
• FAIL - http://www.stackoverflow.com - 58 errors as HTML 4.01 Strict EDIT: Rechecked and now it's a PASS
• FAIL - http://www.dzone.com - 165 errors as XHTML 1.0 Transitional
• FAIL - http://www.codinghorror.com/blog/ - 51 errors as HTML 4.01 Transitional
• PASS - http://www.w3c.org - no errors as XHTML 1.0 Strict
• PASS - http://www.linux.com - no errors as XHTML 1.0 Strict
• PASS - http://www.wordpress.com - no errors as XHTML 1.0 Transitional

(I searched around for more good examples to sit in the PASS category, but sadly they are very few and far between)

Should this be normal?

No, it shouldn't.

Some of the errors in some of the mechanically generated HTML are simply unexcusable, and testify to the general level of sloppiness in the web industry in particular.

Check your HTML. Ensure it conforms to the specs it purports to obey, or accept defeat and step back to a less-demanding level. With tools like one keystroke validation and auto-cleanup HTML Tidy (which is available in module form, allowing you to auto-cleanup content mechanically inline in your site code - see this entry for an example of using Tidy from .NET code), there's simply no excuse.

Many will wave off such criticism, declaring that if it renders fine that's what really matters. Yet the worry about purity has more to do with the code maintenance process, and ensuring that an appropriate amount of care and concern is put into the product, in much the same way that you should strive to have 0 warnings in your projects, even if the compiled output works fine regardless. In the same way that I try (albeit with failures at time) to ensure that I avoid misspellings and typos, even if the message could be successfully conveyed with them.

For those unaware, the Mozilla foundation gets almost 90% of its income — used to pay developers, run servers, do marketing, and so on — from Google: $75 million dollars in 2007 (along with some chump change from Yahoo and Amazon). This isn't an act of charity, though, and for its payment Google gets default start page space, is the default search provider, and of course gets attributed with a lot of goodwill throughout the industry for helping to keep the project alive and robust.

Now that Google is strongly pushing their own browser, however, the relationship isn't quite as solid. Shortly before Chrome's release the contract was extended through to 2011 — probably by some concerned players that wanted to stop any NIHism from undoing what they had achieved — but that's just two measly years and will pass before most people realize.

The Russian deal seems to be one sign that the Mozilla foundation is soberly planning ahead.

And while they're considering who might step in if Google decides to bow out, they might look towards the most unlikely partner of all: Microsoft.

Increasingly Microsoft has been embracing Firefox as a platform, with various divisions working towards more than just locking you into Windows (which has perverted the cause of many Microsoft products for years, destroying potential greatness). Given the relatively pathetic progress of Internet Explorer, I'd go so far as to say that they've put more work into pulling Firefox into the fold than they have improving their own browser.

It's chump change for Microsoft (despite all of the doom and gloom stories about Microsoft, the reality is that they are still disgustingly profitable), it would buy them a tremendous amount of goodwill, it would give their ignored Live Search (haven't they abandoned the whole Live thing yet? I'm waiting for the next wave of inane branding synergies from that marketing midget) some attention, and it would give them a voice on the project that is most likely to continue to enrich and improve the web. Yet it wouldn't give then undue influence or control on this project, which we know because even Google was held away from the reigns of power (which presumably is why they wanted to let Mozilla use their ball while they went and bought another ball and built their own court.)

Quite an embarrassment. It was so bad I had to exile myself from blogging.

Even worse than the online embarrassment was the awkward discomfort and shame that owning an HD-DVD unit brought. When the HD-A3 eagerly popped up its now foolish looking HD-DVD logo as a guest happened in the room, the outcome was generally one of two shades of humiliating: Half the time they'd feel a little sorry for it, and by extension me. The other half was even worse, as they'd eagerly ask "Gosh, what's that? Is that the new high definition format?", forcing me to divert the conversation to whatever conflict was going on in the Middle East at the time (always one to fall back on). I quickly deported the player to the basement, if only to be free of the logo.

Seriously, though, having three young children is quite a lot of work. A former boss once remarked, on learning that I had my first child (my beautiful daughter Elizabeth), that I was a "hobbyist". I laughed at the time, thinking that he was just being elitist, what with his 8 kids or whatever it was. Turns out he was somewhat right, and I now look back at the relaxed life of free time I had with just one child. Even two seems relatively tame in comparison. Three, with them now edging towards the ages of two, four and six, introduces a lot of difficulties, as providing them all with age appropriate activities and challenges is a non-trivial task. The time available to post insightful blog entries has suffered.

But there is hope, faithful fans! I've devised a new technique that I'm going to try over the coming weeks. It's a sort of hybrid of "microblogging", but privately accumulating until it's a worthwhile post. I consider such writings cathartic and often self-elucidating, so I want to do it if only for personal satisfaction.

There's also the matter that with the recent addition of a "recent posts" panel on the right, that ungainly HD-DVD entry still falls in the list...so I've got to push out some content to scroll it off the list....just one more entry and it's outta there.

Oh and I also added the feed bag — again it was to try out a new technology and it proved a useful prototype — of links that I think are worthwhile, for anyone who might be interested.

Thanks for reading and have a great week!

Dennis

Financial Opportunities for Independent Application and Game Makers

While so-called “ISVs” (small software shops, often staffed by just the founding programmer) have largely disappeared from the Windows platform, pushed out by Microsoft’s total dominance of most IT spend alongside a widespread consumer mentality that software should always be free — whether in the libre sense ala open source, or the more prevalent gratis sense of free in the form of piracy (often justified with bizarre logic that goes something like “I paid $499 for this PC! I’m not paying more for software!”) — other lucrative opportunities have opened up for entrepreneurial developers.

Alternative platforms like cell phones (the iPhone, Google’s Android, or the more scattered but numerous Java ME targets) and game machines (such as the Xbox 360) have provided tremendous new opportunities with limited barriers to entry. Vendor online stores and fair revenue sharing splits have made selling your product easier and more rewarding than ever before.

On the Mac the independent software market remains very robust, with a consumer base that is willing to spend for software that enhances their quality of life in some way.

Display Technology

They’ve gotten larger (but lighter and more energy efficient), have faster response rates and better quality, with color range and contrast ratios seeing great improvements, all while remarkably coming down in price. Large-screen displays with 1920x1080 pixels are becoming commonplace in media rooms, even during the downturn.

The displays in our living rooms are capable of displaying the best of converging digital media. The division between the “home computer” and the home entertainment systems (a split that began when computers started demanding better displays, a need which saw the Commodore 64 and later generations leave the family room to take up accommodations in the home office) has dissolved.

JavaScript

Over a short span JavaScript has gone from being unloved and despised, begrudgingly used only out of necessity, to being widely exploited as a language that, while deceptively simple on the surface, is incredibly rich and succulent when you look just a little deeper.

Even long time practitioners of JavaScript regularly discover new functionality they never realized was there.

The increased focus on JavaScript, and the growing richness of the web, kicked off a skirmish between a variety of projects and vendors, with a horse race between the Mozilla project, Webkit (primarily in its Safari instantiation), Opera, and the upstart Google Chrome browser. JavaScript performance in some of these latest generation offerings can even rival native code in isolated scenarios, which is extraordinary for such a high-level, dynamic language long identified as a performance pig.

It’s now worthy of serious consideration whether to use one of the JavaScript runtimes (such as V8, Tamarin, or Tracemonkey) as an engine for projects having nothing to do with the client-side web.

Not all is roses with JavaScript, though. The drive towards ECMAScript 4.0 AKA JavaScript 2.0; representing a pretty dramatic shift in the language; hit some serious road blocks this year, eventually smashing off the road and exploding into mangled bits.

It’s a little fuzzy where JavaScript is heading in the coming years, but at least it’ll do it in an impressive number of iterations per second.

The Microsoft Xbox 360

Microsoft reduced the price and improved the interface and functionality. They diversified the marketing and game catalog to cater to more than just teenaged first-person shooter fanatics. For a unit that is now over 3 years old (with hardware that was probably spec’d out even a year earlier) it shows few age wrinkles, with very decent graphics at ultra-high resolutions, only occasional suffering frame rate slowdowns.

While it’s worth the price for gaming-fu alone, the 360 also doubles as a viable media extender (especially when coupled with a UPnP host like TVersity running on a PC somewhere else on the network). After bouts using full PC media boxes in the home theater setup, with the many time-thieving downsides they bring, I’m very happy to replace all of that with a simple Xbox 360 (playing streamed internet radio, Divx/Xvid, home movies on the networked NAS, YouTube clips, and so on.) With the 1080p connection and the horsepower to decode most anything without glitches, it’s a strong link in the media chain.

The marketplace functionality offers up easy access to a wide range of gaming and entertainment options, including the ability to queue up movies, shows and games on the unit or even on their online site from other devices, making them available on your 360 surprisingly quickly (I’ve queued up movies for the kids on my PC to find them available on the unit mere minutes later, seemingly exceeding the capacity of my cable connection.)

US owners with a Netflix subscription also get instant streaming of a wide range of movies.

With a decent motion sensing controller — which apparently is coming soon via a new controller codenamed “Newton”, though that could simply be more internet lies gaining truth through repeated assertion — the 360 would be uncontested.

Complaints are few, but the unit isn’t perfect.

It has a taste for polycarbonate, for instance, viciously destroying discs if you happen to move the unit at all while it’s on and endlessly spinning the disc.

[SUPER PRO TIP: After my GTA IV got eaten during a foolhardy move of a powered-on unit, the game would no longer launch but instead would freeze up the device. I never got around to replacing it, though I did discover that Rockstar Games does offer replacement media if you ship your old media to them along with $7.50. After the new NXE interface was released a few months back, with its newly offered install-to-disc functionality, out of curiosity I rented GTA IV from a local video and game store, installed it to the hard drive, and then returned it. From then on the game plays perfectly with my now defective disc in the drive. Presumably the sectors it checks to validate ownership are all still fine, and as another benefit I no longer have to listen to the optical disc noise whenever the unit is on, which is a huge improvement regardless.]

The “HD” movies available for rent online are decidedly not HD, at least not comparable to blu-ray or the defunct HD-DVD format. This isn’t surprising given that they clock in at about 4 to 6GB, versus the 30GB or so for many really high definition movies, both options using the latest codecs. They also offer a mediocre 24 hour view window from first play for rented/downloaded videos, and prices that seem a bit excessive (HD rentals come in at around $8 here in Canada, versus the $5.49 to rent from a video store, and the latter option gives me much more liberal usage times). Media embargos mean that a lot of movies simply aren’t available to Canadian users.

Computing Power

Computing power has gotten incredible. Quad-core CPUs, giant caches, ultra fast memory, arrays of massive storage devices. The storage front in particular holds incredible promise now that we’re adding SSDs to the mix. Here’s a pretty neat little product for a DIY SSD, in this case multiplexing 6 SDHC cards for much faster access. Get 6 of those (each with 6 SDHC cards), put them in a RAID10 array…you’d have absurd speed levels for about $800 for an array with 288GB of usable space. Sure that isn’t a lot of space, but it’s enough space for most reporting or OLTP databases, and you’d be enjoying 360MB/second or so read speeds and 60MB/second write speeds or better, coupled with almost instant access times.

Some inefficient-for-processing-but-good-for-development practices and technologies are becoming more tenable with the surfeit of computing power we have available to us.

The Bad of 2008

Computing Power

While it also made a showing in the Best of 2008, available computing power is still underpowered for video tasks. Once you start editing and rendering 1080p videos from the growingly common HD video cameras, and the speed of storage, the speed of memory, the speed of processors (even though video processing is one of the most capable and willing of using quad-core CPUs…it still isn’t enough)…all of it leaves a lot to be desired and makes it a less than pleasant, time consuming experience. An add-in MPEG4 compression coprocessor might be a good choice (my HD video camera has it, this tiny device doing real-time AVCHD compression, AVCHD being a variant of MPEG4, squashing to 17Mbps of video and audio info).

Video Codecs

The video codec world is a mess. If I want to convert from AVCHD to a Divx AVI, why is any transcoding necessary at all (which always reduces quality)? Both are MPEG4 codecs. In fact, virtually every top-tier codec now is just a pretty face and quasi-unique wrapper around MPEG4, albeit usually with just enough secret sauce to screw it all up. So why not just standardize on one wrapper? It seems to be incompatibility for the sake of incompatibility. [EDIT: Ben noted in the comments that I was quite wrong here. Turns out that MPEG4 is a bit of a hodge podge of codecs, and Divx shares little in common with AVC. From now on I’ll be encoding home video clips to x.264]

Blu-ray

The movies are too expensive. The benefits are too few. While the under-featured players (which seldom offer now common DVD functionality like Divx/MP4 playback, USB media playback, etc) are finally dropping in price, it’s still only a good option if you have a reliable source of rentable blu-ray discs (services like Netflix or Zip.ca, for instance, or a well-stocked local video store.)  If you buy discs you have the irritant of likely not being able to play them in the family minivan, the kids’ computer, your laptop, and so on, which is one of the major downsides of blu-ray.

The opportunity window for a new optical disc format is closing quickly given the growing interest and utilization in internet and proprietary cable system delivery (VOD from your cable company, watching downloaded or streamed movies on your Xbox 360, iTunes movie downloads, etc).

Locally, this year’s boxing day consumptionfest featured a bevy of blu-ray player sales, and it is a sign of the times that many of them still have large numbers of units left. Strangely the lack of competition with HD-DVD has left a lot of people uninterested in the genre as a whole, which is quite contrary to many predictions that predicted a blu-ray euphoria once the competition was settled.

It’s a bit surprising that blu-ray discs carry a price premium: They’re harder to rip (though the idea that they are uncrackable has proven untrue), and the outcome is a massive file that is difficult to duplicate on media — unless you want to put it on a burnable blu-ray disc that costs more than buying the movie new, or seriously degrade the quality which would sort of miss the point of ripping from a high definition source (no codec Divx magic here. The codecs used on most blu-ray discs are the cutting edge) — or online. I would expect the media groups to heavily push blu-ray, at no price penalty, purely to try to head off piracy with a more unwieldy format.

Google’s Chrome Browser

Chrome could be the world’s greatest browser and I still wouldn’t like it. Thankfully it isn’t the best browser by a long shot, so I don’t have to rationalize that conflict too far. Aside from a mostly theoretical process isolation model (which seems to have little real-world benefit, as many users find Chrome to be one of the most catastrophically crashy browsers of the bunch), Chrome offers little to justify choosing it. The V8 JavaScript engine would have been a winner if it came out 6 months earlier, but it turns out that Google was far from alone in working to speed up JavaScript.

The reason I grief about Chrome is because it has no reason to exist. The only viable reasons why Google felt the need to make their own browser are decidedly not good for the internet at large. Google owning and controlling a browser brings the same worries and concerns that Microsoft controlling a browser does.

Google makes their money selling ads. It concerns me having an advertising company running a browser project, even if they do provide a related-but-not-quite-the-same source tree as if that makes everything okay.

It bothers me even more that Google’s advertising initiatives have completely focused on subverting the Firefox userbase, cannibalizing a credible alternative to Internet Explorer that was finally becoming mainstream. I’m not entirely sure what Google’s motives are with this plan, but there is no way they can color this as Google the Good And Benevolent. It’s either an insidious end game playing out, or it’s some insular, egotistical developers at Google who just had to control things themselves, not content to work with the existing browser projects when they can stand tall and scream “We’re Google damnit! We’re super smart!”

Macromedia cum Adobe Flash

Many of the problems people encounter with “Firefox “(purported memory bloat and CPU saturation) are Flash related.

On my three-year-old son’s low-end PC — a Pentium IV 1.7Ghz with 1GB (anemic but not that bad) — many of the great online children games sites (such as CBC, NickJr, PBS, TVOntario), which are largely built around Flash, slog to an unusable crawl in Firefox, yet they run with gusto in IE on the same PC. This is odd given that the overwhelming bulk of computation in both cases occur entirely within the Flash environment (the browser essentially acting as a thin wrapper, and I can't imagine the basic communication between environments is so onerous that it could account for the difference), so hypothetically there should be complete equivalency between the browsers.

The #1 problem many users have using Linux-based boxes to browse the tubes are Flash related. This is becoming more critical as more and more Linux-based NetBooks hit the market.

Memory leaks. Sluggishness. Crashes. Flash is quite often the source.

Flash brings a tremendous amount of richness to the web. Without it those game sites really couldn’t exist (I exclude Java and ActiveX given their serious problems, not to mention that Flash is purpose suited for exactly these sorts of projects), and of course it is now the foundation for most every video site. Adobe even donated the Tamarin project to the Mozilla project, though it didn’t turn out being the win it should have been given that Tamarin is largely focused on the DoA JavaScript 2.

Adobe really needs to fix this player. It has become foundational to the web.

Internet Explorer 8

This browser is so incredibly terrible that I have think that all of the MS developers that launched IE to greatness between versions 4 to 6 must have gone on twenty-year sabbaticals with their then-lucrative stock options. This browser is terrible in every way, and unless there is some tremendous forward momentum in the final sprint, it is completely unjustifiable as a browser choice, not even ranking among the top tier browsers (which include Firefox, Safari, Opera, and to a lesser extent Chrome). Using IE 8 would be like using Netware to run your network today, simply because it once was a good choice.

Nintendo Wii

The Wii has a lot going for it.

It has a great range of family friendly games, built in wireless connectivity, and unarguably innovative controls. It has a large catalog of games, and is easy to use with a low complexity barrier to entry.

It has been the definite winner of the next generation console wars, even though Nintendo brought the last generation to the fight.

I want to love the Wii.

But I don’t.

Many Wii games seem to utilize the motion sensing controls in the most gimmicky way possible, featuring shallow gameplay that seems more like a quickly hacked together technology demo than a serious offering.

Even where you aren’t forced to flail around with the control to do rudimentary actions, you quickly find that philosophically many Wii games insist upon “balance”, kicking you in the nads when you’re ahead, giving you a firm push when you’re behind.

Maybe there’s something cultural about that. But when I’m trying to crush my children’s contest hopes with my Mario Kart awesomeness, teaching them a life lesson about competitiveness, it’s a bit flummoxing to continually get knocked down while they get an endless series of speedups.

My five year daughter easily dominates my long-time-bowler father-in-law in Wii bowling, with seemingly random movements yielding amazingly strings of strikes.

Then there are the often unavoidable text bubbles that assault the player in the hundreds, making an unwanted appearance in many Wii games (particularly those sourced from Japan. This seems to be something that Japanese games have relied upon heavily going back to the Phantasy Star / Zelda days). Going past these asinine play obstacles is incredibly irritating, at least for me. Half the game play, it seems, is the challenge of maintaining sanity and an interest in the game after clicking through a hundred sparse text bubbles full of unnecessary and uninteresting filler text.

While the motion sensing element of the controllers are arguably best of this generation (using standard Bluetooth making them theoretically usable on other devices like the PC), the other hardware of the Wii is seriously outdated: Basically last generation’s Gamecube with a paintjob. The SDTV output is underwhelming, offering so little definition that it’s painful to play split-screen multiplayer, which is a waste given that many of these units are hooked up to sets capable of at least 1280x720, often even 1920x1080.

As one of the early buyers of the Wii, and the target demographic given that my family features three young children, I can’t help but express some disappointment in the unit. I really don’t understand how it still manages to get the attention it does, still existing in a honeymoon while people stock up on hardware and games they’ll never actually play.

[Sidenote: Got the kids Wii Music for Christmas to discover it’s more of the same formula. It’s a mile wide and one inch deep. Yet again it seems like a technology demo dressed up as a pretend game]

Online Comments

While it’s expected that the sophistication of comments on a site like YouTube will lie somewhere around the intellectual level of head-injured drunk, there remained the hope that other sites might feature a more intelligent contributor. When some major Canadian newspapers added comments, I looked forward to a presumed onslaught of interesting and thought provoking contributions. Instead I was disheartened by the complete lack of political sophistication, the partisan political team cheerleading, the sophistry, the endless logical fallacies, the idiocy…all of it has reduced my opinion of mankind. I can only hope that it is a grossly unrepresentative demographic that feels the need to post their thoughts on newspaper sites.

The Decline of Honesty

When did lying become so acceptable and commonplace? When did people feel so justified in descending to lies so long as it gets hits?

The Internet has always been home to a large number of hoaxes and exaggerations, but this year saw truth and reality suffering a serious beat down, and it only looks to be getting worse.

Some lies made their presence known via the blogger’s tale of extraordinary interaction with some larger than life caricature (which a gullible readership ate up because it confirmed their own simplified worldview, biases or bigotry.) Honestly, goatse with a chaser of tubgirl was less offensive than some of the obviously manufactured fiction being paraded as truth nowadays.

Lies also appeared with frequency on “citizen journalism” sites, where there’s a strong incentive to egregiously misrepresent or misreport, zooming to top the social sites with the most extravagant and inflammatory story possible. A recent example — only one of an endless stream of alike cases — would be the FDA’s decision that the many benefits of eating fish are so compelling that they outweighed the small potential risks of mercury, so they considered revising their existing recommendations that encouraged avoidance, switching to one that was more moderate, in the same way that exercise is generally recommended even though you might get hit by a car or a stray asteroid. Along came the “citizen journalism” sites simply discarding the kernel of truth, instead declaring that the FDA decided that mercury is “safe” (presumably at the behest of their Mega Mercury Corporation masters).

No nuances or fuzzy grays that require a moment of thought are desired when there’s a crowd to enrage into voting you up and forwarding the story on to others. For all of Fox News’ many, many, many faults, they have a brother in citizen journalism on the net.

Lies also grew via the viral videos that have been saturating the net, with that seemingly no-cost amateur video more likely being an expensive production of a Lying for a Living viral video manufacturer. Lie it up, and afterwards everyone can have a big laugh about it and cheer on your product.

So much content is undeclared fiction now that it’s a bit of a cry wolf story playing out. Many readers have become so jaded they simply believe nothing that they can’t witness directly themselves. You don’t even need to “look at the pixels” anymore: If it’s getting a lot of attention, it’s probably made up from a blogger’s imagination, a gross misrepresentation that has little correlation with reality, or a viral video that somehow is going to get some company attention.

Yeah, there’s a sex toy on the side table of your real estate picture — ha ha ha. Enjoy the PageRank your little hoax earned as every social media site plays right into your devious plans with links, a gullible public all thinking they caught you in a hilarious blunder instead of the planned out web strategy you’re pursuing. Another variation is the "super duper terrible site...let’s all point and laugh", when the only ones that should be laughing are the people fooling you.

True stories have little impact in an environment of gross exaggerations or manufactured tales and caricatures.

The Economy

It didn’t take a psychic to see the dangers of the bubble economy (though many who pronounced some caution now hilarious see themselves as economic seers). Madoff is getting a lot of press for his apparent ponzi scheme, yet in many ways the whole market is one giant ponzi scheme. While people often like to talk about the “true value” of things (homes, oil, resources, etc), the true value is often whatever the market is willing to bear at the moment — whatever the prevailing mindset is — which is a situation naturally prone to booms and busts. Hopefully the downside of the bust is flushed out quickly and we can begin the next round of financial chicken.

Hopes for 2009

I hope the public at large becomes a lot less gullible. Extraordinary claims demand extraordinary proof.

I hope Google abandons the Chrome project, as they have abandoned many other misadventures before. Already the adoption has been mediocre, despite widespread launch press and a strong push by Google.

I hope upstream bandwidth starts opening up. It’s great that I have 10Mbps downstream virtually around the clock, but when I want to send a 10GB home video to a relative (yup, people actually do make their own content sometimes. It isn’t all about P2P sharing of pirated material), suddenly that 0.5Mbps upstream looks rather anemic. A lot of potential uses for the internet are being choked off by the widespread limiting of upstream bandwidth.

I hope SSDs continue to increase in speed and drop in price, and the operating system makers such as Microsoft properly adapt to this storage mechanism (for instance a paging file is not a good idea on flash devices. Personally I think they aren’t a good idea at all on any modern PC). Unlike a hard drive that generally has but one head mechanism, the potential multiplexing within SSDs is virtually unlimited, so the speed potential is virtually boundless.

I hope social link sites move towards more enlightened, individually focused algorithms, moving away from groupthink and herd behavior.

I hope disparate devices and appliances continue to gain capabilities and marketshare, and a robust third-party development community is encouraged and supported for each (and not just in the "create free stuff for our product so we can sell more and make more money, suckers" way, but in a mature model that allows for monetization by partners — even the tiny uISV — bringing value to the product and rewards to the creators).

Which couplings make sense, and which don’t?

The mentioned Microsoft / Yahoo union, for instance, would be a terrible partnership with almost no merit beyond maybe giving a short-term legitimacy to Microsoft’s online efforts, temporarily hushing the naysayers.

A Google and Yahoo partnership makes just as little sense.

Of the mergers that would yield more than a fraction of the sum of the parts, one that just won’t stop buzzing loudly in my brain, is one that my subconscious has just assumed as inevitable– a Sony/Apple merger.

By today’s market capitalization numbers, it would actually be an acquisition of Sony (worth approximately $42 billion today) by Apple (worth a hard to believe $150+ billion), which really is extraordinary to consider.

How could Apple, supplying a small segment of the computer market, along with a couple of portable electronics devices, dwarf the global electronics giant Sony?

It seems incredible, but it’s true.

Apple should cement some of that market capitalization with a merger (a polite acquisition) of Sony.

The synergies possible between these two companies are absolutely incredible, giving Apple a significant presence in the entire consumer electronics spectrum (a huge potential worldwide market for services like iTunes), while gaining industry leading engineering and fabrication capacity.

They’d even have a Windows-equipped PC division, as hilarious as that would be. They’d have a whole media division, including movies and music, including the Gracenotes catalog that was snatched by Sony.

It goes on and on. The more I contemplate how each company could use the other, the more inetivable such a pairing seems.

The only conceivable negative for such a merger would be the infamous distaste Japanese companies have for foreign control (you lose some value when all of the executives jump out windows in dishonor). A polite “merger of equals” would probably be the only tenable option.

If such a merger came to pass, it would be a very bad day to own Microsoft stock.

One such pattern, observes Paul, is that his selected success stories started with no real revenue plans beyond "get big and flip it".

No subscriptions. No advertisements. No pop-ups. No interstitials.

They started more like a charity than a real business, just bringing good to their userbase with nothing asked or expected in return. At least at first.

This isn't a new position for Mr. Graham. He has long advocated the idea that you just need to worry about getting the eyeballs and you can figure out how to make money from them later. Or better still — reading between the lines — you can let the sucker you flip the thing to worry about the gritty details of how to monetize it, suffering the consequences if the userbase burns it to the ground in defiance of any revenue generating scheme.

This isn't a surprising position for Paul to support. It is entirely aligned with his micro-VC organizations' business model, which is to take young, time- and energy-rich grads, fresh out of the college mill, and bankroll them with a small investment (which they need because they won't be earning anything from their Internet baby anytime soon), and then cash in when/if they manage to flip it to a sucker that still buys into the many eyeballs model (a strict "No Returns" policy in effect.)

Given the small investment, only a percentage of YC Combinator's `fundees' need to hit the jackpot for the strategy to succeed, at least for Paul. He's playing house odds in this startup casino.

Paul provides some examples to demonstrate his position that the charity-that'll-make-you-rich approach is the winning strategy: Google and Craigslist. Incontestably successful companies, and most would be over the moon to experience a fraction of their success.

Let's take a closer look at these examples, and see how relevant they are to Paul's central theme.

Craigslist

Craig Newmark, the founder of Craigslist, started compiling a list of upcoming local events in 1995, publishing it to subscribers via a listserver. Later, after the list was well established and had a healthy subscriber base, he started publishing entries to a website, adding functionality to allow users to email directly to categorized lists.

Today Craigslist is an internet superstar, constantly ranking among the top 50 websites worldwide. It pulls in impressive revenue numbers through a model that Craig himself describes in this entry on Yahoo! Answers.

Craigslist is a rare survivor among thousands, tens of thousands, or perhaps hundreds of thousands of exceedingly similar lists/classified upstarts, most of them run ad-free and for free (sites were often ad free as a simple side-effect of the barriers to entry to hosting ads pre-2000. At that time it wasn't as simple as signing up for an AdSense account).

Craigslist had the perfect, rare combination of the real-world personal connections of the founder, an ideal starting locality, userbase, and a progressive evolution that allowed it to build enough momentum that eventually the network effect took over, and you'd use craigslist because everyone uses craigslist, at least in some markets.

What You Can Learn From Craigslist: Craigslist is an extreme anomaly. Holding it up as an example of a path to follow is fundamentally akin to analyzing the number picking "strategy" of the latest lottery winner. It's also an excellent example of how corrupting, and falsely compelling, a survivorship bias can be. Countless sites have followed a close to identical path, failing miserably.

Google

Google hit the scene during the portal craze. This was a period when every other search engine, failing to sufficiently profit off of search alone, started merging with a gangly bunch of dance partners. Excite hooking up with @Home, for example. To "leverage the synergy", each quickly morphed into a "destination for all things" portal, fattening up their content until they featured a landing page absolutely packed from margin to margin with text, news, stock quotes, images, comics, horoscopes, etc.

Ads were almost invisible in this era of visual pollution, and were seldom the problem. Many users were using dial-up, so the content inflation wasn't just aesthetically deplorable, but it also made the search process a slow, unpleasant experience.

Sergey and Larry had been working on some algorithms for internet search during this period, and with the dot com boom peaking they managed to pull together an impressive million dollars in financing before even launching their beta website.

When they did finally get something online, they did the absolute minimal amount possible. Later they described the utter simplicity of the first version as a function of their lack of HTML knowledge: It was the best they could do, or cared to do, at the time.

They had differentiated themselves, however inadvertently, and it worked brilliantly. They had copious long-term financing (not "flip it to someone else" financing) before even launching, so they had no need to worry about making money immediately, using the website as a technology demo that would conceivably allow them to sell search technology and services to third parties and businesses.

While the quality of the search results got the Google buzz started, the dial-up bandwidth-friendly simplicity of their offering really won people over. Yet it was a simplicity that came primarily because the company only really had one product — search — and couldn't link to hundreds of other provided services.

When everyone else went heavyweight, the minimalism of Google got it a lot of attention among technology trend makers. That exposure on sites like Slashdot — amplified when the community learned that Google ran Linux — got them their next $25 million in financing, and the rest is history.

With their advertising initiatives Google took exactly the same approach, and when everyone else was using pop-up, high-bandwidth, obnoxious ads, Google zagged and had text ads. As others have adopted Google's text ad approach, Google has started adding in animated and full graphics ads to their docket.

What You Can Learn From Google: If you have an algorithm or technology that is sufficiently impressive enough to get a million dollars in financing before you've even left the drawing board, maybe you can take lessons from Google beyond "differentiating from the competition is good" (which is fairly obvious advice.)

The Cold, Hard Truth

The vast majority of "go big of go home" web ventures will fail. It isn't a meritocracy. Luck has a lot to do with it. There is little you can learn from the success stories unless you also learn from the failures, yet aside from the huge flameouts (which had to have enough success to even be notable), most failures fizzle out and disappear without a trace.

From the opposite angle, many "grow revenue from day one" websites have succeeded admirably (I just gave a local babysitter directory $39 for 3 months of lookups), albeit not without the "lottery ticket" quick payoff that a miniscule percentage of the winner-take-all players yield.

Ethics and Morals

It's a risky and disingenuous proposition to build a website, baiting a community, on one model — the "Good" and charitable model — and then switch the userbase once the founder's numbers get drawn. It's a scummy behavior to engage in, much less evangelize. It also might have the opposite effect than intended: I might not care whether a site like a social link voting site has a revenue model, but when I was considering photo sites I immediately discarded those that followed the Paul Graham business model, considering the risks too high: Either it would eventually be forced to flail about, obnoxiously trying different approaches at making it pay, or it would fold with a "Sorry We Got Bored Our Numbers Didn't Come Up" notice one fateful day. Instead I went for one with a sustainable business model, and haven't been unhappy with my decision (even if it did cost me a bit per year for all of the features).

Bits And Bytes

• This entry was authored in emacs.
• A quad-core is definitely your best choice, especially given the huge price drops just announced.
• I called the whole Riya thing perfectly.
• The standard for comments in code shouldn't be driven by the need to provide endless guideposts for incompetent programmers. If it describes something that should be obvious by the code, you're fixing the wrong problem (which can be either unclear code, or incompetent programmers, or both).
• Most developers don't rely upon books anymore because the overwhelming majority of technical books are garbage.
• Bits and Bytes was a brilliant educational program on TVO in the early 80s, and it is entirely responsible for beginning my love of computer hardware and software.
• Nassim Nicholas Taleb explores survivorship and confirmation biases excellently in his books the Black Swan and Fooled by Randomness. While I was put off by his ego, and the expansion of a paragraph-worth idea into chapters, they're still great reads.

While some were productive and happy with just the W3C specs and a copy of emacs, many of us only used XML intermittently, building an export, import or transformation that simply worked, promptly forgetting all of the nuances of DTDs versus XSDs versus XDRs, or the quickly changing XSL(T) specifications.

It was a great step forward in the uptake and quality of XML utilization to have such an easy to use, up-to-date tool.

At the time XML Spy was basically shareware, offering a fully featured 30-day trial, at worst popping up the occasional “please register me!” exhortation.

Many just registered it: it was an easy sell at $54 a user, less if you bought multiple copies. That’s almost disposable money, and was an easy pitch to most managers. It was easy enough saying “let’s get a copy for everyone in the group. Even for the guy in the cube near the washroom, anti-XML rage bursting from his trembling lips in a spray of spittle and phlegm.”

Time goes on and we all moved to different projects, divisions and companies, often with long gaps needing little or no in-depth XML. When those instances came up, we’d try to find an old licensed copy, or would download the latest trial, using yet another toss away email address for the validation.

And XML Spy just kept getting more expensive. The company grew and grew (note that the domain on the original archive.org link above actually expired, and now sits in the hands of a domain ad purveyor), and the dollar signs in their dreams had them imagining, apparently, of a day when millions of information workers sat toiling their days away in the pure awesomeness of XML Spy. In emacs-esque form, it had grown more and more functionality, even if many users never used it for anything more than creating and validating schemas and transformations.

By late 2000, the price of XML Spy had inflated to $149 a user. By the end of the next year it hit $399 a user. By late 2006 it was up to $499 a user (at some point dropping the space between XML and Spy, becoming XMLSpy).

As I write this it’s up to $539 a user.

Maybe XMLSpy is developed in a poorly insulated aircraft hangar in Siberia, and thus is strongly impacted by the price of oil?

A ten-fold jump in price in about 8 years seems excessive. What was once a wonderfully priced utility is now a considerably expensive development ecosystem. What was once an easy purchase (at one workplace I just paid for it myself rather than deal with the annoyance of a requisition form) is now a difficult to justify expenditure, requiring vendor comparisons, and negotiations with middle managers. When the money handlers are convinced, often it’s just for partial coverage of the development team.

You end up with the “XML guy”, rather than having a team appropriately equipped with a uniform set of tools.

Of course, clearly my complaints are off base. Altova obviously did appropriate research, and they determined that there really are people and groups who’ll happily pay more for an XML editor than they paid for their entire Visual Studio suite.

But come on, Altova – bring back a, err, “Semi-Professional” version – something with XML schema and transformation authoring and validation and nothing more. No grand vision where your product is the center point of a developer’s existence. Put a reasonable price tag on it – like $59 – and I’m sure you’ll get a lot of sales where right now you get none. I realize you probably have lots of big buildings with expensive lights, and layers and layers of bureaucracy to finance, but don’t do it all on the back of a simple little XML utility.