Survey Results are Meaningless minus the Methodology

If you're in the survey (or "Marketing Intelligence") business, an easy way to gain lots of free publicity is to publish some results that feed into an ongoing skirmish: For little cost a press release will percolate across the web, bubbling up everywhere, spreading your brand. In all likelihood a newswire writer will have the same motivation, and carefully pick and choose factlets with the goal of making a story more inflammatory.

Given that smartphones are big business now, and the battle (RIM, Nokia, Apple and Google's hordes) is a very public one, smartphones and their operating systems are often the focus.

We've seen this play out quite a few times already: Some survey results come out, usually with a very pro-Apple slant (the Apple enthusiast ecosystem is far more receptive to such material, and is more likely to run with it), and is quoted everywhere.

As always, methodology and details actually come second. It's then that critical, mind-blowing flaws in the survey or its presentation/interpretation are revealed. Of course by then the message was already delivered and has become truth through repeated assertion.

So for all you market research startups out there, I encourage you to post a self-selective survey on the website of an already biased group where they can profess themselves as wealthy, virile, mega-sexed super-humans. Release these extraordinary findings via press release and watch as lazy bloggers and tech columnists run with it.

Remember this ridiculous survey? It was the harbinger of doom for Android. Only it wasn't. It is only one of a long line of terrible surveys, and only one example where the tech press and wire writers misrepresented the already poor survey, or at a minimum didn't bother asking the questions that matter.

So we'll wait for GfK's methodology (never expect tech journalists to care to look this up. As an industry it is rife with incredible ignorance and laziness), and the inevitable correction will be a whimper to the roar of the original release. At a glance it is obvious that all is not as it appears, and it isn't simply that the numbers don't confirm my bias: They just don't add up, and have little correlation with other surveys on the same topic.

RIM Isn't Dead

Many years back I applied to RIM. They never contacted me. Clearly that's the source of all of their travails today.

Really, though, I await great things from RIM: The company had some golden handcuffs to their existing platform — the lucrative enterprise messenger system undermined more innovative initiatives in the same way that Windows and Office hobbles Microsoft — but now that they're seeing some serious attacks that can't be ignored, the battle is underway again. The PlayBook looks like it may be a winner, and I'm willing to sandpaper my fingers to use that 7" tablet if it delivers what it promises. QNX is simply a glorious operating system (it is elegant and close to perfect), so that alone has me interested.