Back in May I wrote about the poor performance of Microsoft's SQL Server product when doing trivial queries like simple key-value lookups. In such cases the massive overhead of the query engine led to return rates several orders of magnitude lower than many specialized KV systems, such as Memcached or Redis.
In most operational scenarios this isn't a big problem, as most queries are complex enough that the overhead diminishes to the point of irrelevance, and the "best route" of the query plan overcomes that inefficiency.
At the time I was testing with SQL Server 2008 and 2005. I finally got around to redoing those tests with SQL Server 2008 R2.
What a difference an R iteration makes.
Instead of 5,000 key lookups per second per caller, with 2008 R2 I'm seeing more in the range of 200,000+ simple lookups per second per caller. I've replicated this result on several installs.
Remarkably this has gone with little to no commentary in the community.
For larger queries the performance doesn't show such a deviation from the 2008 baseline, but for the simple purposes it really is incredible.
If you're in the survey (or "Marketing Intelligence") business, an easy way to gain lots of free publicity is to publish some results that feed into an ongoing skirmish: For little cost a press release will percolate across the web, bubbling up everywhere, spreading your brand. In all likelihood a newswire writer will have the same motivation, and carefully pick and choose factlets with the goal of making a story more inflammatory.
Given that smartphones are big business now, and the battle (RIM, Nokia, Apple and Google's hordes) is a very public one, smartphones and their operating systems are often the focus.
We've seen this play out quite a few times already: Some survey results come out, usually with a very pro-Apple slant (the Apple enthusiast ecosystem is far more receptive to such material, and is more likely to run with it), and is quoted everywhere.
As always, methodology and details actually come second. It's then that critical, mind-blowing flaws in the survey or its presentation/interpretation are revealed. Of course by then the message was already delivered and has become truth through repeated assertion.
So for all you market research startups out there, I encourage you to post a self-selective survey on the website of an already biased group where they can profess themselves as wealthy, virile, mega-sexed super-humans. Release these extraordinary findings via press release and watch as lazy bloggers and tech columnists run with it.
Remember this ridiculous survey? It was the harbinger of doom for Android. Only it wasn't. It is only one of a long line of terrible surveys, and only one example where the tech press and wire writers misrepresented the already poor survey, or at a minimum didn't bother asking the questions that matter.
So we'll wait for GfK's methodology (never expect tech journalists to care to look this up. As an industry it is rife with incredible ignorance and laziness), and the inevitable correction will be a whimper to the roar of the original release. At a glance it is obvious that all is not as it appears, and it isn't simply that the numbers don't confirm my bias: They just don't add up, and have little correlation with other surveys on the same topic.
Many years back I applied to RIM. They never contacted me. Clearly that's the source of all of their travails today.
Really, though, I await great things from RIM: The company had some golden handcuffs to their existing platform — the lucrative enterprise messenger system undermined more innovative initiatives in the same way that Windows and Office hobbles Microsoft — but now that they're seeing some serious attacks that can't be ignored, the battle is underway again. The PlayBook looks like it may be a winner, and I'm willing to sandpaper my fingers to use that 7" tablet if it delivers what it promises. QNX is simply a glorious operating system (it is elegant and close to perfect), so that alone has me interested.
On Monday I posted an entry titled Lies, Damn Lies, and surveys, the focus being that surveys — especially online surveys — are often of dubious merit, and worse are often intentionally or ignorantly misinterpreted by the press.
Yet it works wonders for both the press and the marketing research companies: There is a whole industry of lazy tech writers who will run with whatever is sent to them. Any slant they provide merely globs extra vaseline on the lens of accuracy.
In that case I was spurred by some just released results of an online survey done by GfK.
Take a look at the Fortune summary of said survey. As is the case with most other retellings, the Fortune writer seemed to simply rewrite the original Reuters story which itself said:
The survey found that just 25 percent of smartphone owners planned to stay loyal to the operating system running their phone, with loyalty highest among Apple users at 59 percent, and lowest for Microsoft's phone software, at 21 percent. Of users of Research in Motion's BlackBerrys, 35 percent said they would stay loyal.
The figure was 28 percent for users of phones running Google's Android software, and 24 percent for users of Nokia Symbian phones.
The Fortune story includes a big graph with the title "Plan to stick with your smartphone OS?". The iPhone towers at 59%, with Android down at a miserable 28%.
Wow, looks pretty rough for Android!
So I emailed GfK and they kindly responded with the same press release that they had sent out to the press. Unfortunately it still is only a summation, including no notes on methodology or the actual questions asked (which can often be very leading), yet it is far better than most survey companies that simply hide their summary behind a paywall, enjoying the attention as the press mangles it into something more hit magnetic.
Here's the table that was the source for most of these stories. I have to guess (based upon the surrounding wording) that it was the responses to a question asking current smartphone owners what they're going to consider when they next upgrade their phone.
| Smartphone Ecosystem | ||||||
| Overall | Apple | Nokia (Symbian) |
BlackBerry | Windows Mobile |
Android | |
| Will stay loyal to smartphone OS | 25% | 59% | 24% | 35% | 21% | 28% |
| Will stay loyal to smartphone OS but switch handset make |
7% | - | - | - | 8% | 16% |
| I will look at all options | 56% | 36% | 60% | 58% | 61% | 49% |
| Will switch smartphone OS | 6% | 1% | 8% | * | 5% | 2% |
| Don't know | 7% | 4% | 7% | 6% | 4% | 4% |
There's Android at 28% among these single choice options. That's the meat of a lot of easy "news" stories.
Note the second line, though: "Will stay loyal to smartphone OS but switch handset make". Sum those two lines and for Android suddenly you're at 44%. Sure, maybe someone with an HTC Evo 4G is looking longingly at a Samsung Galaxy S, or maybe they're imagining getting that Android-powered Playstation phone when it comes out, but we (meaning virtually every single press reference to this survey) are talking about the OS here.
So Reuters (and any that followed Reuter's lead) couldn't manage to achieve a grade-school reading ability. I'm serious: my daughter had table reading assignments in grade 1 that were just like this, a job that many in the tech press would fail.
Extraordinary.
But it gets even better. Among those who answered "I will look at all options" (which ideally should be 100% of respondants), here are what each current platform's users are considering as their considerations when they upgrade at some point in the future (multiple choices obviously being allowed).
| Current Smartphone Owners | ||||||
| Overall | Apple | Blackberry | Nokia (Symbian) |
Windows Mobile |
Android | |
| Apple iOS | 53% | 85% | 46% | 47% | 43% | 38% |
| Blackberry OS | 33% | 22% | 74% | 34% | 41% | 22% |
| Symbian | 23% | 14% | 18% | 40% | 18% | 13% |
| Windows Phone 7 | 41% | 28% | 40% | 47% | 65% | 31% |
| Android | 51% | 40% | 41% | 48% | 42% | 84% |
It paints a rather different narrative than the common, egregiously wrong interpretation. You end up with hilarious ignorance like this garbage. With what you have learned here, check out the chart here and see if you can figure out what's wrong.
The iPhone still takes the lead (with loyalty oddly dramatically higher in Germany — the home country of the survey organization — than in any other nation), but look back at the original Reuters statement: "The survey found that just 25 percent of smartphone owners planned to stay loyal to the operating system running their phone...the figure was 28 percent for users of phones running Google's Android software".
I suppose that sounds better than the more accurate "44% of Android users have pledged undying allegience to the operating system, regardless of the endless and unpredictable changes among competing choices. Of the remainder, the vast majority still plan on hopping back on the Android train regardless.
Another day, another user account database compromise.
Already there are reports of various accounts with trivial, re-used passwords being exploited to send spam, propagate twitter worms, etc.
It's pretty obvious by now that some users simply won't stop widely re-using using the same password. That security drum has beaten loudly for years, yet with every user account compromise it's the same reaction by so many, including among the technically savvy.
Oh no! The password I use everywhere is now public!
A question I have to ask those who re-use passwords is why did you ever trust Gawker in the first place? Why did you trust every employee and intern of that group to do the right thing? Trust that every backup is treated with the appropriate care and concern, every service patched and monitored? Why did you trust that the site hasn't been compromised all along?
Brute force exposing passwords isn't even necessary when nefarious agents have owned a site. They'll just grab it before it's even hashed.
You shouldn't trust these sites with a shared password. You simply shouldn't.
It's actually a breath of fresh air that Gawker is salting and hashing passwords (albeit with the mediocre DES), so technically they're not the worst, though they're certainly not the best. Yet still, we're assuming that there aren't password siphons logging away cleartext before the password ever makes it to the hash.
I'm not singling out Gawker: You shouldn't trust any site except for those that you must (like your bank). If you're panicking now because your trivial password that you use everywhere is out there, you're doing something wrong because it's entirely possible that it was exploited all along.
While only tangentially related to the Gawker exploit, I would like to take this opportunity to revisit a proposal I made half a decade ago: One Password To Rule Them All. In that proposal I opined that the input type="password" element needs to optionally add (perhaps via a secure="secure" attribute):
Ideally this uses a very computationally demanding hash like blowfish over multiple rounds, pushing the envelope of brute force attacks. In my original suggestion I noted that sites should be able to provide variants to avoid replay attacks or eavesdropping vulnerability, though that's an incredibly weak alternative to simply using the proven SSL for that purpose.
What am I missing? Why shouldn't we do this sooner rather than later? Think of this as bringing digest authentication to forms.
Such a scheme improves security in a number of ways, though it only marginally helps in the Gawker scenario (though it does invalidate the simple dictionary attack being performed for the low hanging passwords, each entry could still be brute force evaluated).
This entry got a lot of attention, as obviously it's a growing concern. I thought some of the comments deserved responses.
It has no notification LED.
It's a decent if unexceptional device. I can live without a real keyboard. I can live without a microSD slot (especially given the broken way that Android uses external storage). NFC is neat if extremely premature and consumer-only at this point.
The lack of a notification LED on Google's flagship device, however, is deadly.
I don't want to endlessly turn the screen on or suffer disruptive notification sounds (that I miss if I'm not nearby at that moment) just to know that an email dropped.
Hence why I like devices to have a notification LED. Preferrably one that is easily visible when it's in a case (beside the power connector would be perfect).
The Nexus One has such a notification behind the trackball. It's suboptimal given that it suffers from a very slow cycling rate by default (meaning you have to stop and look at the device for 10 seconds to see if it pulses), and the fact that it's hidden when the phone is in a case, yet it's far better than nothing.
Purportedly there are some fixes conceived to use several of the OLED pixels as a hacked out notification LED. Aside from the poor positioning and minimal visibility of the low brightness screen, my big concern there is OLED fade. Like plasma OLED has a finite lifespan, and the pixels that shine the brightest live the shortest, so you may end up with a notification burn in pattern on your screen.
My LED notification is just as valuable when it isn't blinking. It communicates with accuracy and reliability (meaning I'm assuming the battery hasn't run dry) that there is nothing demanding a response.
This "negative notification" concept eludes many, yet it's incredibly valuable.
Tracks in snow tell a lot, but snow without tracks tells just as much. A dog barking at an intruder tells you plenty. A dog quietly sleeping tells you as much.
Many are predicting that the smartphone market is set to explode in 2011, some speculating that Android is poised to capture much of that growth.
Not a very risky prediction, really.
MG Siegler weighed in via a TechCrunch piece in which he repeats a couple of myths that somehow bear truth only through repeated assertion.
And so the only way for the iPhone to “beat” Android would be for Apple to either open iOS up in the same way that Android is, or to create a huge variety of iPhones spread across the spectrum in terms of features and price. Neither of those things is going to happen.
My young daughter happens to enjoy Angry Birds, so recently I thought I'd grab her a little portable device to play it and her movies and music on. Being a casual mobile app developer, targeting Android for almost arbitrary reasons, I looked to see what Android devices were available so it could double as a convenient testing target.
There is nothing. The few devices I found were in the $400 range and generally featured underwhelming processors, terrible screens, and shoddy builds.
You simply can't find an Android PMP as economical as what Apple has achieved with the iPod Touch. A4 processor. Brilliant display. Great GPU. Facetime. Dual cameras. That thing is just a crazy value.
Sure, technically the iPod Touch isn't an iPhone, but for how most people use their smartphones (where the "phone" part of it is utterly irrelevant, and it's merely a mobile computing device), with the widening deployment of WiFi it is increasingly just as capable.
And make no mistake: The iPod Touch is a critical element of the iPhone's domination. We're talking platforms here, and while Google was busy actually undermining the non-smartphone devices (until recently Google treated tablets with incredible antipathy. Whatever the internal politics — perhaps the perceived birthright of ChromeOS — they were not supportive whatsoever, at one Google I/O acting both irritated and confused when an audience member asked about tablets. Nor are they supportive of PMPs where they pull market access and essentially cripple the device at birth), Apple was busy spreading iOS far and wide, creating a healthy ecosystem for developers to target.
Apple knows how to price down technology with the best of them, and they have demonstrated a strong ability and willingness to do so. The dated 3GS is still being manufactured and sold as new, just as the 3G hung around long after the 3GS became the starter.
The 3GS is incredibly dated by contemporary standards, and its slow processor, limited RAM, and old-school screen resolution place it firmly in the "low end" market. Apple still sells it. Read what Siegler had to say above and consider that.
When Apple feels pressure to segment the marketing of iOS more than they already do ("You can get an iPhone minus the 3G radio and fine-GPS for just $240 full retail, or you can get your subsidized 3GS if you're a chump and can't afford a couple hundred when signing up for a two thousand dollar contract commitment...."), they will. Any illusions that they won't, or moreso that they haven't already is just blind zealotry, lost in the weeds desperately trying to feel around for some comfort to assuage against Apple's marketshare competition.
I recently gained an interest in American football (after becoming aware that there was so much more to the game than just a bunch of guys smashing into each other, not to mention the "be prepared to explain man things" motive given that I have three young sons) and one of the things I have noticed in that live-can't-fast-forward world is that Apple advertises like crazy. From my limited sampling they are easily the overwhelmingly dominant pitchman on television, alternating between iPod Touch, iPad, and iPhone commercials. They are as mainstream as a corporation could get.
Stop your PVR fast forwarding for a while and pay attention to the commercials. Microsoft, Intel, Dell, even McDonalds...they are drown out by Apple's message.
They’re perfectly happy “at the top of the market” where they make a ton of revenue and profit. Billions more, in fact, than Google does with Android.
A recurring claim is that Apple will be pleased to retreat to their profitable little niche: That they're a purveyor of high markup, limited-run technology devices for the connoisseur, and they'll continue happily in their grey poupon ways, Android be damned.
Of course for much of Apple's history it was an incredibly unprofitable enterprise. Through the 90s, despite having their confident bevy of expensive devices and their single-digit saturation, Apple was an economic basket-case, posting billion dollar losses or cutting to the bone and barely breaking even. At one point in Steve Jobs' reign they were bailed out by Microsoft (an investment that would be worth untold riches for Microsoft today had they not quickly divested it).
The iMac started to turn that around by providing an inexpensive all-in-one computer that was novel and accessible for everyperson USA, going back to its middle class Apple II roots. My wife still berates me for not hopping on the AAPL train after correctly predicting what the iMac's impact would be.
After it, of course, the iPod stormed the market. It was a good device that was marketed right, and it laid the groundwork for the iPhone to make some major inroads on day 1. Again, the iPod sat at virtually every price point, easily winning the value race in most categories.
Cars and televisions and stereo receivers came with iPod connectors or docks. They didn't for any competing device.
Apple's profitability grew.
Apple's ability to squeeze a growing profit margin out of their business came by escaping from the niche. In the smartphone market there was a period where Apple was effectively closing on 100% of the market, which is exactly how developers and the media treated them (see the countless organizations that made an iPhone app and that was it -- they felt the consumer was served). Sure, the graphs would include Blackberries and Nokias, but come on -- Apple had the market to itself outside of things your work makes you carry and "What, you mean this thing is supposed to be a smartphone?" devices.
And like most businesses Apple is great at working the synergy. Many of their PC sales came about because they first got the user with an iPod or an iPhone. In the development world, a million developers ran out and bought an Apple equipment stack to try to write their own lottery ticket on the grand iPhone AppStore bet.
Apple has no plans on retreating to a niche. Nor was that niche ever profitable for them. Their strategy pounds home this point, with a focus that is engineering around significant if not dominant market positioning, with network lock-in that makes it difficult to challenge or break. Their application strategy, the top to bottom segmentation, and the "you're with us or you're against us" decisions like Facetime, there should be no confusion that Apple is targeting anything less than being the Microsoft of the decade.
You can't get more mainstream than Apple. Apple is no more exclusive than the GAP or Target or eating at Red Lobster.
They make great devices, no doubt about it. I've been riding the Android train for 17 months now based upon the promise of many capable companies working together, but until the latest generation of devices, such as the Desire Z, I have never felt confident in recommending the devices to anyone else. Really I'm a little surprised it survived thorugh those birthing pains.
But it did. Android today is a very decent device and it's only getting better. Pundits like Scoble (whose fame still confuses me) lament that Android isn't deserving because the apps are better on the iPhone. Apps don't appear overnight, Scoble, and the surge in developer sentiment towards Android has been overwhelming over the past months. On my device I'm getting app update notices literally daily now, as development teams rush to fix up what previously was nothing but a second thought.
It's getting better everyday.
We came periolously close to a unipolar smartphone market, and despite the pains that I had to suffer with Android during its terrible toothing years, I'm glad that we now have a market that's open for any and all. Like Firefox did for Chrome, by making multiplatform support the default Android leaves an opening for Blackberry, Microsoft, Nokia, or anyone else to come along with something better, without being crowded out by an existing and impenetrable ecosystem surrounded by an unassailable network lock-in.
The flip-side to this Android domination is what we’re already seeing. The carriers (again, in the U.S. in particular) are using Android’s openness to perform many of their same old tricks. I can’t help but think sometimes that it looks as if Google actually did the carriers a huge favor in the long-run because they’ve taken many of the bells and whistles that drove users to the iPhone in the first place and opened them up for the carriers to use as glittering lures to rope customers back into their traps.
This is the myth that I find simply extraordinary, the overwhelming evidence that it is utter nonsense being so obvious.
In the US Apple has been laying in bed with AT&T for years. Want to go to Verizon? Sorry, not an option. T-Mobile? Not an option. There is zero competition if you want an iPhone in the entire US of A.
Gave the power to the users, did it? First they came for your choice of carrier, with AT&T paying heavily into Apple's treasure for the priviledge.
Apple originally had a no app policy — as demanded of them by AT&T due to fears that apps would go crazy and take down the network — just as Facetime refuses to work over 3G, and various apps that threaten the AT&T / Apple relationship will get quietly squashed.
While Apple doesn't have NASCAR apps or similar bloatware that have very unfortunately appeared on a small minority of Android phones, every single choice on the device is made for you if you're set on an iPhone.
Compare that to Android: Pick your carrier among all available. Pick your device among literally dozens of top rank devices. Use standard consumer discretion to align interests with vendors.
Heard that Verizon stuck some bloatware and a Bing default on their phone? Okay, as a consumer you have countless alternative choices to choose instead while still using the apps you know, the data you've accumulated, and the platform that you've invested in, sending the message that it isn't viable. If Apple, on the other hand, decided that one half of your display will henceforth show nothing but ads (another ridiciulous myth is the proposition that the iPhone represents a purer transaction, which is utter tripe. Apple is locking you in as a consumer to their various media, application, and advertisement platforms), you have no options aside from simply abandoning the platform and whatever you have invested in it.
The iPhone only empowers users in a 1984-esque redefinition of the meaning of empowering. Any control it took from the carriers it placed firmly in the hands of Apple.
Some wager that the upcoming iPad 2 will pixel double both axis, similar to what the iPhone 4 did relative to its predecessor, while others believe that it will keep the resolution of the current generation.
Doubling both axis is a formidable technical challenge and would be a unique, likely expensive display. Continuing with the current resolution would represent a significant competitive disadvantage. As people acclimate to high density smartphones, such as the iPhone 4, the iPad's low density is really starting to stand out.
Few believe it will do anything in between. It won’t, the common wisdom goes, go to say 1536 x 1152 or 1280 x 860, or any other fractional improvement less than an outright doubling or quadrupling. The logic is that pixel scaling issues eliminate the possibility of such a half measure.
This harkens to discussions that occurred over 20 years ago.
It should be an embarrassment that such a discussion is occurring in 2011.
In the TiPb article linked above the author leads off with a slur towards Android, saying “Either iPad 2 will have a standard 1024×768 display or a doubled 2048×1538 Retina Display, or developers and users will be in for the type of frustration usually ascribed to Android.”
That makes for an odd, if not outright ignorant, statement: I can’t recall ever reading anyone complain about the density independent pixel of Android, or its awareness and accommodation of a wide variety of profiles. That’s a problem that it has solved very well, and a large ecosystem of sizes and resolutions of displays exist in remarkable harmony.
Consumers like being able to choose between 3” – 15”+ devices with a wide variety of densities. Choice is good.
Because of course the DPI issue has long been solved. Otherwise you would be lamenting that your 72dpi word processor isn’t compatible with your 300dpi printer: “Everything prints out all tiny-like”. Is that the case?
Vector fonts with pixel independent abstractions have been around for a long time (in TrueType and Postscript form), with Apple as one of the primary inventors. Most GUI frameworks, including iOS, have the ability to scale UI rudiments to virtually any resolution and pixel density with ease.
That is an ancient problem, long solved.
But what about icons? What about bitmap graphic artifacts?
In an ideal world icons would come in vector graphic form. That isn’t the case on Android (the platform doesn’t support SVG, including in the browser, which is a huge deficiency), but it is still shocking that Apple, which usually takes the lead on such innovations, doesn’t use them for iOS, as had been widely speculated as a given before the iPhone OS was first released.
With a vector graphic the rendered image is always perfect for the target, ideally with hints that suppress decorations at very low sizes.
Even with bitmap graphics, however, while it’s easy to contrive ridiculous examples to demonstrate the worst of scaling, the reality is that given that text should always be UI generated from vector fonts, perfect for the target, and graphics are usually just supplementary decorations, where scaling up or down by partial multiples is often perfectly adequate.
For your consideration below are some iOS icons (used for fair use purposes but owned by Apple) at their original pixel size, and then scaled to 125% and 150%. Scaling was done using Sinc (Lanczos3), which is a good algorithm to use when scaling up and you want to maintain fine detail.



The horrors! Just to be clear (as it's hard to imagine what the larger images would look like when shown in the same physical space), we're comparing this to simply pixel-doubling, which would look like the following (cropped to avoid exceeding most reader's screen bounds).

There is no universe where a straight pixel-doubled image looks better than an interpolated image, unless you have fine detail in the image (like text) which shouldn't be in the image to begin with.
Not only do they still look great, but remember that in such a case the actual viewed sizes would also decrease proportionally, so the marginal artifacts would be rendered completely irrelevant. Reading some of the blog entries on scaling you would think you’d end up with some sort of blob.
Not to mention that most iPad apps would be fixed up to handle the new platform shortly after the SDK were released.
I find it incredibly hard to believe that Apple will maintain the same resolution. The device was already considered under-pixeled when first released, and has succeeded despite that deficiency. With the appearance of actual competition over the coming months it would put the successor at a serious disadvantage out of the gate. Apple has always emphasized and often led in the spec department (the thinnest, lightest, brightest screen, widest viewing angle, highest pixel density, etc), so I don’t believe that Apple would allow such a scenario.
At the same time, doubling the pixels is a big expense because such a panel doesn’t yet exist, and that sort of density is rare in larger displays.
Then again, Apple, in talking about their blockbuster quarter yesterday, spoke about a mysterious $3.9 billion dollar supply chain investment. It would not surprise me in the least if they managed to get the volumes and process in place for such a display given how critical the iPad has become to their bottom line, and Apple's tendency to set new benchmarks.
If I had to place a wager, however, I would bet that Apple neither keeps their current resolution, nor will they double the resolution. I would guess that they will go to 1536 x 1152, or more likely 1280 x 960.