A Level Gaze

A Level Gaze

"What effect must it have on a nation if it learns no foreign languages? Probably much the same as that which a total withdrawal from society has upon an individual."
--G.C. Lichtenberg

Thursday, October 12, 2006

Statistics Illustrated

Lindsay's got a great post up in which she lists some of the more prominent and ridiculous objections to the Johns Hopkins University-Lancet study that estimates that somewhere in the neighborhood of 655,000 Iraqis have died as the result of the American invasion.

I suspect some, maybe most, of the bloggers she enumerates know better, and are just trying to muddy the water to dilute the gut-wrenching horror they believe they helped precipitate. Many Americans, sadly, don't have much of a grasp of statistics, and, not wanting to believe their government has the blood of 655,000 men, women, and children on its hands, will fall for the argument that it's impossible to conclude such a thing from a mere 547 known deaths.

So, in the spirit of public service, here is an illustration I think will help:

Imagine that you are in charge of Best Buy's main warehouse. You're gearing up for the holiday season, and are receiving huge quantities of gadgets and gizmos. You have just received a million units of a hot new laptop that management thinks is going to be a huge seller this year. To keep expensive customer service and repairs to a minimum, you need to make sure the laptops aren't duds. This is especially important in the case of a new product with no track record.

You have 10,000 pallets of 100 laptops each. You instruct your crew to pick out 50 units, each from a different pallet, plug them in and boot them up. The guys extract the laptops, take them over to the testing area and get to work.

About 20 minutes later, one of the crew runs over to you, and says that smoke started coming out of one of the machines and that the guy who was working on it got a nasty shock. You think for a minute and ask him if the rest of the machines are running. "Yeah," he says, "the one that died was one of the first 10 we booted. It had been running about 15 minutes."

It could have been a fluke, right? Dud units are a fact of life in the electronics business. Besides, this is a lot of expensive inventory, and there would be no way to get more in time for the big rush. "Get everyone to a safe distance," you tell him, "and let's see what happens." Over the next half hour, three more laptops go up in smoke.

Well, you think, only 4 units died. Profit on these things is $200 each--$200,000,000 to the company's bottom line. Not to mention the hassle of shipping these things back to Taiwan and the fight to get reimbursed for the machines and for lost profits. Could take years. A headache like that could cost you your job.

On the other hand, though, if more than a small number of customers have these things die on them, it's going to be a customer service nightmare. That smoke has got to be toxic, and then there's the shocking. Both of those could expose the company to serious lawsuits. More than a few of those, and the hit to Best Buy's reputation would cost a lot more than $200 million.

So, how big a problem are you looking at?

You know that 4 out of 50 laptops were smokers. That's 8%. Out of a million, that's 80,000. If that many found their way into people's homes, Best Buy would go out of business. Hell, management would probably blame you. It's possible you could be charged with criminal negligence.

You're in charge. What do you do? Even if you don't know anything about statistics, it's obvious that the one thing you don't do is ship the laptops to the stores. And you know this from only four defective units.

The real number of smokers could be 80,000; it could be 140,000 or 3,000. It's even (extremely) remotely possible that it's only four. The more units you test, the closer you will get to knowing for sure. But, even from that first 50, there's enough to go on for you to singlehandedly put a huge dent in Best Buy's bottom line.

The moral here is that a lot can be inferred about populations from relatively small samples. The math that determines how reliably these samples predict larger populations is solid and has been exhaustively tested. It's still possible that something in the design of the survey caused the projection to be significantly off, but it isn't the math.

posted by David at 10/12/2006 09:43:00 PM ::

HOME

Links