Return to Main Page | Ragout: Cell Phones Killing Polls?
A Spicy Stew of Economics, Politics, Data, Food, Carpentry, etc.
Saturday, September 18, 2004

Cell Phones Killing Polls?

Via electoral-vote.com, I learn that Jimmy Breslin, a great reporter and schmoozer, but not such a great statistician, has proclaimed the death of polls:

Anybody who believes these national political polls are giving you facts is a gullible fool.

Any editors of newspapers or television news shows who use poll results as a story are beyond gullible. On behalf of the public they profess to serve, they are indolent salesmen of falsehoods.

This is because these political polls are done by telephone. Land-line telephones, as your house phone is called.

The telephone polls do not include cellular phones. There are almost 169 million cell phones being used in America today... There is no way to poll cell phone users, so it isn't done.

He's wrong about the last point: the reason why pollsters don't call cell phones is because it's illegal, just as it's illegal for telemarketers to call cell phones.

Breslin is right that cell phones present a problem, but they don't present a very big problem, or even a very new problem. As of February of this year, about 6% of households have dropped land lines in favor of cell phones, according to the FCC [pdf; page 2, footnote 2]. That means they can't be reached by telephone polls. But this isn't anything new: 20 years ago 8% of households didn't have telephones at all. Even today, 6% still don't have either a land line or a wireless phone.*

More generally, cell phones are just a new reason for low response rates. In any poll, pollsters never contact most of the people they set out to, even though these people have land lines. People use caller ID or answering machines to screen their calls, they hang up on pollsters, or they're just never home when the pollsters call.

Response rates have been dropping over time. Today, most telephone polls have response rates of less than 30% [pdf]. Response rates of 10% [pdf] are common. This sounds like an awful problem, but there are numerous studies that show that response rates don't actually affect the results that much. Samples with low response rates remain pretty representative demographically (in terms of race, age, sex, and education) . (See this brief review of the literature by ABC's director of polling, and this brief review by two high-powered academics, presumably consulting for the British polling firm YouGov).

Even if, say, young people are less likely to be found by pollsters, because they only have cell phones or because they're always out partying, or whatever, that's not enough to conclude that the polls are biased. Weighting the poll data to compensate for any unrepresentive demographics can always adjust for it. To have problems, the young people with cell phones instead of land lines also have to have different voting patterns than other young people. This is possible, but I don't see any obvious reason why this should be true.

Casual evidence also suggests that low response rates don't matter much. Overnight polls get pretty much the same results as polls taken over longer periods (which get higher response rates because they call people back multiple times). And, as I pointed out in a post yesterday, election eve polls are typically spot on, at least within a couple percentage points of the final result.

So, adding cell phones to the mix could turn out to be the final burden that breaks the camel's back. But they're really just one of many things that make pollsters lives difficult, and all the other problems haven't caused polls to become worthless so far. It's definitely too early to throw out the polls and go with the Breslin plan of just making stuff up. "Common sense would say that the majority of the 18 to 25 who do vote would vote for the Democrat," he writes. Although common sense would certainly be a much cheaper way of gauging opinions, I'm pretty sure that polls remain a lot more reliable.

* FCC, [pdf] table 1, page 6. My guess is that some cell phone only households are answering the survey question wrong and the true percentage of households without any phones at all is about 5%.

Addendum: You can also read the response of pollster John Zogby, who Breslin misquoted, on Rox Populi.

Number 1 in Ragout Economics!

March 2004 / April 2004 / May 2004 / June 2004 / July 2004 / August 2004 / September 2004 / October 2004 / November 2004 / December 2004 / January 2005 / April 2005 / May 2005 / June 2005 / July 2005 / August 2005 / September 2005 / October 2005 /

First Team
Angry Bear
Crooked Timber
Brad DeLong
Economist's View
Mark Kleiman
Nathan Newman
Political Animal
Max Sawicky
Brian Setser
Sock Thief
Talking Points Memo
Matthew Yglesias

Second Opinion
Stephen Bainbridge
Marginal Revolution
Andrew Samwick
The Volokh Conspiracy

Third Way

Fourth Estate
Economic Reporting Review
New York Times
Washington Post

Fifth Republic
Le Figaro
Le Monde

Sixth Sense
The Intersection
In the Pipeline
What's New

Politics & Polls
Daily Kos
Donkey Rising
Electoral Vote Predictor
Rasmussen Tracking Polls

Art Sucks
Enzo Titolo
L’esprit d’escalier
A Level Gaze
Approximately Perfect

ragoutchef at yahoo dot com


Powered by Blogger