Cell Phones Killing Polls?
, I learn that Jimmy Breslin, a great reporter and schmoozer, but not such a great statistician, has proclaimed the death of polls:
Anybody who believes these national political polls are giving you facts is a gullible fool.
Any editors of newspapers or television news shows who use poll results as a story are beyond gullible. On behalf of the public they profess to serve, they are indolent salesmen of falsehoods.
This is because these political polls are done by telephone. Land-line telephones, as your house phone is called.
The telephone polls do not include cellular phones. There are almost 169 million cell phones being used in America today... There is no way to poll cell phone users, so it isn't done.
He's wrong about the last point: the reason why pollsters don't call cell phones is because it's illegal
, just as it's illegal for telemarketers to call cell phones.
Breslin is right that cell phones present a problem, but they don't present a very big problem, or even a very new problem. As of February of this year, about 6% of households have dropped land lines in favor of cell phones, according to the FCC
[pdf; page 2, footnote 2]. That means they can't be reached by telephone polls. But this isn't anything new: 20 years ago 8% of households didn't have telephones at all. Even today, 6% still don't have either a land line or a wireless phone.*
More generally, cell phones are just a new reason for low response rates. In any poll, pollsters never contact most of the people they set out to, even though these people have land lines. People use caller ID or answering machines to screen their calls, they hang up on pollsters, or they're just never home when the pollsters call.
Response rates have been dropping over time. Today, most telephone polls have response rates of less than 30%
[pdf]. Response rates of 10%
[pdf] are common. This sounds like an awful problem, but there are numerous studies that show that response rates don't actually affect the results that much. Samples with low response rates remain pretty representative demographically (in terms of race, age, sex, and education) . (See this brief review
of the literature by ABC's director of polling, and this brief review
by two high-powered academics, presumably consulting for the British polling firm YouGov).
Even if, say, young people are less likely to be found by pollsters, because they only have cell phones or because they're always out partying, or whatever, that's not enough to conclude that the polls are biased. Weighting the poll data to compensate for any unrepresentive demographics can always adjust for it. To have problems, the young people with cell phones instead of land lines also have to have different voting patterns than other young people. This is possible, but I don't see any obvious reason why this should be true.
Casual evidence also suggests that low response rates don't matter much. Overnight polls get pretty much the same results as polls taken over longer periods (which get higher response rates because they call people back multiple times). And, as I pointed out in a post yesterday, election eve polls are typically spot on, at least within a couple percentage points of the final result.
So, adding cell phones to the mix could turn out to be the final burden that breaks the camel's back. But they're really just one of many things that make pollsters lives difficult, and all the other problems haven't caused polls to become worthless so far. It's definitely too early to throw out the polls and go with the Breslin plan of just making stuff up. "Common sense would say that the majority of the 18 to 25 who do vote would vote for the Democrat," he writes. Although common sense would certainly be a much cheaper way of gauging opinions, I'm pretty sure that polls remain a lot more reliable.
, [pdf] table 1, page 6. My guess is that some cell phone only households are answering the survey question wrong and the true percentage of households without any phones at all is about 5%.
Addendum: You can also read the response of pollster John Zogby, who Breslin misquoted, on Rox Populi.