The Best National Polls Trump the State Polls
The web is swimming in state-level polls this year. The spiffyest is electoral-vote.com
, but they're everywhere. Slate
has a state poll map, as does Real Clear Politics
, the Wall Street Journal
, and no doubt many others. Sam Wang at Princeton
does some complicated calculations combining all the polls (he currently gives Kerry a 27% chance of winning).
The state polls are a lot of fun. But I think the national polls are the ones to watch.
In theory, state-level polls should be better. Even though each one typically has about half the sample size of the national polls, their combined sample size is big. And, after all, it's state electoral votes that determine the President (at least usually). But in polling, having a truly representative sample usually outweighs sample size. Overall, the state polls have weaker methodologies than the best of the national polls, and I suspect their samples are less representative.
Five polling companies put out most of the state polls: Rasmussen, Survey USA, American Research Group, Zogby Interactive, and Strategic Vision. All have flaws. Strategic Vision
put out very little information about their methodology, which is a big strike against them in my book. Strategic Vision is a Republican consulting firm, which also raises warning flags.
, the state poll reported in the WSJ, is an internet poll of a panel selected by Zogby to be representative. (Zogby puts out a national telephone poll too). Zogby claims that 90% of likely voters are on-line and that their weighting allows them to produce results similar to traditional telephone polls, but still this is still a new, relatively untested methodology.
are automated "robo-polls," that use a recorded voices and accept touch-tone responses. Here's SUSA's description:
SurveyUSA surveys are conducted in the voice of a professional announcer. SurveyUSA is the first research company to appreciate that opinion research can be made more affordable, more consistent and in some ways more accurate by eliminating the single largest cost of conducting research, and a possible source of bias: the human interviewer.
It goes on, and is fairly convincing, as far as it goes. Rasmussen says pretty much the same, arguing that their recorded voice is actually better than old-fashioned humans.
What neither mention is how they sample within households. The best polling firms take pains to choose someone at random, for example by asking to speak to the adult with the most recent birthday. Neither Rasmussen nor SUSA claim to do this, and I highly doubt that they do. They just "speak" with whoever answers the phone.
Second, SUSA gets a response rate of about 10%, probably because people are more likely to hang up on a robo-voice than a human being. Rasmussen doesn't say, but their response rate is probably similar. For comparison, the best of the national polls get response rates of 25-30%
Finally, to make up for their less representative samples, all three firms have to weight more intensively than many national polls. In particular, Zogby and Rasmussen weight by party ID. SUSA doesn't say, but my guess is that they do too. Although this methodology has some supporters, the weight of opinion
(including mine) is against it.
Since I think the big national polls -- ABC/WaPo (but not their tracking poll), NBC/WSJ (but not their state polls), Democracy Corps (despite their partisanship), CBS/NYT, Gallup/CNN/USA Today, Fox, Pew, and the LA Times -- have the most representative samples, I'll take them over the state polls any day.