Return to Main Page | Ragout: The Best National Polls Trump the State Polls
A Spicy Stew of Economics, Politics, Data, Food, Carpentry, etc.
Thursday, October 14, 2004

The Best National Polls Trump the State Polls

The web is swimming in state-level polls this year. The spiffyest is electoral-vote.com, but they're everywhere. Slate has a state poll map, as does Real Clear Politics, MyDD, the Wall Street Journal, Pollkatz, and no doubt many others. Sam Wang at Princeton does some complicated calculations combining all the polls (he currently gives Kerry a 27% chance of winning).

The state polls are a lot of fun. But I think the national polls are the ones to watch.

In theory, state-level polls should be better. Even though each one typically has about half the sample size of the national polls, their combined sample size is big. And, after all, it's state electoral votes that determine the President (at least usually). But in polling, having a truly representative sample usually outweighs sample size. Overall, the state polls have weaker methodologies than the best of the national polls, and I suspect their samples are less representative.

Five polling companies put out most of the state polls: Rasmussen, Survey USA, American Research Group, Zogby Interactive, and Strategic Vision. All have flaws. Strategic Vision and ARG put out very little information about their methodology, which is a big strike against them in my book. Strategic Vision is a Republican consulting firm, which also raises warning flags.

Zogby interactive, the state poll reported in the WSJ, is an internet poll of a panel selected by Zogby to be representative. (Zogby puts out a national telephone poll too). Zogby claims that 90% of likely voters are on-line and that their weighting allows them to produce results similar to traditional telephone polls, but still this is still a new, relatively untested methodology.

Rasmussen and SUSA are automated "robo-polls," that use a recorded voices and accept touch-tone responses. Here's SUSA's description:
SurveyUSA surveys are conducted in the voice of a professional announcer. SurveyUSA is the first research company to appreciate that opinion research can be made more affordable, more consistent and in some ways more accurate by eliminating the single largest cost of conducting research, and a possible source of bias: the human interviewer.
It goes on, and is fairly convincing, as far as it goes. Rasmussen says pretty much the same, arguing that their recorded voice is actually better than old-fashioned humans.

What neither mention is how they sample within households. The best polling firms take pains to choose someone at random, for example by asking to speak to the adult with the most recent birthday. Neither Rasmussen nor SUSA claim to do this, and I highly doubt that they do. They just "speak" with whoever answers the phone.

Second, SUSA gets a response rate of about 10%, probably because people are more likely to hang up on a robo-voice than a human being. Rasmussen doesn't say, but their response rate is probably similar. For comparison, the best of the national polls get response rates of 25-30%.

Finally, to make up for their less representative samples, all three firms have to weight more intensively than many national polls. In particular, Zogby and Rasmussen weight by party ID. SUSA doesn't say, but my guess is that they do too. Although this methodology has some supporters, the weight of opinion (including mine) is against it.

Since I think the big national polls -- ABC/WaPo (but not their tracking poll), NBC/WSJ (but not their state polls), Democracy Corps (despite their partisanship), CBS/NYT, Gallup/CNN/USA Today, Fox, Pew, and the LA Times -- have the most representative samples, I'll take them over the state polls any day.


Number 1 in Ragout Economics!

March 2004 / April 2004 / May 2004 / June 2004 / July 2004 / August 2004 / September 2004 / October 2004 / November 2004 / December 2004 / January 2005 / April 2005 / May 2005 / June 2005 / July 2005 / August 2005 / September 2005 / October 2005 /

First Team
Angry Bear
Crooked Timber
Brad DeLong
Economist's View
Mark Kleiman
Nathan Newman
Political Animal
Max Sawicky
Brian Setser
Sock Thief
Talking Points Memo
Matthew Yglesias

Second Opinion
Stephen Bainbridge
Marginal Revolution
Andrew Samwick
The Volokh Conspiracy

Third Way

Fourth Estate
Economic Reporting Review
New York Times
Washington Post

Fifth Republic
Le Figaro
Le Monde

Sixth Sense
The Intersection
In the Pipeline
What's New

Politics & Polls
Daily Kos
Donkey Rising
Electoral Vote Predictor
Rasmussen Tracking Polls

Art Sucks
Enzo Titolo
L’esprit d’escalier
A Level Gaze
Approximately Perfect

ragoutchef at yahoo dot com


Powered by Blogger