Return to Main Page | Ragout: Shooting the Pollster
A Spicy Stew of Economics, Politics, Data, Food, Carpentry, etc.
Monday, September 06, 2004

Shooting the Pollster

Every time a poll comes out with bad news for one side or another, partisans leap to denounce it. A few months ago Bush's pollster rushed to denounce a LA Times poll that showed a big Kerry lead. A few days ago, a Time magazine poll came out that showed Bush with a 9 point lead among registered voters. Many were quick to condemn it, including Atrios and numerous posters on DailyKos.com, who charged that the poll was "massaged." Atrios claimed that the poll was "biased in favor of getting a male respondent," because the poll first asked to speak with the youngest male in the household, and only asked to speak with the oldest female if no male was available.

Then the next day, a Newsweek poll came out that showed Bush with an even bigger lead, 11 points, leading to some embarrassment for the critics.

Now, I do think that the critics were right on the big point. The Time poll was taken during the Republican Convention, and the Newsweek poll during the final night of the convention and the following day. It seems quite possible that Republicans were more likely to home that day, and more interested in talking to pollsters. Newsweek released enough information about the poll's respondents to suggest that they did in fact skew Republican. Ruy Teixeira at Donkey Rising has a good discussion.

But criticizing the poll's methodology is just silly. Both polls actually seem better than average to me, because they release more information than usual about the composition of the sample: that's why the critics can criticize. As Teixeira says,
What I do favor is release and prominent display of sample compositions by party ID, as well as basic demographics, whenever a poll comes out. Consumers of poll data should not have to ferret out this information from obscure places--it should be given out-front by the polling organizations or sponsors themselves. Then people can use this information to make judgements about whether and to what extent they find the results of the poll plausible.
It's this bottom line that matters: whether or not the pollsters managed to draw a random sample.

Oh, and Atrios' complaint about a male-skewed sample? Although the pollster's methodology did sound odd to me too, it turns out to be perfectly standard. The Pew Research Center--who I think are particularly reliable, because they're non-partisan, quasi-academic, and release their data--use this same technique of asking first for a male. Women are more likely to be home, so pollsters try harder to get males, in order to produce a gender-balanced sample.


Number 1 in Ragout Economics!

March 2004 / April 2004 / May 2004 / June 2004 / July 2004 / August 2004 / September 2004 / October 2004 / November 2004 / December 2004 / January 2005 / April 2005 / May 2005 / June 2005 / July 2005 / August 2005 / September 2005 / October 2005 /

First Team
Angry Bear
Crooked Timber
Brad DeLong
Economist's View
Mark Kleiman
Nathan Newman
Political Animal
Max Sawicky
Brian Setser
Sock Thief
Talking Points Memo
Matthew Yglesias

Second Opinion
Stephen Bainbridge
Marginal Revolution
Andrew Samwick
The Volokh Conspiracy

Third Way

Fourth Estate
Economic Reporting Review
New York Times
Washington Post

Fifth Republic
Le Figaro
Le Monde

Sixth Sense
The Intersection
In the Pipeline
What's New

Politics & Polls
Daily Kos
Donkey Rising
Electoral Vote Predictor
Rasmussen Tracking Polls

Art Sucks
Enzo Titolo
L’esprit d’escalier
A Level Gaze
Approximately Perfect

ragoutchef at yahoo dot com


Powered by Blogger