Free Markets, Free People

About all those polls… (Updated)

If you listen to the podcast, you may have noticed that, over the past couple of weeks, we’ve talked a lot about polling, and why Obama is doing so well. We’re not the only ones. A lot of people are wondering why Obama is polling well when the things are so bad. One of the criticisms I’m seeing about a lot of the polls is that they skew so heavily democratic. Except for Rasmussen, almost all of the polls coming out seem to have larger numbers of Democrats than one would expect. They have been as high as a D+11% advantage in the population.

This is seen by some as proof that the pollsters are skewing the respondent population towards Democrats. I’m not impressed by the argument, because most pollsters don’t actually try and set up a likely voter model for the poll. Instead, the poll is a sample of usually between 1,000 and 1,500 randomly selected voters. The Democratic advantage in this poll, therefore, is not an artifact of the selection method, but is actually the result of what the respondents identify themselves as. If you call 1,000 people, and 380 of them say they’re Democrats, then that’s the sample.

The poll, then, reports what the respondents say. It’s not the result of selecting a particular number of Democrats or Republicans. That’s a vitally important distinction, because voter identification changes over time. The poll reports what voters say their party affiliation is, but a voter may say he’s a Democrat this week, and a Republican or Independent two weeks from now.

So, the key here, it seems to me, is to look at a set of polls from a particular pollster and see if the party affiliation varies widely from poll to poll. If it does, then there’s probably a problem with their methodology. You might see a shift in party affiliation over time, but the change between consecutive individual polls should probably be fairly small. But in general, if a pollster uses the same methodology for every poll, and is not explicitly looking to create a voter response model, then the results are probably fairly accurate, and show small movements–if they occur–to party identification from poll to poll.

What I’m hearing from a lot of conservatives this week is the idea that the polls are horribly skewed, as if there’s some industry-wide conspiracy to make Obama look good. That doesn’t seem very likely, especially since nearly every pollster uses a bipartisan polling team, i.e. one Democrat and one Republican. So, what I’m hearing from conservatives sounds like the response Democrats made in the 2004 election, when John Kerry was polling badly. Then, as now, there was this feeling that the polls were horribly wrong, and their candidate wasn’t actually losing. But the losing candidate was, in fact, losing.

So if the polls are off, then it must be the result of either a gross, industry-wide incompetence that is causing them all to use a faulty methodology, or a gross, industry-wide conspiracy–between both Republican and Democrat pollsters–to push a pro-Obama narrative. The alternative is that the polls aren’t off, and within a 3% or so margin of error, are reporting accurately what the electorate is saying. The latter seems to me to be far more likely.

Now, as to why so many voters are identifying as Democrats, I don’t have a clue. But consider this: pretty much everyone knows Bill Clinton is smarmy liar, and if he could run for a 3rd term…he’d win.

Also, consider that everyone remembers the Bill Clinton presidency as a time of economic growth and balanced budgets. They remember the end of Bush’s two terms as a time of complete economic collapse. The underlying reasons don’t matter, because most voters neither understand nor care. It may be that voters simply trust Obama more on the economy than they do Romney, because they fear a return to economic collapse. Maybe they think Obama has done as well as could be done. But simply dismissing that with a "the polls can’t be right" explanation is just whistling past the graveyard.

UPDATE: More here, including this graphic.


Now, let’s split this out and look at correlation:


That’s a pretty weak correlation. Look at the blue diamonds for the Obama lead. What is that, a bell curve? Seriously?

No, unless the poll makes a specific effort to model a voter turnout, and specifically samples for a given percentage of R-D-I, then the poll is just telling you what the respondents are telling the pollsters. They may tell them something different next week or next month, but the R-D-I sample is simply a result of respondent self-identification.

Dale Franks
Google+ Profile
Twitter Feed

Tweet about this on TwitterShare on FacebookShare on Google+Share on TumblrShare on StumbleUponShare on RedditPin on PinterestEmail this to someone

32 Responses to About all those polls… (Updated)

  • They know where they can poll to get essentially the sample they are looking for.   I remember enough of college statistics for that.
    Not sure why you think they are any less likely to be in the tank than entire news networks or news paper organizations yet there are undeniable tilts by both groups (and the tilt goes either way).  I hardly think polling organizations are necessarily any less likely to hire/retain according to self selecting bias.

  • Watch and see if in the last few days before the election if their “results” don’t shift 180 degrees.
    And remember that in 1980 at this point, Carter was up by 7-9%, but we didn’t have two generations of publik skool sludge in the voting booths.

  • as if there’s some industry-wide conspiracy to make Obama look good.

    Such as the Mushroom Media? Nah!, just happenstance!

  • Fair, but there’s LOTS of contradictory info flying around, such as the general party identification, and “enthusiam gap” etc. It’s my sense that Obama is a drop ahead overall, including some of the swing states. I don’t believe he’s running away with anything. I also believe that GOP turnout will be at 2010 levels. Dem turnout remains to be seen. If they approximate 2010 levels, they’re toast. If they approximate 2008….well that’s gonna be a rough night for the GOP.

    • Obama doesn’t have to run away with anything. He only needs to be 1 vote ahead in VA, OH, or FL, and the race is his. Romney has to run the table in those three states.

  • There’s another reason the pollsters might be biased, and it has nothing to do with politics. It’s money.
    Pollsters get paid for polls. It’s what they do. It is also a fact that in a close race there’s a lot more polling than if it’s not close. For proof, look at the number of polls in Ohio compared to any hard red or hard blue state.
    So, a close race is very much in the pollster’s financial interests to show.
    Oh, they have to be concerned about accuracy, and I’m sure they will be by about two weeks before the election (the election being the one and only way of measuring their accuracy).
    So, is Romney ahead or behind? The hell if I know… but I do know that it is in the pollster’s best interests to show a tight race, even when it’s not.

  • Supposedly they should be taking the Democrat responses and Republican responses as well as other factors and re-weighting various subgroups in the poll to reflect the voting demographic.  And if they are doing that and they are going by same incorrect ‘voting demographic’, they will all be wrong.
    Otherwise the propaganda of the ‘old media’ is even more effective than before and things like this blog are simply a cathartic release and accomplish nothing in terms of swaying voters.  And basically NBC and NYT run this country.

  • Supposedly they should be taking the Democrat responses and Republican responses as well as other factors and re-weighting various subgroups in the poll to reflect the voting demographic.

    Actually, I think RAsmussen is about the only one that does that.

  • 1) There could be systemic reasons for the bias. For example, we know that Dewey would win because telephone polls said so…except people reachable by telephone happened to not be representative of the voting populace, and Truman won.
    So you can easily come up with scenario after scenario where the pollsters, despite their best efforts, could have a skewed pool.
    Say, who answers the phone and responds to a poll now? The unemployed person who is bored or the busy exec?
    2) The people being polled do not want to be known to be anti-Obama. Me, personally, I never, ever want to be known as a non-Democrat in casual life. I have had way too many people talk to me and assume I was a Democrat. I don’t want to argue politics with people, etc. Society and the media seem to be mainly left-leaning. Do you want to go up against that? Ask economists what happens in polite company when they mention they are against the minimum wage.  A subset may be people who don’t want to be against the first black president. Hell, I had the power company call me about their green programs and I bet a whole lot of people lied about what they really cared about – cost or saving the planet.
    3) There are millions of people now dependent on the government. These people must know that the gravy train will end, and its not really good gravy, but what the hell is a 57 year old roofer going to do when Romney cuts his SSI? Normally these people would vote for change, but they don’t see change as helping them. They figure, rightly, their job is gone and their skills are useless. Best keep the borrowing going as long as possible. Head in the sand is the best path for them.
    4) We have to give the first black president another term or we will be racist. I think we on the right discount this vague feeling that we have to give him an extra special chance to succeed. Keep in mind the low information voter has NO clues about how bad things are and how little Obama has done. They lap up stories about the GOP “obstructing” budgets, etc.
    1&2 would say the polls could be wrong.
    3&4 say the polls are right and explain why.
    We should work hard to make sure that Romney has a shot at winning, because he obviously does. But if he does not, we will slide promptly into recession, Obama won’t be able to do much, and people will blame him then, I think. Once we get past the “second chance” we will see more people start being honest. Oh, maybe not in 2012…but in 2014-2015 its going to be brutal.

    • Note also that Romney is hamstrung in overcoming 3&4. Those are very hard obstacles to overcome with even detailed policies.

    • If Obama is reelected there is no 2016 election. At least none that makes any difference. This is a tipping point election. Obama will make us a fascist welfare state with a second term, We will be under control and there will be hyperinflation and large double digit unemployment. There will be no coming back from that short of a revolution.

  • I believe that the polls are skewed by a self selection bias. Pew recently published a report that stated that in 1997 they got responses from calls of 36% in 1997 but only 9% in 2012.
    Many people are refusing to participate. I, myself, have hung up on at least 10 pollsters this year. Consider that the exit polling from the Wisconsin recall election showed Walker losing badly. He won handily. This disparity was explained as self selection on the part of the interviewees.
    If the pollsters are relying on self identification of participants as to party affiliation and they are largely Democrats then perhaps this mirrors the faulty methodology of the recall exit polling.
    Additionally, it is assumed by most pollsters that you get a different mix in mid-term elections so they base their turnout models on 2008 instead of 2010. Since nothing has improved the conditions of 2010 I would expect that 2012 will look about the same as 2010.
    I’ve been watching these rodeos since 1952. My gut tells me Romney by 7-10%.

    • I think that many people connect the polling companies with the media companies and are disgusted and its a waste of time.
      People don’t want to play a game that is run by the media.

      • I think there’s waaaaay more in that statement than you’re going to get credit for.
        As to the “republicans” in the polling company – “Morning Joe”, he’s a conservative guy, right?  I’m trying to imagine my rabid Republican neighbor to my left, and my rabid Democrat neighbor to my right working together on a poll.   It would be like working in old Beirut (not the really old peaceful one, the one that had junk car walls set up to keep the snipers from popping people while grocery shopping).
        Just saying.   Dale could be 100% right about this, and we’re just whistling past the corpse depot and big ears will be elected again come November 7th.  And whatever happens after that happens.

    • I think Obama will win by a squeaker due to the media and then we will have to hear whining about the GOP Congress for the next 4 years along with mounting evidence of Obama phoning it in.
      The low information voters will only get this when things start shifting rapidly. “The Chinese are selling dollars?” or “We have to make cuts…wait didn’t Obama say that would not happen?”
      In reality, a parliamentary system would be much better in these situations to let pols be responsible and not try to pass off their problems. Instead, we have an electorate that believes the President is magical, but not too magical, and that free stuff should flow if it weren’t for those math witches.

  • So you feel the recall polls in Wiscn. have no bearing on the 2012 situation?  what is your reasoning there?

  • Actually, I think RAsmussen is about the only one that does that.
    Doesn’t Rasmussen have the best track record the last several of elections, including correctly calling for 60+ seat gain in the mid-term elections? I’m not saying that all polls I don’t like are skewed, but it seems to me that while past results may not me indicative of future results, it’s at least worth looking at how people have done in the past.

  • Let me just offer a hypothesis (which is likely testable):  Voters who lean conservative are less likely to answer the phone when the caller ID indicates that the caller represents a political survey.  If this is true, then conservatives would be undersampled.
    And why might conservatives be less likely to answer the phone?  Because our time is like private property, and we are more likely to guard it against would-be intruders; and because we are suspicious of polling.
    Again, I am hypothesizing.
    —Tom Nally, New Orleans

  • Very depressing.
    If he gets voted in then this pretty much put paid to the end of the union.  There was a small chance for recovery if he did not get in , absolutely none if he does.
    Maybe the great divide between the (re)gressives and the regular people is too much and it is time for some areas to go alone and find a better way for themselves.

  • O Dear, breach in the echo chamber detected.  Quick, someone page Fail Dranks 😉

  • Looking at the charts, it seems to me that Obama’s lead is consistently lower then the Dem sample advantage.

    But the Dem sample advantage in many polls is greater then it was in reality in 2008. I’m pretty sure that isn’t going to happen.

  • Rasmussen is showing Romney up, while Gallup gives Obama a 1 point lead. The Obama bounce is gone.

    • So now we trust polls again! Yay!

      • I’ve always trusted Rasmussen, when it shows Obama ahead it worries me.

        I think it is a near tie, which is dissapointing, because Obama should be losing by 70/30 at least. I’m not just looking towards a Romney victory, but a massive victory which aqppears unlikely. This should be total defeat for the Dems all along the board, and if that does not happen it means we don’t have what it takes to fix things.

  • Conservatives have always sought to discredit polls—in one way or another—when the results are not to their liking.

    • Have they? You mean like when they were saying the polls were all wet in 2010 and that the Dems were going to lose more than 25 seats in the House? In fact, as stated here, they’d lose 60? Hmm … maybe they have a point.

    • As opposed to liberals, who believe everything they read, eh?

  • Isn’t the discrepancy between the party identification in the poll sample and the party identification in registration?  I’m sure I’ve seen analyses explaining the Obama poll advantage against the overall sample population as officially recorded at the voter registar offices.

    Hence, the sample of respondants did not match the registration by party.

    Looks like a systemic error in the polling method similar to the one in Truman vs Dewey.  I’d go with the “conservatives don’t answer pollster’s calls” theory myself.  I know I never answer my land line – might be a bill collector!