As you weigh the results of various polls in the coming couple of months, this might be a handy tool to use when considering their credibility (via Vox Populi). The following list of polls, from a Fordham University study, is in order based on their accuracy last election (2008). Rasmussen and Pew were the only one’s that were spot on. The rest, to varying degree, missed it, either by an inch or a mile (at least in terms of polling):
1T. Rasmussen (11/1-3)**
1T. Pew (10/29-11/1)**
3. YouGov/Polimetrix (10/18-11/1)
4. Harris Interactive (10/20-27)
5. GWU (Lake/Tarrance) (11/2-3)*
6T. Diageo/Hotline (10/31-11/2)*
6T. ARG (10/25-27)*
8T. CNN (10/30-11/1)
8T. Ipsos/McClatchy (10/30-11/1)
10. DailyKos.com (D)/Research 2000 (11/1-3)
11. AP/Yahoo/KN (10/17-27)
12. Democracy Corps (D) (10/30-11/2)
13. FOX (11/1-2)
14. Economist/YouGov (10/25-27)
15. IBD/TIPP (11/1-3)
16. NBC/WSJ (11/1-2)
17. ABC/Post (10/30-11/2)
18. Marist College (11/3)
19. CBS (10/31-11/2)
20. Gallup (10/31-11/2)
21. Reuters/ C-SPAN/ Zogby (10/31-11/3)
22. CBS/Times (10/25-29)
23. Newsweek (10/22-23)
Frankly, if they’re not in the top 5, I’d take them with a grain of salt. Why? Because of things like this:
The latest CNN/ORC poll released today shows a wider lead for President Obama than the previous CNN/ORC poll but it is doubly skewed. It massively under-samples independents while it also over-samples Democratic voters. The CNN/Opinion Research Corp. poll official reports Obama at 52 to percent and Mitt Romney at 46 percent. Unskewed, the data reveals a 53 percent to 45 percent lead for Romney.
This new CNN/ORC survey, unlike many other analyzed, not only over-samples Democratic voters, but also massively under-samples independent voters, to produce a result more favorable to Barack Obama. This survey’s sample includes 397 registered Republicans and 441 registered Democrats. But the survey included a total of 822 registered voters, leaving only 37 independent voters at most. The survey clearly under-sampled independent and Republican voters.
Note that CNN’s history has it tied for 8th place for accuracy last time around. So, what should those numbers likely be? Well let’s hear from the polling group that came in tied for first (and this is probably the reason why):
Rasmussen Reports recent reporting of partisan trends among voters, based on tens of thousands of voters surveyed, showed the voting electorate made up of 35.4 percent Republicans, 34.0 percent Democrats and 30.5 percent “Unaffiliated” or independent voters. Clearly 6.5 percent of a sample as independents is a large under-sampling of those voters compared to 30.5 percent.
However, what CNN used was this:
The sample for the CNN/ORC poll includes 50.4 percent Democrats and 45.4 percent Republicans and appears to have only 4.2 percent independents. This means independents are under-sampled 25 percent while Democrats are over-sampled 12.1 percent. Both of those are larger variations in sampling than seen in most polls that are likewise skewed by such sampling variations.
And as it stands now, independents lean toward Romney. So unskewed, or perhaps “properly skewed”, the results would be quite different. In fact they would give Romney a 53 to 45 percent lead over Obama.
That sort of sloppiness is an indicator of why CNN was in 8th place before in a race in which there were actually a large population of self-identified Democrat voters, a population that likely doesn’t exist in this election, or at least not to the extent it did in 2008. And if they’re in 8th, you can imagine how sloppy those below them on the list are.
Look at Gallup for heaven sake. 20th? Of course, the right should also note Fox and IBD/TIPP in 13th and 15th positions when they get excited about results there.
This is not just a phenomenon in national polling. It is also happening in swing state polls as well. For example PPP’s recent Ohio poll.
Finally, remember this when considering the RCP poll average. Many of the polls making up the RCP average are found way down on this list.
Just a word to the wise as you watch everything unfold. There are polls and then there are, well, “polls”.
Make sure you know which one’s to watch.