Our Polling in IL-10

[brooklynite]

Public Policy Polling:

Obviously our polling on the 10th Congressional District Democratic primary in Illinois was way off, one of our worst misses ever. We apologize to our clients, the Progressive Change Campaign Committee and MoveOn, for letting them down on this race.

PCCC and MoveOn have been transparent with their polling on the race and have posted all the polls we did for them over the last 3 months here. This error is on PPP, not them.

We did see the race tightening significantly over the final week. The tracking poll we did on Sunday, March 11th found Sheyman leading by 21 points, 46-25. The tracking we did on Wednesday, March 14th found Sheyman's lead down to 14 points at 44-30. The numbers that were released publicly last Thursday combined the interviews from Sunday and Wednesday, for the purposes of a larger sample size.

The final night of tracking we did on Sunday, March 18th continued to show Schneider gaining ground while Sheyman stayed in place, and that the lead was down to 11 points at 44-33. Still we never would have expected this.

We have a few theories on what went wrong:

-We called people who had a past history of voting in Democratic primaries, as we do for all primary polls. But I wonder if with the open primary and Schneider's status as the more conservative candidate if some people who usually vote in GOP primaries crossed over to support him. We would have missed those folks.

I think the bigger possibility though and a lesson learned for our work with all of our clients is this: we may have overpolled the district:

-The pool of likely Democratic primary voters in this district is small enough that we had to call everyone for both a message testing poll we did at the end of February and for one of the 3 nights of tracking we did over the last week. Calling people twice in such a short period may cause respondent fatigue.

There's 2 reasons I think this may have benefited Sheyman. The first is that on the message testing poll a much higher percentage of Schneider supporters hung up in the middle of the call than Sheyman ones, presumably because they thought the poll was being too negative to him, even though the poll tested negatives against both candidates and was nothing out of the ordinary for the polls we conduct. I think it's possible that when we called again for the tracking over the last week Schneider voters may have been more reluctant to participate. We actually had a different person record the tracking poll than the message poll so people previously called would hear it with a fresh ear but there may have still been some aspect of that.

The 2nd reason I think respondent fatigue would have helped Sheyman's numbers is that we did continuously find that his supporters were more excited than Schneider's were, and that may have extended to their being more excited to answer polls than Schneider's were too.

One thing I know is that it didn't have anything to do with the way the data was weighted. Weighting had very little impact on the final results.

For PPP I think the lesson is that a Congressional primary electorate is too small to call for multiple polls in less than a month, especially if there's some aspect of negative message testing involved. It needs to be a big enough pool of voters that you can call completely different universes every time.

We apologize that our polling skewed perceptions of where this race stood. When you poll as many races as we do you're going to badly screw one up every now and then. But we strive to be as close to perfect as possible and in this instance we fell badly short.

Sometimes, the fact that your candidate lost is simply because more people voted for the other guy...

[frazzled]

1. This is bad news for PPP

This poll sounded weird to begin with, and these statements of "I wonder" seem very weak. Do you really think that Republicans facing a hotly contested presidential primary in their party are going to cross over in great numbers to vote in a Democratic primary in which the district had sent a Republican to Washington for the last dozen years?

It sounds like it may have been something of a push poll. Lesson learned: whether you want to believe or not, never believe polls commissioned by interest groups, no matter what their political bent.

I would have voted for Sheyman if I lived in the 10th (I live in the 7th, where Danny Davis is the foregone conclusion every two years). But it's a pretty conservative district for Dems. And I have to say that it's probably a stretch for a lot voters to think about sending a 25-year-old to Congress. Besides, there weren't a lot of voters. This was the lowest turnout in a presidential primary year ever in Illinois.

[ellisonz]

2. Interesting.

Sheyman doesn't need to win big, he just needs to win.