Exit poll study

[jmknapp]

I read the study on the exit poll failure and have a few comments if anyone might please chime in.

http://www.exit-poll.net/election-night/EvaluationJan192005.pdf

The gist of the report is that the discrepancy between the exit polls, showing a healthy lead for Kerry on election night, and the actual results of a narrow Kerry loss was caused by "differential non-response." Call it DNR. That is, presumptively, Bush voters were more likely to refuse to be interviewed by an exit pollster.

However, as I read it, this is a very unsatisfying conclusion, as there is no evidence presented that DNR leaned one way or the other. Rather, when the measured discrepancy between the Kerry poll and actual margins, called "within precinct error" (WPE) could not be explained by any other means, DNR was simply the default explanation.

WPE in the report is positive for error favoring Bush and negative for error favoring Kerry. E.g., if the poll said Kerry led by 5% but the actual results have Kerry losing by 2%, the WPE is -7. There are lots of tables in the report trying to show correlation between WPE and selected characteristics of the interviewer, polling location and other factors.

When discussion turns towards what might have caused the DNR favoring Kerry, general handwaving ensues. One idea spreading around is that somehow young 20-something interviewers with clipboards just attract Democratic voters--differentially of course.

But even on the basis of the numbers given in the report, this conclusion seems weak:



So were Kerry voters also attracted to 55-64 year old interviewers?

I would like to see a measure of the statistical significance of the conclusion that "Older interviewers had a lower WPE than the youngest interviewers."

Moreover, since the whole idea is predicated on a supposed tendency of Bush supports to refuse to be interviewed, the following result in the study would seem to run counter: "There was no significant difference between the completion rates and precinct partisanship."

So if the premise is that Bush voters were more likely to be refuseniks, wouldn't we expect to see an effect based on precinct partisanship? And since such an effect is not observed, might not the whole premise be rejected?

Here is the data presented in the report on this score:



So where is the beef behind this DNR hypothesis, or is it simply what it appears: just the default hypothesis when no other explanation can be found.

There are so many variables of course that are hidden in such simple charts and tables. For example, maybe the age of the interviewer just happened to be clustered differently among Bush and Kerry precincts.

Here's an example of just one such faulty conclusion I think in the report (where other variables are not considered). The report states: "The WPE was greater when there were three or more precincts at the polling place." This was attributed to confusion among precincts (where the interviewers were supposed to sample only one precinct). But Cuyahoga County data with which I'm familiar shows that the multiple-precinct polling places tended to be in the exurbs that were heavily Bush. So rather than showing an effect of number of precincts on the WPE, they might just be seeing a restatement of the main effect of greater WPE in Bush precincts.

Here are a couple of m,aps illustrating the distribution of multiple-precinct polling places in Cuyahoga County:




So to my mind we are still left with the basic, unexplained result, that for some reason the exit polls in heavily-Bush precincts had large errors. Here;s that data from the report:



The report prefers to throw out the heavy Kerry and havy Bush precincts since in an 80% Kerry precinct there is little room for overstating Kerry support, and vice versa. So it concentrates on the middle three groups which show a more or less consistent error towards Kerry of 6-8%.

I have to get back to the finding that they observed no effect based on precinct partisanship. To give an average WPE of -7%, Bush voters would have to refuse to be interviewed at approximately double that or 14%, compared to Kerry voters.

That should show up in the completion/refusal graph above, but doesn't.

Seems to me that only by ignoring Occam's Razor, and multiplying causes willy-nilly, might the DNR hypothesis be maintained. For example, in an 80% Bush precinct maybe the Bushies refuse at a high rate, but that is made up for by 20% Democrats, who, outnumbered in such precincts, are more anxious than usual to stand up and be counted. So the completion rate stays constant. Suuure.

[On the Road]

1. The Refusal Rate Was Over 30%?

That's amazing. That leaves all kinds of room for error.

The last graph is very telling. It actually supports the theory by suggesting that only in dominant Republican areas did the exit polls work properly. Which is consistent with Republican voters refusing in higher numbers.

[jmknapp]

4. Damn

Messed up the label on the graph. It's Bush % in precinct, not Kerry % (now fixed). So the mosr accurate polls were actually in the 0-20% Bush precincts, which are BTW largely black precincts.

[qwghlmian]

5. The non-response rate -

including refusal and missed voters - was 47%. The refusal rate was 37%. This population sample that they used was a very self-selected one.

Compare this to the German exit polls that Freeman touts as extremely accurate. The non-response rate there is 20%.

[RaulVB]

6. Yes, good talking point, isn't? (n/t)

[On the Road]

8. Yes, It is a Talking Point

but it's also a reasonable one. To overcome that, you have to show why it's incorrect. It matches my own experience canvassing for Kerry that obvious Bush supporters simply did not want to talk. At all.

I really don't know the answer. I'm glad Mitofsky's feet are being held to the fire. But the statistics, if they're correct, provide a plausible explanation.

[RaulVB]

14. No, incorrect.

I have tons of ANECDOTAL evidence that shows the opposite.

I don't have to "prove" anything. I have to go with Mitofsky's "modus operandi."

I NEED TO INVENT A THEORY THAT FITS MY OWN INTERPRETATION OF REALITY, BASED ON FALSE OR INACCURATE PERCEPTIONS.

[On the Road]

16. Your Anecdotal Evidence

has a much higher "nonresponse rate" than 47%.

I would like nothing better than to show that the exit polls were right and the difference was all due to fraud.

But discounting voter suppression, none of the fraud investigations that I've seen so far would be sufficient to cause a 6% national swing.

You don't need to prove anything to come to your personal conclusion. But you do need a more compelling rationale to move it forward in the court of public opinion.

[emcguffie]

22. in my phone calls before the election

I had the opposite experience. Bush voters couldn't WAIT to tell me who they were going to vote for. And I wasn't asking, I was being non-partisan, asking "registered Dems" if they needed any assistance getting to the polls.

That stuff.

[euler]

66. I implore you, please say something intelligent...

...once in a while. It's not difficult

[On the Road]

9. I Missed That, Too

I thought missed voters were part of the 37%. So it appears that the exit polls only asked a littel over half the voters leaving the polls. That would seem to be enough to allow an actual 3% Bush lead to be measured as a 3% Kerry lead.

[qwghlmian]

11. Combine the huge non-response rates with

this study that shows that Republican-leaning voters are less likely to respond to exit polls than Democrats:

http://www.duke.edu/~mms16/non_response2000.pdf

"Our results also lead us to conclude that exit polls are likely to over-represent the opinions of younger and non-white voters. Because non-white voters tend to vote for Democrat candidates, over-representation of this social class will skew an exit poll’s results in that direction."

They also cite some studies that show that non-whites respond to exit polls a lot more readily than whites (Brehm, John. 1993. The phantom respondents: Opinion surveys and political representation. Ann Arbor: The University of Michigan Press.) and studies that show that young people respond to exit polls more readily than whites (DeMaio, Theresa J. 1980. Refusals: Who, where and why. Public Opinion Quarterly 44 (Summer): 223-233.) and (Herzog, A. Regula, and Willard L. Rodgers. 1988. Age and response rates to interview sample surveys. Journal of Gerontology: Social Sciences 43 (No6): S200-5.)

Since other polls have consistently shown that younger people and minorities are less pro-Bush than older people and whites, this pretty much establishes that Bush voters were less likely to respond to an exit poll. The question is by how much Mitofsky would correct for that. I think he corrected for it, but guessed wrong on the magnitude of the skew.

[On the Road]

13. Fraud May Have Entered Into the Equation

but it's hard to tell how much because it's mixed up with the response variable. Response variables are tough to account for -- I spent most of senior year in college working on a study for my psychology major in so-called "perceptual defense." It should have been been an easy thing to establish, but the response variable were so tough to account for that the debate raged for at least a decade.

[skids]

28. Then how come...

The heavily republican precincts had the highest response rates?

[qwghlmian]

30. If you look at the graph that the original poster made

correlating non-response rates with the % of Bush vote, you will see that the differences are negligible - on the order of couple of percent.

[skids]

43. addressed below n/t

[jmknapp]

47. The use of the term "Democrat candidates"

Rather than, of course, Democratic candidates, identifies the authors as wingnuts who may be ignored.

[qwghlmian]

50. Is that really the best you can do?

http://www.montcodemocrats.com/candidates.htm uses the term "Democrat candidates". Are these wingnuts as well?

[skids]

54. No, not wingnuts...

...but when someone reaches that far to defend a well-known propaganda-driven ligual tick, it does make me wonder a bit.

[qwghlmian]

59. "reaches that far"? It was a simple google search

people do not always express themselves correctly, you know. Dismissing a scientific study because of a "lingual tick" is ridiculous, especially seeing how the same "lingual tick" apparently afflicts Democratic web sites as well.

[jmknapp]

61. Only rarely

99:1 it belies wingnuttery. That is beyond dispute.

It's quite useful really as an indicator, so I hope its use continues to spread among the far right.

[jmknapp]

17. When I posted the incorrectly labeled graph

with "Kerry %" on the axis rather than Bush %, you said it was consistent with the non-response hypothesis.

Now that you see the correct graph, do you conclude that it contradicts the the non-response hypothesis?

[On the Road]

18. No, I See What You're Saying Now

It's the opposite. Mitofsky has to explain why Republicans were LESS likely to respond in dominant Bush precincts than in dominant Kerry precincts.

You're right. I don't know how the response theory would explain that.

[qwghlmian]

19. I think (at least according to the studies I showed) that

the non-response theories do not care about the precinct leanings. What they are saying is that younger people and minorities (and, slightly, women) are more likely to answer the exit poll. This refers to Republican or Democratic younger people, minorities and women. But because those subgroups of the population are usually more left-leaning, this results in oversampling that favors the Democrats.

[RaulVB]

20. You are using the wrong talking point this time

Mitofsky CONTRADICTS your "very informed" comments.

Usually "more left leaning" is a judgment NOT SUPPORTED by EVIDENCE after Mitofsky's study "HAS DEBUNKED" all KNOWN voting patterns in the USA, prior to 2004.

Nice try, though!

[On the Road]

21. I Would Imagine It's More Complicated than That

It would depend on demographics of the voter, the poll taker, and also the environment. It's easy for me to imagine that a Bush voter (who may privately be embarrassed by a lot of what Bush does) would not want to talk with a 20-something pollster who looks like a Kerry supporter, especially in a pro-Kerry neighborhood.

But this pattern is something different. It does look more possible fraud from that breakdown. If it is -- God, I hope somebody finds the smoking gun.

[qwghlmian]

23. If you look at the graph that the original poster

made that graphs non-response and refusal rates vs. Bush % in precinct, you will see that there is basically no correlation. That supports what I said.

[On the Road]

25. The Theories May Not Predict a Correlation

but there could easily be one. Nonrespondents have certain characteristics which could vary based along with the demographics of the precinct.

But these heavy Bush precincts with the large exit poll discrepancy in favor of Kerry: that suggests recounts or audits in those precincts. Especially if there's a big variation.

An across-the-board poll shift in favor of Kerry argues for a problem with the polls rather than fraud. A subset of precincts with big discrepancies, regardless of where they are, argues for fraud.

[qwghlmian]

26. Most (all?) states allow public access to ballots for a period

of time after the elections (2 years?). All it takes for a private individual to go an recount a precinct is filling out a couple of forms and ponying up some $ (not much - in Florida it is $10/hour) to pay for the county workers who handle the ballots during the recount.


So - if anyone suspects fraud at some precincts, they can go and recount those precincts today. So far every recount conducted, whether in NH or in Florida (see www.recountflorida.com) has turned up no discrepancies. I don't expect to see any bombs uncovered in such recounts, for the simple reason that these recounts are possible, thus making any fraud that could be uncovered by recounts way too risky.

[On the Road]

27. Are You Suggesting Fraud by Some Other Means

or no fraud at all?

There are a lot of ways of committing fraud, some of which would show up in recounts, some of which wouldn't. The old "stuffing the ballot box" approach wouldn't show up because the ballots match the vote -- that is, not unless ballots cast could be reconciled with voters marked off the list.

This is why I think tracing individual ballots to individual voters is absolutely necessary for us to have full confidence in elections.

[qwghlmian]

I am sure there was fraud here and there -

I am also pretty sure there was no massive fraud on the scale of hundreds of thousands of votes in Ohio or on the scale of 3.5M votes country-wide. In order to accept that it requires suspension of too much disbelief. Ballot stuffing may have occurred on small scale but it is just not possible to do on such massive scale without it coming out - there are too many watchers of different political persuasions. Central tabulator theories can be easily discounted because of one simple fact - each precinct knows what its results were, and so do the precinct workers, and if they see in the published state results the numbers for the precinct that they know are wrong, do you think all of them would keep quiet? Lower level tabulators being programmed to selectively miscount is also too hard to imagine - the OH recount, however flawed, had too much of a chance of catching such fraud, and it didn't (if you'd like I can expand on that). The only thing where the fraud theories may contact reality is the no-paper-trail electronic voting machines, but as you can see from Mitofsky's data the exit poll skews did not limit themselves to DREs, and, in fact, were higher for some other voting methods than for DREs.

Of course, quite a few people have almost no disbelief left, so they have nothing to suspend :)

Secret ballot elections are the cornerstone of democracy. "Tracing individual ballots to individual voters" opens the door to such incredible abuse that I really cannot see how anyone who values democracy can seriously suggest it. If you are worried about the power of the state today, just think how much you want the state to know exactly whom you voted for.

[RaulVB]

31. Great talking points!

"Good job", guy!

[On the Road]

35. I Tend to Agree with That

the fact that the red shift was so geographically broad makes it more likely that it was real.

However, given that it was real, there still is the question of whether fraud was committed, and if so was significant.

I've sometimes been criticized here for being skeptical about so-called proof of fraud. But at the same time, I'm also skeptical about so-called proof of no fraud, or of insignificant fraud. Not all information has been open to the public, including some of the critical data in Ohio that could show whether the election was swung.

----

As far as secret ballots go, they are in no way a cornerstone of democracy. In the first presidential election, voters walked into a polling place and told the room who they were voting for, at least in Virginia.

I believe in secret ballots, just as I believe in secret medical records. That's no reason they should be anonymous or impossible to track to an individual. Whatever the potential for abuse, the potential for abuse of anonymous ballots is well established.

[FogerRox]

71. I disagree-- 10 to 11 million votes nation wide

An incumbent gets his may-june approval rating in Nov. Bush was at 44% SO minus 1% for 3rd party votes --Kerry gets 55%.
About 112 (IIRC) million votes cast--so we were told---
Kerry-62 million votes
Bush--55 million votes

Funny how when one looks at some of the exit polling studies--they show a Kerry win of similar size. SO how does exit polling done Nov. 2nd--compare favorably with a Kerry/Bush Poll done in May or June?

[skids]

33. Think about the reality of the situation for a minute.

In a group of precincts where there are 80% or more, usually more, republican voters, how can the difference between the non-response rates between democrats and republicans really make a difference in the average non-response rate. For every one democrat that refused, four republicans would have to refuse in order for things to be equal. And that's not possible given the overall response rate.

So the fact that the response rate is higher in highly republican precincts pretty much blows a hole in the idea that *all* republicans exhibited this behavior.

If there was a differential non-response, then it had to be situation-specific. The way the numbers work in this case, the non-response rate would have to be high among republicans in 50/50 split precincts, much lower among democrats in heavily republican precincts, and it should be noted, low among republicans in heavily democratic precincts.

That's a pretty contorted hypothesis.

[qwghlmian]

36. You are missing the point, skids -

the non-response rates in the studies correlate not with political leanings (in fact, they did not check for that at all) but with age, minority status, and gender. The distribution of those demographic variables should be pretty much the same across all precincts, whether they lean D or R. Thus it is not surprising to see basically the same non-response rates in all precincts, whether D or R.

The reason the non-responders would skew the exit polls to the D side is that the young people and minorities lean to the D side, and because of the non-responder effect above are more likely to respond to exit polls.

[skids]

42. No, I think you are.

But we know (roughly, plus or minus any fraud) how these people actually voted. And that was broken down versus the non-response.

If it is the premise that people who actually voted for bush had higher non-response rates, then precincts that voted for Bush should show that, or, the only other explanation, the non-response rate was extremely severe among a particular sub-group of Bush voters, and for some reason, that subgroup happened to be located mostly in precincts that were split 50/50.

How small could that group be, and still have an impact that would sway the entire poll by so much? How decorrelated would their distribution have to be from any of the demographics that were broken down, in order not to reflect that extreme number?

Finally, a variance of the response rate of the amount shown in the charts is not "pretty much the same" when you are talking about a 5-10% effect on the final outcome being blamed on this effect.

I'm sorry but this differential non-response theory has no evidence to back it up, and seems to me to be more tenuous the more I look at the report.

[qwghlmian]

46. We keep talking past each other -

I see I will have to resort to actual numbers to explain
(round figures picked for simplicity):

Precinct A: young people vote:               10% B  90% K
            older people vote:               60% B  40% K

            young people non-response rate:  30%
            older people non-response rate:  70%

Let's also say that the young people proportion to old people is 50:50

Results:  actual    35% B   65% K
          exit poll 25% B   75% K 

Precinct B: young people vote:               30% B  70% K
            older people vote:               90% B  10% K

            young people non-response rate:  30%
            older people non-response rate:  70%

Let's also say that the young people proportion to old people is 50:50

Results:  actual    60% B   40% K
          exit poll 48% B   52% K 

As you see, one precinct votes hugely Democratic, the other vites hugely Republican. The age correlation of non-response is the same, the demographics are the same. The skew in one is 10%. The skew in the other is 12%.

Do you see how it could be that R and D precincts would show the same non-response rate (and the same skew due to it)?

[skids]

52. Yes I get that.

Qualitatively, I agree with your synopsis.

My point is that, quantitatively, it doesn't add up.

Note that for the purposes of the charts provided by the NEP, a precinct isn't "hugely democratic" until 80% partisanship.

Nothing in the study you posted suggests a non-response rate among any of the utilized demographic indicators which is high enough to make this hypothesis stick as the main cause of the discrepancy, especially since the demographic indicators in the posted study were many of the same indicators which were used as weighting criterion.

Now just because I can't think of a particular group characteristic that would cut across the grain of the weighting classification and account for non-response being lower in 80%+ Bush voting precincts but still be strong enough among the remainder of the Bush voters to cause a swing, doesn't mean there isn't one. Nevertheless, the numbers in the report don't give me any reason whatsoever to believe that the assessment that non-response by Bush voters was the cause of the discrepancy can hold water. And I'm sorry, I'm done taking what Mitofski says on faith.

When tackling these matters, you not only have to consider the various dynamics of correlation that are feasible, but also the limits which apply. Without the latter, well, you could build a case that anything could have happened. Doesn't mean it did.

Oh wait. I did think of one type of Republican voter that might have a very high non-response rate... the kind that only exists on paper.

[jmknapp]

48. Excellent points

These non-responding Republicans would have to be the equivalent of Maxwell's demons, exhibiting ad hoc behavior depending on the situation.

[anaxarchos]

37. Ha... this is the best one yet... n/t

[emcguffie]

24. Excuse my lack of science, but --

It is absolutely absurd to attribute any such one trait to Bush voters in all those states and precincts. There are too many different types of voters of all persuasions, and that is why exit polls work, isn't it? The randomness?

And, please, the lower income Dems are the ones most likely to be rushing off to work or to pick up baby at daycare before being fined.

The notion that Bush voters were less inclined to speak to the interviewers holds no water. Too bad I can't express it scientifically. So I'm frustrated. But there are lots of gloating rightwing loudmouths that can't hold it in! Oh, no, gee, Bush voters are less likely to consent to be interviewed. That's nonsense.

Maybe a registered Dem who voted for Bush might feel guilty and not want to be asked. Similarly, a registered Republican who voted for Kerry might be iffy as well. A torn independent? Who knows, they could go either way.... ;)

I wish you statistic brains would take apart that chart with the specific info, like the percentage of women that voted for Kerry, the percentage of Dems that voted for Bush.

[On the Road]

29. What You're Proposing is Not So Much It's Not Randomness

as lack of correlation. It doesn't make sense to you that the population of Bush voters would have a different propensity to talk to an exit pollster.

I haven't seen it shown one way or another, although there's that Singer paper that claims to show it, which I only glanced at.

But fundamentally, there's every reason to think that a group of people voting for one candidate would have different characteristics in many other ways. Talking to a pollster may be one of them. Those two things can absolutely be correlated.

That's one reason a high response rate is so important. And it turns out they only measured 53% of people coming out of the polls. And if there are signficant patterns in where people responded (which precincts) or who they responded to (who the pollster was), that increases the likelihood that the exit polls were not a good match for the vote.

[RaulVB]

32. HIS SUPPORTERS ARE VERY VOCAL (n/t)

[RaulVB]

2. Thanks, Mr. Knapp

Great information!

[bruised]

3. NEP report on their exit polls - the real story

Yuur analysis looks serious - but I need need more time to digest it.
In an earlier post
http://www.democraticunderground.com/discuss/duboard.php?az=view_all&address=102x1168254
I point out what is practically a blatant lie in the report.

[Igel]

56. I didn't read through the entirety of the second thread (

your link was locked, and referenced a second, much longer, thread).

Many stations call states based on M's stats that he himself hasn't yet called. He calls states, but he's so cautious about it that usually stations don't care what he says by the time he says it. I've read that at times he's even argued with networks about calling elections based on his numbers--they wanted to, but he wasn't sure.

M's miscalled elections in the past. He was wrong. He hates being wrong.

The question is: Did *Mitofsky* call any of the states that his early exit poll estimates said were runaway Kerry?

[L. Coyote]

7. Is your spreadsheet available? n/t

[jmknapp]

10. Hi Coyote

No spread sheet -- just transcribed the numbers from the pdf linked above. The report mentions some data repository at the ISR at the University of Michigan. Don't know if that's publically available though.

[L. Coyote]

12. Looks like Excel graphs

If possible, e-mail me your Excel file.

[bruised]

15. The spreadsheets for all the tables on exitpollz.org

are available directly form the sight. The links "Check calculations" or "check the above calculations" take you to the .xls spreadsheets.
Similarly all the tables have links to the documentary evidence.
This is why there are no exit poll figures on exitpollz for the 4 states (NY NJ NC and VA) I know there are figures quoted - but we don;t have any documentation.

[skids]

34. jmknapp: take a look at the median versus the mean on those charts.

Ponder that. What do you think?

[smartone]

38. simple question

i have a question i have yet to get an answer for..

the premise is that Bush supporters did not respond when asked to participate in exit polling -

my question is did the same deviations take place in the Senate races?
especially in swing states arizona colorado, penn etc.. were the exit polls wrong in senate races also?

[skids]

39. The non-response rate is for all races -- you take the survey for all.

The deviations observed in the senate races were there and followed
the same pattern, but were smaller.

[anaxarchos]

45. Significantly smaller.... n/t

[intensitymedia]

40. Exit Polls in Wisconsin nailed both Presidential and Senate races

because there was no fraud.

The repeated assertions on this thread that Republicans were nonresponsive to exit poll questions is completely unsubstantiated.

We're arguing with smoke - and maybe chimp smoke - if we argue against a hypothetical assertion that then rests on even more hypothetical assertions (Bush voters don't respond because they're "different").

Only as long as we talk around in circles about various theories as to the exit poll discrepancy, can the central point be missed.

The exit polls prove fraud, in Ohio just as in the Ukraine.

Oh, and don't buy any nonsense about their being a different "design."

Another unfounded assertion, published even, with no evidence that there is anyone who knows what that design was, and how it varied.

This thread and other are an amazing phenomenon - distraction and sleight-of-hand replacing evidence, reason and logic.

We're going to change the world.

Don't get in the way


Che

[qwghlmian]

41. Incorrect

Your use of "unsubstantiated" reminds me of the Inigo Montoya's phrase "You keep using that word. I do not think it means, what you think it means."

I have posted references to several studies to substantiate the non-randomness of the non-responders phenomenon. As for the exit poll design differences:

http://www.mysterypollster.com/main/2004/12/what_about_thos.html

This describes how the design of German exit polls varies from the US exit polls - and why German exit polls can be used to detect fraud and US polls cannot. Basically German exit polls us much bigger samples, poll many more precincts, have better trained interviewers and a much higher response rate, and have the benefit of demographic data that is collected by the government census bureau inside the polling places.

You're welcome.

[RaulVB]

44. I like your "source"...

"I cannot claim expertise in European exit polls..."

"You're welcome"!

[Yupster]

62. My favorite movie

and Andre the Giant's best work on the big screen.

"Anybody want a peanut?"

[jmknapp]

49. Which charts? n/t

[skids]

53. The very same ones...

...you used to generate those graphs.

[jmknapp]

60. Well, I see some relatively small differences

between the medians and means generally, so I wondered if you were talking about a specific chart. What is your take on it?

[skids]

64. Just this...

...the difference between the mean and the median can indicate when there is grouping or large outliers in the sample (and in which direction the outliers were.)

                            avWPE midWPE WPEskew
High Dem (Kerry>=.80)        0.3 -0.4    +0.7
Mod Dem (0.60<=Kerry<.80)   -5.9 -5.5    -0.4
Even (0.40<=Kerry<.60)      -8.5 -8.3    -0.2
Mod Rep (0.20<=Kerry<.40)   -6.1 -6.1     0
High Rep (0.00<=Kerry<.20) -10.0 -5.8    -4.2

...so it would appear that the high republican support was not a very homogenous group of precincts when it came to their WPE. Granted, the sample size borders on small (N=40 precincts)

Other standouts were:

Size of Place > 500,000 (-2.0 N=105)
Urban Paper Ballots (+5.5 N=5)
Rural Lever machines (+2.2 N=26)
4 or more precincts (-2.2 N=66)
...and a couple of the interviewer age/education questions where I think a better question they should have asked was "what were you wearing?"

Anyway, aside from the Urban Paper ballots, which is such a small sample that data grouping is really not something we can assess,
nothing comes close to the data grouping in "High Rep".

If one were rolling forward with the theory that fraud is responsible, the most logical assumption is that fraud took place in a sizeable but limited percentage of precincts. This could be a clue where to look: the "High Rep" category. (Note that suppression and vote deletion in "High Dem" wouldn't show up here, since it would be applied indiscrimately to whole Democratic stronghold precincts, as the loss of Republican votes would be considered a small sacrifice.)

[jmknapp]

69. High Rep median

I see what you mean. There must have been some really high errors in some Bush precincts to get that average error up to over 10%.

[southwood]

51. Fine work, as always!

kick!

[Time for change]

55. This report is a joke

They automatically assume that any difference between any group within precincts is due to WPE, rather than a problem with the election. So the WPE is synonymous with a bias towards Kerry, by their definition. They never even comment on the fact that none of the differences between any of the groups could account for the exit poll discrepancy because none of those differences are large enough to account for it.

Here are some of the comparisons:

Interviewers who have graduate degrees are associated with a high WPE. I guess that means that if an interviewer is too educated he or she produces a bias in favor of Kerry because they are too likely to follow the written procedures.

Also, the swing states have much higher WPEs, and precincts that use paper ballots that are counted by hand have the lowest WPEs of all. Hmmm, I wonder why?

[Igel]

58. So, it's *all* fraud.

A couple of questions.

Why is it that fraud and rain are correlated? Fraud and distance of interviewer from polling place, fraud and experience of interviewer? If these really are significantly correlated, I have trouble believing there's significant, widespread fraud. Maybe in some precincts, but not generally.

I think M figures there are several reasons for high WPE, but the only one that's consistent and shows interesting correlations is non-response. Some of the others he can't quantify, or are intrinsic to just a relatively small number of precincts.

As for paper ballots counted by hand ... I'm curious as to which large precincts this is done in. Or which small ones. I remember reading the report and thinking at the time that his rationalization for the lowest WPEs seemed reasonable.

As for swing states (and, I'd add, some devoutly blue states) having high WPEs ... I can't help but wonder if there's a correlation between turnout and WPE. M says there is, in general. I'd like to know how it devolves to the precinct level. I don't recall seeing turnout figures for 2000/2004 broken down by state, but I think I'd be curious to examine them now. Just rambling at this point ...

[Carolab]

67. Is there any correlation between precinct size and WPE?

If vote switching/padding were occuring, it would be less noticeable in larger precincts and if a few votes were added to the * column and subtracted from the Kerry column, but the exit polls indicated a higher Kerry turnout, wouldn't it be more likely that the pollsters could claim/find a higher WPE in those larger precincts?

As for the hypothesis above, that the precinct workers would necessarily "see the errors" in the final tallies, that is not true unless the ballots were all individually reconciled against the precinct machine totals and against the polling books themselves. It is my understanding that the precinct tallies are simply uploaded via machine or the cartridges are removed and delivered to the state elections office for central tallies. Do the precincts reconcile the machine counts to the actual ballots (where they exist) and to the pollbooks before signing off and reporting/delivering those results? Do they count the number of Republican, Democratic or other votes? If not, then the switching could occur and the ballot counts would match, but the actual vote counts per candidate wouldn't necessarily match, right? The precinct workers didn't know the actual votes cast for each candidate by counting them on the ballots, right?

[Igel]

70. No, don't make me think in the morning.

Need coffee ...

You're right. Fraud would be less salient in larger precincts. Having some precincts show massive WPE through fraud can't be ruled out. And having very, very slight pervasive fraud would also slip under the statistical radar. I just don't think it could be a very high percentage: flipping votes would increase the WPE, but I have trouble believing that the error would show large correlations with things like weather or distance of interviewer from polling station.

Then again, I could be wrong, and it wouldn't be the first time.

I know where I was a poll worker that we kept two running totals of voter counts--one in the poll book, one with names and voter number on a lined pad. The unofficial copy was usually more accurate, since there was no page flipping involved. And then we made two copies of the voting results from the machines (which were mechanical, so computerized uploading just wasn't gonna happen). Those copies yield early returns. And all the unofficial notes and the official incident reports were turned in.

Then the county BOE is supposed to reopen the machines and double check our reporting.

One of us always tallied up the votes for each race, just to gauge undercount. I can't imagine what we'd do if there was an overcount. Undercounts were usually as expected: Some no votes for prez or the highest ranking office; lots of no votes for things like BOE or offices that people didn't know existed.

One other correlation that would be cool to run, come to think of it, is WPE vs. party affiliation of the BOE chair of the precinct.

[Time for change]

74. I don't understand everything you're saying here

But I certainly agree that precinct workers would not notice errors on the final tallies. That is not their job. And even if they intended to do that, I don't see how they possibly could. How could they possibly know what the final tally was supposed to be unless they hand counted every single vote? And that doesn't happen.

[Igel]

77. It depends on how the voting's done.

I know that the precincts where I was (we used lever machines) we kept scrupulous track of voters. We knew how many votes each prez candidates, congressional folks, even the local dogcatcher candidates (if there was a race for one) got. But we couldn't match up voters with votes.

And we made sure that the machine count and the poll book were in perfect synch. That was our job. The undervote patterns typically made sense, and there couldn't be any overvotes. Moreover, the vote totals for no race ever exceeded what was in the poll book (this should make sense, but in principle the ways to hack lever machines could lead to a discrepancy).

In principle we could each have made copies of the vote totals for our precinct and later checked them against local BOEs online precinct tallies. But we never did.

That the pollworkers managed to screw up at least half of all the affadavit ballots (aka "provisional ballots) is beside the point. Old, tired, not fully competent, and poorly trained. (Different discussion, that).

[Time for change]

73. With regard to the "corellations" with distance and weather

This is reasonable for you to bring up, but there are two important points to make about it:

First, as with all of the "corellations", even when one looks at the group which is associated with the least "bias", substantial "bias" still remains. For example, take distance of the interviewer from the polling place. Even when you look at the closest distance in the survey (inside the building - see page 37), the WPE is still -5.3, as compared to -6.5 for the entire national sample. Therefore, even if there is a difference by distance from the polling place, that only accounts for a small portion of the bias.

But there is one major exception to this rule -- there is a very large difference in WPE between paper ballots and other types of voting.

Second, no statistical computations are given for any of these comparisons, so we have no way of knowing whether any of these differences are statistically meaningful (which is why I put "corellations" in quotes.

For a more detailed explanation of this, see my post from a couple days ago:
http://exit-poll.net/election-night/EvaluationJan192005...

[madison2000]

57. I don't understand why the exit polls are more accurate in some states

than in others.

[anaxarchos]

63. That is the $64,000 question....

You might look at TIA's thread on this:

http://www.democraticunderground.com/discuss/duboard.php?az=view_all&address=203x296774

The bar chart is striking.

[L. Coyote]

65. Conyers isn't buying the report either.

Open Letter to Warren Mitofsky and Larry Rosin from Rep. John Conyers, Jr. 1/20/05

http://nov2truth.org/article.php?story=20050120151520111

[floridadem30]

68. I have a question, why is it that repub. refuse exit polls and dems do not

and how can this claim be made in Florida where a majority of the voters were not exit polled because they voted by absentee or overseas ballots? Or do they just believe all of us that voted absentee support bush. Most of the democrats I know voted absentee and therefore no exit poll.

[cyn2]

72. Bush voters were ashamed of their vote....

that's why they don't want to answer the exit polls.

[Carolab]

76. That is pure conjecture. n/t

[Beth in VT]

75. Good point, I wonder if the absentee ballots were skewed to Kerry? n/t