A non-statistician's view of the E-M exit poll controversy

[Time for change]

There has been a great deal of highly technical discussion and argument on the DU and elsewhere over the proper interpretation of the Edison-Mitofsky report http://exit-poll.net/election-night/EvaluationJan192005.pdf ,which purports to explain the wide discrepancy between the E-M polling results and the official 2004 Presidential election results. This discussion and argument is needed. However, one problem with it is that much of the discussion is too technical for non-statisticians to fully understand. I believe that it is very important for non-statisticians to understand the essential issues of this controversy, because the great majority of U.S. citizens are non-statisticians, and they need to understand this issue. This post is my attempt to frame the discussion in a manner that all DUers will be able to understand (although I welcome criticism by statisticians).

Although I am not a statistician I have taken several statistics courses, and I work with statistics (though not as complex as some of the discussion recently appearing in this forum) almost every day in my work as an epidemiologist for the FDA. I have read the E-M report and the USCV responses to it, and I strongly believe that the good majority of DUers are capable of understanding the crucial issues involved if they are discussed in lay language.

In the interest of full disclosure, I admit that I am a partisan Democrat, and that I believe that if the United States remains under the control of a Republican President, Congress, and judiciary for an extended length of time the consequences for the great majority of people in this country and the world will be terrible. However, I don’t believe that that makes me biased in my interpretation of the exit poll evidence or any other evidence bearing upon the possibility of 2004 election fraud. I say this because I recognize that in order to correct a problem (in this case, a Republican U.S. President and Congress) one must first understand the reason for the problem. So if I and thousands (or millions?) of others are wrong in believing that John Kerry won the 2004 election, and that George W. Bush was installed as President because of election fraud, then our efforts to expose this fraud may not only be wasted but actually be diverting us from our main goal, which is to bring a better government back to our country. On the other hand, if we are correct in our view, then fixing our election system is our best (and very likely only) hope of winning our country back, and I believe that exposing the election fraud is a very important step towards that goal. Therefore, it is in the interest of almost all of us, not to prove a specific point of view, but to investigate this matter until it is fully understood, and presented to the American public for all to see.

Having said that, I will also they that I am not 100% convinced that John Kerry really won the 2004 election. Maybe 97% -- so I continue to keep an open mind on the subject. This is based both on the exit poll evidence, which I discuss below, and also on a lot of other evidence, which I will not discuss in this post.


Here is my view and explanation of the exit poll controversy:

The 2004 E-M national exits polls predicted very different results than the official Presidential election results. Whereas Bush won the official results by 2.5%, the exit polls predicted a Kerry victory nationally by 3% -- a 5.5% difference. In addition, state exit polls predicted a Kerry victory in four states that Bush won – Ohio, Iowa, New Mexico, and Nevada – and a virtually even race in Florida, which Kerry lost officially by 5%. Of these states, the difference between the exit polls and the official results were statistically significant only in Ohio and Florida. In Ohio, Kerry lost officially by 2.5%, while winning the exit poll by 4.2% -- a difference of 6.7%. Winning either Ohio or Florida would have meant an electoral victory for Kerry. On the other hand, none of the states that Kerry carried in 2004 were predicted in the exit polls for Bush. None of this is controversial or denied by Mitofsky.

When exit polls differ substantially from official election results, there can be only three reasons (or combination thereof):
1. Random error, or chance
2. Biased polls
3. Impaired election integrity
This is not a controversial statement.

Given all that, here are the reasons, taken together, why I believe that the Mitofsky report supports the likelihood of election fraud, notwithstanding his statement in the executive summary of his report that “Exit polls do not support the allegations of fraud due to rigging of voting equipment”.


1. The role of random error (or chance)
The first step in the assessment of any statistical discrepancy is to assess the role of chance in producing the discrepancy. The likelihood of the discrepancy between the national exit polls and the official national results occurring by chance has been estimated by Jonathan Simon and Ron Baiman http://www.freepress.org/images/departments/PopularVotePaper181_1.pdf as being close to one in a million. The original response to the E-M report by US Count Votes (USCV) http://www.uscountvotes.org/ucvAnalysis/US/USCountVotes_Re_Mitofsky-Edison.pdf estimated that the likelihood of the discrepancy between the combined state exit polls and the official state results occurring by chance was about one in ten million. A proper combined likelihood of such a large discrepancy in both the national and state polls would multiply those two numbers, to give a result of one in ten trillion. Although the exact number can be and has been computed in different ways by different investigators, nobody, including Mitofsky, disputes the fact that the likelihood of this discrepancy occurring by chance is so small that it should not even be considered.

So that leaves two possibilities: Exit poll bias or impaired election integrity.


2. An initial look at the potential role of exit poll bias
Exit poll bias can be broken down into two components: Biased sampling of precincts AND bias within precincts – referred to as “within precinct error” (WPE). The former can be easily tested, and the latter cannot be easily tested (and many question whether or not it can be accurately tested at all). Mitofsky tested bias due to sampling of precincts and concluded that this bias actually favored Bush. Therefore, the hypothesis that the exit poll bias (which would have to favor Kerry) might explain the discrepancy between the exit polls and the official election results becomes less likely, since all of this bias must be concentrated within precincts (WPE), and this bias must account for not only the discrepancy between the poll results and the official election results, but also the bias due to sampling of precincts, which works in Bush’s favor.


3. Mitofsky’s explanation for the bias – the reluctant Bush responder (rBr) hypothesis
Mitofsky asserts in the executive summary of his report, as an explanation for the bias (henceforth referred to as WPE) that “Kerry voters were more likely to participate in the exit polls than Bush voters”. This has been referred to generally as the reluctant Bush voter hypothesis, otherwise known as rBr. However, Mitofsky does not refer to this as a hypothesis – he simply states it as a fact. Not only that, but neither in the executive summary nor in the body of the report does he provide ANY evidence to support that contention.


4. Testing the rBr hypothesis by looking at response rate by partisanship
If the rBv hypothesis was valid, where would you expect the lowest voter response rates to be? I would think that the lowest voter response rates would be most likely to occur in precincts that leaned heavily to Bush – since the hypothesis postulates that the reason for the biased polls is reluctance of Bush voters to participate in these polls, more Bush voters should mean lower overall response rate.

However, when USCV analyzed the data presented in the Mitofsky report, they found exactly the opposite: Precincts with the highest percentage of Bush voters had the highest, not the lowest response rate. This must certainly count as evidence against the rBv hypothesis.


5. Testing the revised rBr hypothesis by looking at WPE by partisanship
But don’t yet give up hope on the rBr hypothesis. It can be revised to say that, although Bush voters in general were more reluctant to participate in the polls than Kerry voters, this did not apply to precincts where there were a very high percentage of Bush voters, because in those precincts the Bush voters would perhaps feel more comfortable participating in a poll.

This revised hypothesis can also be tested. If the rBr hypothesis applied only to precincts without a heavy preponderance of Bush voters, then one would expect that those precincts would be where the bias (i.e., WPE) would be found. But in fact, by Mitofsky’s own data, precisely the opposite is the case: The average WPE is highest, not lowest, in precincts where there were a very high percent of Bush voters (80% or more). This too must count as further evidence against the rBr hypothesis.


6. Consideration of election fraud by assessment of voting machine type
As noted above, Mitofski dismisses the possibility of election fraud in his executive summary by saying that his polling found no evidence for it. To support this contention he notes that they “… found no systematic differences for precincts using touch screen and optical scan voting equipment”. This is the sum of his evidence for the absence of election fraud.

But both touch screen and optical scan machines count the votes by computer. And so do all other methods of vote tabulation except for the hand counting of paper ballots. Deep in the report, but not in the executive summary, is the average WPE data by type of voting equipment:
Paper ballot: -2.2
Mechanical voting machine: -10.6
Touch screen -7.1
Punch cards -6.6
Optical scan -6.1

Note that the average discrepancy (as manifested by a more negativeWPE) is considerably less in precincts where paper ballots were used, compared to any other method. USCV reported this finding but did not make a big deal of it, because they didn’t feel that it was conclusive, and they were concerned about the possibility of over-interpretation.

There are two problems with attributing this finding to election fraud. First, there were no tests of statistical significance presented by Mitofsky. It is not possible to assess the statistical significance (i.e., role of chance) of this finding without the raw data, and Mitofsky is not releasing the raw data to anyone. Secondly, it is possible that this data might be “confounded” by some other factor – for example, the WPE was less negative in rural areas than in urban areas, and rural areas were more likely to use paper ballots, and therefore, the use of paper ballots in rural areas could partially explain this finding. I find this somewhat unlikely, partly because the spread in WPE between urban and rural areas was less than the spread between paper ballots and other types of voting. Therefore, the use of paper ballots in rural areas could not possibly explain all of this discrepancy by voting machine type. However, it could explain some of it. Again, it is impossible to test this hypothesis without access to the raw data. Nevertheless I certainly believe this finding to be suggestive of election fraud.


7. Comparison of election fraud by assessment of swing states versus non-swing states
If the 2004 Presidential election was fraudulent, one would expect more fraud to have occurred in those states where there was a reasonable chance of switching their electoral votes from Kerry to Bush (i.e., the swing states). Of the 11 main swing states (OH, FL, PA, WI, MN, NM, IA, NV, NH, MI, CO), according to Mitofsky, in five of them there was a discrepancy between the exit polls and the official election results that were outside of the margin of error (I’m defining outside of the margin of error as less than a 5% probability of occurring by chance). These five swing states included Ohio, Florida, Pennsylvania, Minnesota, and New Hampshire. Of the remaining 39 states, only 8 were outside of the margin of error (In all 13 states that were outside of the margin of error, the exit polls favored Kerry, compared to the official election results). I believe that this finding also supports the suggestion of election fraud.


8. Comparison with previous elections
Mitofsky admits in his executive summary that the “exit poll error” was higher in 2004 than in previous years for which he has data (going back to 1988). I have a hard time believing that this higher discrepancy in the 2004 election and its concurrence in time with a greater ability than ever to use secret software codes to fix elections is just a coincidence. It’s possible, of course. But when taken together with everything else, I think it is very suspicious.


9. The controversy between USCV and Febble
I mention this last issue only because there has been so much discussion on this, and I would like to attempt to put it in perspective. I seriously do not believe that this issue has much importance, compared to any of the issues noted above, and therefore I think that spending too much time on this may be harmful to our efforts to shed light on the main issues.

The controversy over this issue has involved very complex statistical concepts. Because of the complexity of the issue, the language used, and the sheer volume of material, I have been unable to find the time to fully understand this argument, and even if I did have the time I’m not sure that I could understand it well enough to have a valid opinion on who’s right and who’s wrong.

But I do believe that I understand it well enough to know that it has peripheral importance to the central issues, as discussed above. USCV (or at least some of their members) has claimed that the data support going beyond what I’ve discussed above. Specifically, they have devised complex simulations to show that the rBv hypothesis is untenable. Febble, on the other hand, claims that the rBv hypothesis is possible, based on her computations (I don’t believe that she goes beyond “possible”, but I don’t fully understand exactly what she is trying to say, so I may be wrong about that.) In any event, whether the rBv hypothesis is untenable or merely possible, certainly the above noted considerations cast considerable doubt on it.


Summary

For all of the above reasons, plus the existence of volumes of non-exit poll related evidence, I strongly believe that John Kerry won the 2004 Presidential election. But even if others may find the evidence weaker than I do, certainly there is enough evidence to cast enough suspicion on the integrity of the election to warrant serious investigations into this all important issue, as well as concurrent vigorous efforts to improve the integrity of our election system. This effort should include, among many other things, public access to the raw exit poll data, as well as the secretly programmed voting machines that determined the results of our last Presidential election.