The Gun Is Smoking! Election Archive Releases Analysis of OH!

[Stevepol]

Kathy Dopp and National Election Data Archive (NEDA) has just issued the data about the OH results.





The Gun is Smoking - 2004 Ohio Precinct-Level Exit Poll Data Show Virtually Irrefutable Evidence of Vote Miscount





The National Election Data Archive (NEDA) is the first mathematical team to release a valid scientific analysis of the precinct-level 2004 Ohio presidential exit poll data. NEDA's analysis provides virtually irrefutable evidence of vote miscount.

(PRWEB) January 17, 2006 -- There is significant controversy about whether the 2004 presidential election was conducted fairly and its votes counted correctly. According to results of the major national election exit poll conducted for the National Election Pool by Edison/Mitofsky (E/M), Kerry won Ohio's pivotal vote, though the official tally gave the state, and thus the presidency, to Bush. The conduct of Ohio's election was formally debated by Congress in January 2005.

The National Election Data Archive (NEDA) is the first mathematical team to release a valid scientific analysis of the precinct-level 2004 Ohio presidential exit poll data "The Gun is Smoking: Ohio 2004 Exit Poll Discrepancies Are Consistent with Outcome-Altering Vote Miscount" available at http://electionarchive.org/ucvAnalysis/OH/Ohio-Exit-Polls-2004.pdf. NEDA's analysis provides significant evidence of an outcome-altering vote miscount.




The news release mentions several precincts where the exit poll / voting result discrepancy was far beyond chance, indicating fraud. Link: http://www.prweb.com/releases/2006/1/prweb333209.php

[pat_k]

1. Kick'n it. Recommending! n/t

[rd_kent]

2. So what happens next?

Can we insist that Bush step down and Kerry step in as the rightful president?

[LoKnLoD]

3. Oh...

that would be too sweet, but I would feel sorry for anyone who has to take over this country, it's a mess.

[Febble]

4. Well, if you believe that the analysis is valid

which I don't.

But then who would you trust?

[Chi]

66. Off topic question.

If reluctant (Bush) responders caused an inaccurate exit poll, why wouldn't they cause the same slant in pre-election phone polls?

[MrModerate]

5. What's scary is that this won't mean anything . . .

Americans simply don't believe that the election was stolen. Even those who recognize that 2000 was stolen don't buy this.

I'm not sure what you need to do to get the point across, short of a few whistleblowers coming forward to say "Yes, I cooked the tabulator."

[Febble]

14. Yes, I admit it.

I think that Gore won Florida. I don't believe Kerry won 2004.

The way you need to get to point across to people like me through some decent data analysis of decent data. This is not decent data analysis, and the authors themselves complain about the quality of their data.

[roseBudd]

29. you say you don't believe Rove stole 2004...

Do you honestly believe Rove would allow Bush to lose in '04?

Do you honestly believe Blackwell wasn't willing to do anything to prevent a Kerry win?

Anything.

Do you honestly believe mass psychosis caused Bush voters in SW Ohio to pull C. Ellen Connaly's name out of their asses even though they had NEVER heard of her.

I live in SW Ohio BTW.

[Febble]

31. I was perfectly prepared to believe

Rove would have done if he could. But I think the evidence suggests he didn't.

I think the evidence strongly suggests Blackwell was willing to to a heck of a lot to prevent a Kerry win. I'd like to seem him in jail. But I don't think the evidence suggests he massively switched votes, although I keep a pretty open mind on Ohio.

But I don't think Kerry won the popular vote.

[roseBudd]

42. Yet you expect people who have absolutely NO ACCESS to the evidence...

to prove it to you. I truly wish we did have the right to request forensic analysis of the ballots, poll books and the machines. Blackwell ordered the poll books be locked down, remember.

I would like to know how many of the punchcard ballots in Warren County have fingerprints on them, since a ballot once it is detached from it's stub has no identification to indicate what batch it came from.

I would like to be able to analyze the code the tabulation programs use.

So please explain what I can do to prove it with the limited resources of a citizen who has no right to an accurate vote count.

[OnTheOtherHand]

43. consider the context of the original post

Look, it's a thread about exit polls. That's what it is.

It's not a thread where Febble (or I) set out to argue, "Bush won Ohio, bwahaha." It's certainly not a thread where anyone set out to argue, "Hey, the election system is transparent, no problems here!"

Pardon if I am saying the obvious, but a lot of folks around here seem to get the impression that if we have the effrontery to find any particular argument for fraud unconvincing -- and to say so -- then we are rooting for Bush. And we get tired of explaining ourselves over and over. Everyone here, as far as I can tell, wants clear evidence about what happened in 2004, and everyone wants an election system that is worthy of their trust.

[Febble]

47. No I don't expect you to prove it to me

But nor do I expect people to claim it has been proven from the exit poll data when it hasn't.

I wouldn't have got into this story if I hadn't wanted to know what happened. I agree that the things you want investigated need investigated. In my view, what you suggest is far more likely to yield useful evidence than the exit poll data. I don't think that the evidence is in the exit poll data, and I don't think this paper provides anything like a smoking gun for fraud. In fact, as I've said downthread, I think it is a very poor piece of analysis.

I think you are right - the information you want is the information you need. Not analyses like this one.

[nashville_brook]

6. exit poll discrepancy -- FIVE times the number expected

Exit Polls were conducted in 49 of Ohio’s 11,360 precincts. At least 40% of Ohio's polled precincts show statistically significant differences between Kerry’s exit poll percent and official vote count percent. 35% of these discrepancies underestimated the Kerry official vote share. This is five times the number expected. Three of the most glaring examples are:

1. In E/M precinct 27, with an estimated 100 respondents, Kerry’s official vote count was 29% less than his exit poll share, creating a 58% difference between Kerry and Bush exit poll and official vote margins. There is less than a one in 867,205,500 chance of this occurring due to chance.

2. In E/M precinct 25, with an estimated 62 respondents, Kerry’s official vote count was 28% less than his exit poll share, creating a 56% difference between Kerry and Bush exit poll and official vote margins. There is less than a one in 234,800 chance of this occurring due to chance.

3. In E/M precinct 48, with an estimated 100 respondents, Kerry's official vote was 16% less than his exit poll share, creating a 32% difference between Kerry and Bush exit poll and official vote margins. There is less than a one in 17,800 chance of this occurring due to chance.

[OnTheOtherHand]

8. my last post on E/M precinct 25 was so good, here it is again

http://www.democraticunderground.com/discuss/duboard.php?az=show_mesg&forum=203&topic_id=397259&mesg_id=397543

Liam_laddie has reported that he interviewed a couple of Democratic Party Poll Observers from Cincinnati 4M. (Of the Ohio NEP polls we have been able to identify, Cincy4m has the biggest WPE. But it does not have the biggest WPE of all the Ohio NEP polls.)

According to the observers, the NEP interviewer at Cincy4m was totally out of his depth. He managed to conduct 31 interviews out of the 1754 voters (times 2, because they went by him twice) fanning in/out from the polling place (the polling place held 4 precincts). It was raining and he was stationed 100 feet away from the door, in a group of campaigners who were vying with each other for the attention of voters entering the polling place.

The observers said that "Republican types" just streamed by him. They didn't refuse to be interviewed. They didn't even notice him.

Maybe the guy was able to note the age, gender and ethnicity of every Nth person in the fanned mob, but I doubt it very much.

He was totally discouraged, took several long breaks and went home early.

Cincinnati 4M appears to be "Mitofsky Precinct Number 25" in the ESI report. (I'm not too worried about jeopardizing confidentiality here, given that (1) I am quoting widely available sources and (2) less than 2% of the voters were interviewed anyway.) Table 1 in ESI's data document reports a 68% Kerry share in the exit poll (that would be Kerry 21, Bush/other 10), with a possible range between 42% and 80%. So we can infer that the interviewer reported about 19 misses and refusals, for a total of 50 attempted interviews. Kerry's percentage would range from 21/50 = 42% to (21 + 19) / 50 = 80%. Looks like a match.

OK, so if an interviewer reports having attempted to interview 50 people out of 1700+ (for the four precincts sharing this polling place), and completes 31 interviews, with a result of 21 Kerry, 10 Bush/other (i.e., 68% Kerry) -- and the final result in the precinct is Bush 312, Kerry 211, other 3 (40% Kerry) -- and the same precinct gave 23% of its vote to the Democratic candidate for governor in 2002 -- who here is prepared to call this virtually irrefutable evidence of vote miscount?

--The above is what I posted back in November, and the report doesn't address it. What's that about?

Folks, again, I'm not giving Ohio a thumbs up, I'm just saying that the exit poll evidence is awful. If you're looking for precincts to recount, Phillips and Mebane & Herron have listed a bunch already. If you're looking for junk science, well, OK.

[mgr]

10. Honestly, you can't know the WPE is accurate to begin with

Ohio's use of multiple precincts at one polling location, as your example indicates, makes it a headache unless one separates the singular sites out. Cinci-4M was confounded with other precincts with smaller margins between Bush and Kerry that would in aggregate reduce the WPE. As your example shows, what does the WPE relate to, the single precinct; all the precincts combined; a multiplier effect with using Mitofski's shoe size as the constant,....

My advice to Ms. Dopp and company is that they better cultivate that professional working relationship with Mr. Mitofski. Save the invective in favor of that cold dispassionate number crunching that shows that he is wrong.

Your example shows why I hate discussing data analysis with most engineers and mathematicians is that they fail to understand why data verification is so important. It is a pleasure to have the statistically adept around.

Mike

[OnTheOtherHand]

22. right, "WPE" (or WPD) isn't actually within-precinct in this case

I don't think that explains the discrepancy in this case, but it does knock some zeroes off NEDA's calculated odds. (Hard to say how many, given that we actually don't even know the sample size.) For what it's worth, by my crude reckoning, the pooled Bush percentage in the official counts for those four precincts is about 53.93%, rather than 59.65% for 4M alone.

When people get hung up on the number of zeroes, they are missing the point. The point isn't that 21 Kerry voters out of 31 respondents (or whatever the actual exit poll result was) probably happened by random chance alone. The point is that if we put down our calculators and think about the facts on the ground, this just isn't strong evidence.

Worse, NEDA's supposed refutation of the ESI analysis actually privileges exit poll results over returns from recent elections. This is a very dangerous move. Folks can be as suspicious as they like of past returns, they should certainly be attuned to legitimate sources of variation, but we should not just chuck overboard the empirical fact that more often than not, the most Democratic precincts in one election tend to be the most Democratic precincts in the next.

[mgr]

26. Hasn't that been the problem with the controversy from the start:

Skepticism without the empirical underpinnings--

"Worse, NEDA's supposed refutation of the ESI analysis actually privileges exit poll results over returns from recent elections."

The thing was, aside from the zeros, is that precinct 25 is specifically flagged by NEDA, is such a flawed sample; and its shortcomings openly discussed on this site; as well as ascribed to in the NEP paper in January.
If that isn't support for NEP's veracity, nothing can be.

What I am curious about is whether these excessive WPEs or WPDs(or WPA for that matter) affect the calculation of the post election weightings for the state (if these are done). After-all, I am not convinced that NEP is immune from GIGO; and no, I am not going into TIA land (he covers it so much better).

Mike

[OnTheOtherHand]

39. I'm not sure what your question is

(Also, I'm not sure what you mean when you say, apparently, that precinct 25 is referred to in the E/M evaluation report from January 2005 -- or did you mean...?)

Whether the WPEs affect the post election weightings? All else equal, the bigger the discrepancies between the exit poll results and the official returns, the bigger the weights have to be. As I understand it, after all the other weighting (e.g. for visible non-response) is done, the weights are adjusted so that the official returns in each "geostratum" match. There may be further constraints, I'm not sure. NEP is surely not immune from GIGO. I think we can try to interpret the results, but we should keep a sense of humor about it.

[mgr]

54. I'm probably trying to suggest too much

That was one invitation to belabor my point of view.

My question is less related to issues of identification of fraud, and more in proper identification of demographic trends--the intent of the exit poll. One of the things I don't think has been discussed is whether the demographic trends are only reweighted after the election results are in for the country, or are they done for the state level as well.

The January 2005 report specifically identified issues with the exit poll conducted in Ohio that suggest a greater potential of problems that would have biased the outcomes--specifically pollsters located 100 feet from the poll that had to be legally remedied. In other portions of the report, where Ohio is not specified, the elevated non-response, and the inexperience of the pollsters are cited. Precinct 25 or Cincy 4M, in the narrative provided by LiamLaddie fits these issues to a 'T'. I suspect it sounded as if I was arguing from an existential to the universal, when I was putting it the other way. Since we have few examples of where NEP polled, and what the outcomes were, Precinct 25 becomes a test of NEP's veracity--that we can conclude that Mitofski was engaged in an honest evaluation of what was occurring in the election, and is not a republican shill. Therefore we can accept with a measure of confidence, his assertion that in the context of the exit polls, the election outcomes are more accurate a measure of the overall voter intent, than the raw results of the polling. Which sure beats the exhortation to expertise that has been made in the past.

The problem which we discussed previously is how did NEP address the multiple precinct problem. It would appear from your discussion with KiwiExpat that WPE was calculated for precinct 25 based upon the outcome from that precinct alone, rather than the combined precincts. Therefore generating a WPE that overstates the sampling discrepancy. If we have several precincts like this in Ohio, the aggregate effect is that the state appears more staunchly red than it is, and that an election strategy similar to Gore 2000 should then be followed.

As to humor, I tend to the bilious :)(Of course, I could cite Avicenna in the Arabic, as others do their Latin, to show this--but that would be sanguine).

Mike

[OnTheOtherHand]

55. exit polls and demographics

It's important to distinguish between direct demographic weighting and weighting to election results that has the side effect of changing demographic proportions. (By which I mean, the estimated percentages of, say, union members in the sample. If you want to know what proportion of those union members voted for Bush, that is yet a different figure that can be indirectly influenced by weighting to election results.)

The results are weighted to compensate for observed (interviewer-recorded) non-response bias with respect to age (three categories, I believe), race, and sex. I am not sure at what level those weights are reckoned. (There is also a sampling probability weight.) Then the weights are further adjusted to match the official returns by geostratum (for state polls) or region (for the national poll).

And, therefore.... Dunno. Apparently there will be a paper at AAPOR in May that explores the demographics, and I am curious how the author will deal with the methodological riddles.

Yes, the E/M report definitely points to some big interviewing problems in Ohio in particular. I agree with your assessment. To me, that discussion of Cincy 4M here on DU offered unexpected independent support for the view that Mitofsky is calling it like he sees it, and he just doesn't have the confidence in his own raw data that NEDA has. (By the way, I think all this also applies to the principal of Edison, whose name is well known, but for some reason he has escaped most of the misdirected wrath, so I won't drag him into the line of fire.)

Some folks here seem to perceive it as contradictory for Mitofsky to be an expert, and to boast about the accuracy record of his calls (setting aside that unpleasantness in Florida 2000), but also to say that his data aren't all reliable. I don't see it that way. The midwives who delivered our older daughter were experts, and they made good decisions -- and the fetal monitor they were required to use obviously generated bogus distress alarms a few times an hour. One measure of their expertise was that they didn't take all the data literally. Nor did they ignore it.

Yes, WPE for precinct 25 was calculated based on the outcome from that precinct alone, which led to a larger error estimate than if the four precincts had been combined. (I can essentially confirm this directly from the ESI data tables.) I don't think this has much bearing on future election strategy, but it certainly should be a wake-up call for NEDA.

I tend toward the melancholic, myself. At least I think I do, but I worry that I might be wrong about that. ;)

[Name removed]

13. Deleted message

Message removed by moderator. Click here to review the message board rules.

[Febble]

15. Absolutely right

Ms Dopp.

[OnTheOtherHand]

19. what part of my criticism did you find invalid, and why?

Stand and deliver. In light of the facts related in the preceding post, is there any rationale for treating the exit poll respondents at that polling place as an unbiased random sample of Cincinnati 4M -- although probably most of them didn't even vote in precinct 4M, but rather in one of the three other precincts in the same polling place? and although observers said that the sample was clearly not unbiased?

Lacking such a rationale, what is the point of the calculation? How does one calculate the standard error of a biased sample?

Incidentally, if you are saying that there were probably 62 respondents at that polling place, not 31, well, that may or may not be true. As I assume you know, E/M reports that there were 114,559 sample voters in the exit poll analysis -- a figure that may not include several thousand telephone interviews -- and there are about 77,000 observations in the ICPSR/Roper Center dataset. I cannot help but notice that 77,000 is substantially more than one half of 114,559. A plausible inference is that E/M half-sampled to save time when necessary, but it is hard to imagine why it would be necessary with only 31 responses over three calls.

Be that as it may, if anyone is reassured because the interviewer may have interviewed approximately 62 out of 1754 voters instead of 31 out of 1754, then I do not know why.

(EDIT to remove three grafs that will make no sense now that the post has been deleted)

[RonB]

46. Why our Critics are Wrong

1) E/M and ESI have continuously played fast and loose with the data. We've found for example that exit polls calculated from the Roper/UMich "raw data" does not match the "exit polls" reported by ESI for the Ohio precincts. In one precinct ESI had reduced Kerry's exit poll value by 9% relative to UMich/Roper. Adding back that 9% would give a huge discrepancy with odds of less one in hundreds of trillions. In another precinct ESI reduce the Kerry exit poll by 6%. This kind of data inconsistency is extremely suspect to say the least - see the paper.

2) I asked Mitofsky about this over a month ago. He had no explanation for it - said he'd "look in to it". I then politely asked him to stand by a full data release to avoid this kind of problem. He ended up cursing me out on e-mail.


3) We "match" the data by ordering ESI and Roper/UMich (to get sample sizes) basically from lowest to highest. These means that in aggregate we are minimizing the discrepancies. Of course individual precincts may be "out of order" - but this would imply that other are also and so generally larger discrepancies will result. This is non "junk science" but a conservative matching method that is the best that can be done given the utter unethical and unscientific methods used by E/M and ESI (no data release, no transparency in calculations, no statistical analysis) - just "trust us" to tell you the truth...reminds one of another powerful group....


4) Even if a few precincts come out better with the actual data (as OTOH claims) this does in any way invalidate the thrust of the analysis which is extremely robust. A large number of precinct discrepancy values can be changes, sample sizes can be halved - but the basic overwhelming and unexplained discrepancy patterns will remain.

5) Even after taking out "pervasive exit poll bias" effects , these discrepancies and the unexplained patterns remain.

6) The ESI papers and Mitofsky presentations relating to the 2000 precinct level values are useless empirically and logically - they don't prove anything - see Appendix in paper on this.

7) There is no way to dodge a real analysis. E/M, ESI, Liddle, OTOH have no substantial basis to claim that these discrepancies can be explained by exit poll error unless and until the link the factors that ostensibly cause this (beyond random) error to the patterns of the error. So far no real statistical explanation has been offered, instead we have inconsistent data "releases" and misleading "proofs" such as ESI's and E/M's in their original report - all non-transparent and with no independent verification.
In contrast all of our methods, calculations, and data, are presented in our report. Who's more scientific?

E/M and ESI have violated very cannon of scientific integrity - this is and should be a national outrage.

[Febble]

48. A response from a critic:

1) E/M and ESI have continuously played fast and loose with the data. We've found for example that exit polls calculated from the Roper/UMich "raw data" does not match the "exit polls" reported by ESI for the Ohio precincts. In one precinct ESI had reduced Kerry's exit poll value by 9% relative to UMich/Roper. Adding back that 9% would give a huge discrepancy with odds of less one in hundreds of trillions. In another precinct ESI reduce the Kerry exit poll by 6%. This kind of data inconsistency is extremely suspect to say the least - see the paper.

This has been explained to you a great many times, Ron, but I'll do it again. And the reason I can do it is that rather than complaining, I bothered to actually find out the answer, which is that the Roper data is the subsample of questionnaire responses that were phoned in in full on election night. No other responses were ever entered into any data base apart from aggregated full totals of the answers to the presidential vote question plus age/race/sex numbers of non-respondents (i.e. those who were selected for the poll but who were "missed" or who refused). The subsample was random, and the randomisation protocol was done at the phone bank end. The calls were made on three occasions. The subsampling was designed to net sample sizes of 50. So if an interviewer was doing well, and was clearly going to achieve more than 50 responses in total, the subsampling protocol would be designed to take a random proportion of the total. If an interviewer was doing badly (as many in Ohio clearly were) the entire response set might have been entered. The point was to get a good haul of responses for the crosstabs. This dataset was then lodged with Roper. So the reason there are discrepancies between the proportions for the presidential question in Roper and the proportions in the ESI dataset is that in most cases, the Roper subset is a smaller subset; however, in some cases it won't be. As a rough guide, where the Roper sample is under 50, it is more likely that the total sample will be small.


2) I asked Mitofsky about this over a month ago. He had no explanation for it - said he'd "look in to it". I then politely asked him to stand by a full data release to avoid this kind of problem. He ended up cursing me out on e-mail.

I am not very surprised. Mitofsky consulted with AAPOR over his stand on the confidentiality issue, and they supported his view that confidentiality issues precluded release. I happen to agree. I also happened to see the text of your emails to Mitofsky. Frankly, I would have got pretty irritated myself at being repeatedly importuned to take a stand that I believed was ethically incorrect. I don't know about economics, Ron, but in social sciences we take a pretty stringent view on the confidentiality of participant data.


3) We "match" the data by ordering ESI and Roper/UMich (to get sample sizes) basically from lowest to highest. These means that in aggregate we are minimizing the discrepancies. Of course individual precincts may be "out of order" - but this would imply that other are also and so generally larger discrepancies will result. This is non "junk science" but a conservative matching method that is the best that can be done given the utter unethical and unscientific methods used by E/M and ESI (no data release, no transparency in calculations, no statistical analysis) - just "trust us" to tell you the truth...reminds one of another powerful group....

Your matching is not conservative, whatever you assert. You don't have good data, as you continually complain. Given that you don't have good data, you cannot draw "virtually irrefutable" conclusions. But ranking as you have done is not a conservative process, nor is assuming that sample sizes will consistent be double the Roper sizes. If very small total sizes happen to belong to precincts in high leverage positions then you will get a wrong result. Your error term needs to include the probability of erroneous sample sizes and erroneous ranking. Both will certainly have larger error. Guesses do.


4) Even if a few precincts come out better with the actual data (as OTOH claims) this does in any way invalidate the thrust of the analysis which is extremely robust. A large number of precinct discrepancy values can be changes, sample sizes can be halved - but the basic overwhelming and unexplained discrepancy patterns will remain.

You are certainly correct in claiming that at both precinct level and at state level the exit polls are way way outside the MoE. But we knew this. ESI also reported it. You consider: non-response error; lying; and clerical error (though you dismiss the last on the stretch of an assumption that clerical error would have been removed through outlier removal). You simply do not consider selection bias, for which we have both anecdotal and statistical evidence. "Proving" over and over again that the exit poll discrepancy was not due to chance does not prove that it was fraud; it certainly does not prove it in the face of substantial evidence that the poll was biased; and it certainly does not prove it in the face of evidence that exit poll discrepancy was not correlated with Bush's vote share after (and, in fact, before) controlling for his vote-share in 2000. Kathy's claimed "proof" that this analysis is illogical is, as you must know, no such thing. If it were, it would invalidate your own analysis as well.


5) Even after taking out "pervasive exit poll bias" effects , these discrepancies and the unexplained patterns remain.

If you take out less than there is, some will remain. You have no a priori information concerning the size of the effects. If you did, we wouldn't need to do the analysis. To find out the size of exit poll bias effects (I have no idea what the word "pervasive" means in this context) you have to regress your discrepancy on some predictor. I am not aware that you have done this. I am not aware that you can, because, as you say, you don't have the data.

6) The ESI papers and Mitofsky presentations relating to the 2000 precinct level values are useless empirically and logically - they don't prove anything - see Appendix in paper on this.

Well, I beg to differ.

7) There is no way to dodge a real analysis. E/M, ESI, Liddle, OTOH have no substantial basis to claim that these discrepancies can be explained by exit poll error unless and until the link the factors that ostensibly cause this (beyond random) error to the patterns of the error. So far no real statistical explanation has been offered, instead we have inconsistent data "releases" and misleading "proofs" such as ESI's and E/M's in their original report - all non-transparent and with no independent verification.
In contrast all of our methods, calculations, and data, are presented in our report. Who's more scientific?

Saying no explanation has been "offered" (actually it has, but I agree, the E-M report does not give statistical details) is not the same as saying no explanation is possible. You are right that no analysis can be independent of those responsible for the data collection. It is, unfortunately, the way life is. E-M collected the data. Any analysis is going to involve their collaboration. However, given that proviso, ESI's analysis was independent (note that Dingman and Mitofsky, at the E-M end, are listed in the author list as "with" not "and"), and is currently undergoing peer-review. I believe they hope to extend it to other states. This would seem to me to be the correct scientific procedure, given the need for respondent confidentiality. Fritz Scheuren ex-president (apparently, I was out of date) of the American Statistical Association actually invented the "blurring" technnique which was used in their Ohio study. I'd say the approach was pretty scientific.


E/M and ESI have violated very cannon of scientific integrity - this is and should be a national outrage.

Your case seems to be that by conforming to the ethical guidelines of their own professional organisation they have violated scientific integrity.

Ummm.....

[OnTheOtherHand]

49. one nitpick about AAPOR

I'm not aware that AAPOR as an organization ever took any position on whether Edison/Mitofsky should or should not release the data -- although, based on the AAPOR Code, there is a solid argument for "should not." Reportedly the American Statistical Association's Privacy and Confidentiality Committee did endorse E/M's position.

Ron seems either not to understand or not to care that "continuously play(ing) fast and loose with the data" is a very serious charge, and he has very weak evidence to support it.

[Febble]

51. You are correct

it was the ASA confidentiality committee who investigated the issue at Mitofsky's request, not AAPOR. I just checked.

[OnTheOtherHand]

50. "continuously played fast and loose with the data"

If you had convinced survey professionals of this claim, you would probably get more support from them. But you haven't, because as Febble explained, your matching algorithm isn't reliable. You are making a very strong claim on very weak evidence. You have offered no reliable warrant for your vague complaints about "inconsistent data 'releases.'" Not have you justified your dismissive reference to "misleading 'proofs' such as ESI's and E/M's." As far as I can tell, the words "proof" and "prove" do not even appear in their work. Unsubstantiated accusations that powerful people are manipulating the data to obscure the truth are, in my professional opinion, fairly characterized as junk science.

Reporting gee-whiz statistical odds as if they prove something in particular is, similarly, a cardinal error of statistical analysis. Most survey professionals, if they have considered this issue at all, have concluded that sampling and non-response bias are very likely in Ohio ("'Republican types' just streamed by"). Your P values offer no basis for rejecting that hypothesis. If you really believe that the maximum plausible non-response bias in Cincinnati 4M (or any other Ohio exit poll precinct) is 1.12:1, you have little company in this belief.

You have alluded to "Mitofsky #4," which ESI reports to have an exit poll percentage of 70% Kerry compared with an official vote percentage of 55% Kerry (but which you apparently believe actually ran more like 79% Kerry). Note that ESI reports a 15-point difference between Kerry vote share and exit poll share, which likely implies about the same difference in Bush shares. Looking at ESI's Figure 3, we can infer that Bush's vote proportion in 2000 was probably within a few points of his 2004 vote proportion. Most political scientists will therefore regard it as a priori unlikely, absent other evidence, that Kerry actually had a precinct vote share over 20 points higher than Al Gore's. It's not a matter of "proof," but it is certainly a matter of learning from experience.

Of course, if ESI and E/M are faking their data, then who knows? One can always can argue that any anomalies evince fraud in the vote count, and any non-anomalies evince fraud in the data. However, this argument is not likely to convince any independent analysts. This is perhaps why so few of them even bother to report that they disagree with you.

But I do not want any DU reader to be able to claim, "No one ever tried to explain to me why NEDA's analysis was wrong. And no one ever defended the integrity of the experts who disagreed with it." People can believe what they want, but if they want to be progressives, they shouldn't do so in a bubble.

[RonB]

56. More Evidence on E/M Deceptive Data

OTOH and Febble,

I've got plenty of evidence.

For example, E/M states in its methods statement that Ohio had 41.7 average respondents per exit poll. I originally did all the calculations with this average (again very high number of unexplainable discrepancies).

It so happens that UMich/Roper data for Ohio averages out to exactly 41.7.

When I found the discrepancies between ESI and Roper/UMich I mentioned that the UMich/Roper data averaged out to exactly the number stated in the Methods statement to Mitofsky. He dismissed this (never commented on it) and instead found some obscure statement in their Jan 19 report that states that UMich/Roper data is just a "sample" of the data. He claims "everyone" knows this. I guess the guys that put out the methods statement on a very public national website don't!

I wonder how many academics and news organizations know that the E/M "raw data" release is just a sample of roughly half the data that doesn't agree in key precincts with the "actual" data - that only E/M and ESI and I guess you and Febble have access to. E/M have certainly not make this clear - to the contrary. Looking at their most public and visible statements one would conclude that this is "the data".

Now, of course, if you double the average sample sizes(as Mitosky now claims you should) you get even more significant discrepancies.

You can play with these numbers all you want - but you can't make the basic findings go away.

Except perhaps by dancing around with different and inconsistent data as E/M and ESI
have done.

OTOH, you're a smart guy and have caught some important errors in past work - but in the interests of truth you should be able to see the light now.

[Febble]

57. Well, Ron, check your email

That number is the number of questionnaires in the crosstabs. That's why it matches the Roper data, because the Roper data is what is in the crosstabs.

But we know that the crosstabs were a subsample, supposed to be 50% but actually rather more this year, probably because the response rate was low, but I don't actually know that.

And if all you did was double the Roper numbers you won't have got them right, and more to the point you will have missed a major trick because small numbers in the Roper data will tend to indicate precincts where the interviewer was struggling. They were aiming to get 50 in the subsample, so if there are only 20, it may mean there are only 20 in the total. But it may not. There is sampling error in there, and presumably the randomisation protocol was set on the first call, so if the response rate changed during the day, or simply the voter numbers changed during the day, the final sub-sample size might or might not match the intended sub-sample size.

The point is you can't tell. And if you can't tell you can't do the stats. And if you can't do the stats you can't conclude there is a smoking gun.

Also, remember the problem with multiple precincts in polling places. It's in my email.

And please stop alleging that OTOH has access to the data. He hasn't. I think you know this, and it is only going to create trouble if you post in a public place that he has.

I know these things are not all that clear, but if you spend enough time thinking about them they become clear. As it should have become clear to you, while preparing your last smoking gun, that the numbers didn't add up. That was because you had misunderstood the nature of the data. OK, as I said, the data is confusing. But your job is to puzzle it out, not to run analyses on misunderstood data, and then blame other people for not explaining it to you properly.

[OnTheOtherHand]

58. evidence of what?

I just wasted some time trying to explain this to you in e-mail, not realizing that you had already made it public. What do you consider deceptive in the fact that the N in the methods statement matches the N in the UMich data? Why shouldn't it? Do you think that Mitofsky is somehow, and for some reason, trying to conceal the existence of another 35-40,000 interviews?

(I will try not to repeat Febble's points, although I don't mind repeating my own as necessary.)

You seem so invincibly confused about the basic workings of the exit poll that I'm not sure I have an audience for this post. But I will try. All exit poll tabulations -- i.e., all splits of Bush vs. Kerry by gender, by party ID, by whatever -- are based on individual questionnaire data, essentially the same data incorporated in the UMich dataset, aka the "subsample" data. When people analyze those tabulations, the subsample N is the N that matters; questionnaires for which only the votes were tallied are irrelevant.

In fact, those questionnaires are irrelevant for most purposes approximately fifteen minutes after the polls close. Assuming that Mitofsky and Lenski took their own advice, as the CNN Decision Team reflecting on the lessons of Florida 2000, on how to proceed in the future, then we know that they would call a state based on exit poll interviews alone only if its T score was over 4 -- that's something like 30,000:1 odds. Otherwise, they would disregard the exit poll interviews, and wait to call the state based on vote counts alone. Even if they haven't gone so far, the influence of an extra "N" interviews, while all the interview data are being replaced with vote count data as quickly as they can obtain it, must be minimal.

I have no idea how many academics even care about these points, but the subsample was discussed in Mark Blumenthal's blog in mid-November 2004, and -- as you apparently concede -- it was addressed in the E/M evaluation report.

Mitofsky said on PBS a few days after the election that the exit polls had overrepresented Kerry voters. That is his acknowledgment of a discrepancy and his (very broad-brush) explanation of its origin. You disagree with the explanation, and everyone agrees that there is a discrepancy; no one has tried to make it go away. However, you are now claiming in public that E/M and/or ESI have engaged in some sort of deception in or about the data. You should either document this claim or withdraw it. So far, your conviction that the precincts just don't seem to match up right does not seem to rise to the level of proving misconduct.

I will reiterate Febble's statement, lest anyone be confused: I do not have access to any more data than you do, Ron. Mitofsky has supplied me with a few plots, but I do not have the underlying data.

[Name removed]

21. Deleted message

Message removed by moderator. Click here to review the message board rules.

[OnTheOtherHand]

23. self-delete

[Febble]

24. Well, I'm not, Ron.

I was hired by Mitofsky to run some analyses. I'm not paid to post on DU. If only.

I do not believe your assumptions are "conservative"; I don't believe you can do the analysis you claim to do without the data. And the fact that you don't have the data is not my fault nor that of OTOH. I have no idea why you think that OTOH "has the exact data".

And because you don't have the data, your analysis is not "straightforward". It relies on a great many assumptions and each assumption has an error term, which your probability estimates appear to ignore. One of your assumptions appears to be that because the subsamples are about half the total sample, that each precinct sample size is double the subsample size. This is unlikely to be the case, particularly in the case of small subsamples, which are likely to comprise a much larger proportion of the whole.

Nonetheless, I quite agree with you that the discrepancy between the exit polls and the vote counts in Ohio were not due to chance. They were due to something. The question is: what?

You mention two sources of polling error: non-response error and lying. You do not mention selection bias, which is likely to be large source of error. You therefore commit the fallacy of the excluded middle. You also make brave assumptions as to the way that the pollsters might have dealt with outliers. You seem to assume that the outliers are somehow too small to be clerical error but to great to be anything other than fraud. And yet you concede that some precincts were removed because of egregious error. You simply do not know what criteria Mitofsky used for deciding on outlier criteria. As a good data analyst I expect he, like you, that he was conservative.

I am not feeling very polite, myself, Ron, so I will forgive your description of my words as "dribble". I am pretty fed up at having my own analyses described in far worse terms, so yours seems quite mild.

I suggest if you want your paper to be taken seriously, you submit it for peer-review, as ESI have done with theirs. But before you do so, I suggest it is reformatted into a comprehensible form, with a proper methods and results section, and that some of the mis-paraphrases of the conclusions of other analysts are checked for accuracy.

I also suggest that if you and your co-author want to be taken seriously by the scientific community in general, that your co-author stops publicly accusing other analysts of lying. Even if you think its true. Which, in my case, it isn't.

I also respect your own integrity as a data analyst. But I do not think your analysis is valid. I do not think that there is a significant correlation between redshift and Bush's vote share, and I do not think you have demonstrated that there is. Moreover, I would maintain that the ESI study, which also controlled for the variance in Bush's vote share expected on the basis of his vote share in 2000, to be perfectly valid, and they have publicly released an analysis that demonstrates that after controlling for vote-share in 2000, there is no correlation. Similarly, on a national scale, there is no correlation either. Dopp's paper claiming to have proven that these analyses are illogical, is, I believe, simply wrong. I have attempted to explain why. So has Mark Lindeman.

Nor do I think that your measures of "K" and "B" are valid, and I have also attempted to explain why.

I would be happy to agree to differ were it not for the stream of vitriolic criticism directed at my own intellectual and moral integrity, and that of other respected, decent data analysts, by your co-author.

[Febble]

28. Sample sizes

Here's a chart of the Roper subsample sizes by state. Two states (Maine and Texas) had a couple of precincts with outlyingly large sample sizes (Maine and Texas), so I truncated the axis to show the interesting stuff at the bottom. As you can see, the Ohio subsample sizes are low. This suggests that the actual voter samples were also low - it would also be a safish bet that where the actual voter sample were below what seems to be the target size of 50 (drawn as the horizontal reference line), a much larger proportion, if not all, the questionnaires would be included in the subsample.

I've posted the plot because Roper Sample sizes themselves are informative. Sample sizes below 50 in the Roper set suggest a likely small total voter sample. And where voter sample size is low, you'd expect more bias, either from non-response bias or selection bias. In other words either a lot of people were refusing, or interviewers, like the one Liam_laddie reports, were not randomly sampling.

So unless you can match actual sample sizes to actual precincts, you can't do the analysis that this report claims to do. You certainly can't just double the Roper numbers, even if the total Roper numbers are about half the total voter sample. Aggregate data is not a substitute for actual data points.

I realise it is frustrating to want to do analysis and NOT to have the actual data. But NOT to have the actual data, and then to do the analysis, and then to claim that "Precinct level Ohio exit poll data show virtually irrefutable evidence of vote miscount" seems, well, a stretch.

[mgr]

30. small observation on the chart

I can understand the omission of Oregon, but where is Arizona and Washington?

Mike

[Febble]

32. Ah, good point.

Nothing suspicious. I'll try and dig them out. I should have them somewhere, but it may take me a couple of days.

[mgr]

34. I don't agree, it's very suspicious n/t

[Febble]

35. No it's not

it's fixed. Magic!

[mgr]

36. Ah, now you've gone and scrubbed the site.

[Febble]

37. I am just doing it

to mislead you.

[mgr]

53. Now to get surgery

to detach my tongue from my cheek.

Mike

[Febble]

9. There is absolutely no doubt

that the discrepancy between the exit poll and the vote count in many precincts in Ohio, and in the state as a whole was not due to chance.

It was due to something else.

This analysis claims that to demonstrate that "the gun is smoking" - that the evidence favours miscounts over mispolling as the "something else".

I do not think the evidence does any such thing, and I do not think this analysis is valid. I am happy to say why later. But for now, I will simply register that I think the analysis is shoddy, misleading and irresponsible.

It is shoddy because it repeatedly mis-states the conclusions made by other respected data analysts, including, it might be said, the president of the American Statistical Association.

It is misleading because it conveys a level of probability that is simply not supported by the data.

It is irresponsible because there is plenty of good evidence that the election in Ohio was unjustly conducted and that Kerry was disproportionately the loser; adding bad statistical arguments to good evidential arguments simply damages the credibility of the case for a thorough investigation of What Went Wrong in Ohio.

And no, I'm not feeling particularly polite today. Any author who can on the same day post a paper claiming some kind of mathematical objectivity, and the kind of obscene ranting against other analysts (including myself) that Kathy Dopp posted today on another forum (see post above for link) raises questions as to her own capacity for objective analysis.

Elizabeth Liddle

(edited for error made in fury)

[Name removed]

11. Deleted message

Message removed by moderator. Click here to review the message board rules.

[Febble]

12. Well, not exactly.

Tell me why you think so.

[Name removed]

16. Deleted message

Message removed by moderator. Click here to review the message board rules.

[Febble]

17. If you want your posts

to remain undeleted, I suggest you stop alleging that the DU moderators can be manipulated.

And if you want to respond to my post I suggest you do so here.

[Name removed]

18. Deleted message

Message removed by moderator. Click here to review the message board rules.

[Febble]

20. Kathy, I have absolutely

no problem with posts in which you propose a civil counter-argument. I have I huge problems with posts in which you impugn my integrity. I don't expect the mods like it either.

If you want a debate, I am happy to debate. I am happy to discuss areas of disagreement. I am even happy for you to disagree with my arguments.

But I am not happy with "unsubstantiated name calling and labeling". I find it neither impressive nor convincing.

[mgr]

25. Someone is confusing message with messenger

I'm not quite sure what the Delilah analogy is all about, but I don't find it personally convincing, but find it sexist, and offensive. My gloss is that you might think that you have the strength of argument, and all that is keeping Febble in the race is her feminine wiles, but I will assure you that would be fallacious. You suggest that you are losing the battle of rhetoric, but you do not put forward the Socratic clarity that puts the sophists to run necessary to convince me of your superior argument. Your argument to put it bluntly is tawdry, and more befitting a wife of an adulterer speaking of the harlot.

Your statement that you don't find the discussion on the level here does not wash. If your posts were deleted, it is because they violated the rules. You would certainly be free to re-post that information which verifies the 'why' of your claim that refutes Febble without the personal attack, ad hominens, et. al. that got the original post deleted (I don't see that you've been tombstoned).

It is incumbent on you, for the members of the progressive movement you may represent, to put forth the best diplomatic face that is possible; and if you cannot, to abdicate your responsibility in favor of others. It hurts the cause.

What I may personally recommend is that you take a few days off, get ten hours of sleep each day, and to re-read what others have said, and what you have said. What you may recognize is that you confuse a personal attack/ad hominen that is an inference; with conclusions that appear to be so that follow from inferences. IOW, It may be persuasive to show an argument's failings before claiming they are bunk.

Mike

[kster]

7. Kick-n- Recommend....nt

[mgr]

27. With all constant reference to a smoking gun,

I'm beginning to get the image of a Colt-45 with a lit cigarette in the barrel.

Mike

[Wilms]

33. .

[OnTheOtherHand]

40. whoa! that's just amazing -- I tip my hat n/t

[Febble]

41. My personal favorite:

[FogerRox]

59. I am constantly amazed by Wilms and the pics he finds

[foo_bar]

38. or an oversized flag that reads "Bang!"

It's a lethal weapon in the Looney Tunes universe.

</schadenfreude>

Okay. Back to work everyone, if you want your performance bonuses.

[Imagevision]

44. People with common sense know this election was illegal

[kster]

45. Great post Stevepol 20 votes,104 votes and counting

on similar post in Gd by Kpete Keep spreading the truth. :thumbsup: http://www.democraticunderground.com/discuss/duboard.php?az=view_all&address=364x161871

[papau]

52. today's typo corrected version of the email to the media below:

Media Release: January 16, 2005 by the National Election Data Archive

The Gun is Smoking - 2004 Ohio Precinct-Level Exit Poll Data Show
Virtually Irrefutable Evidence of Vote Miscount
http://www.prweb.com/releases/2006/1/prweb333209.php

There is significant controversy about whether the 2004 presidential
election was conducted fairly and its votes counted correctly.
According to results of the major national election exit poll conducted
for the National Election Pool by Edison/Mitofsky (E/M), Kerry won
Ohio's pivotal vote, though the official tally gave the state, and thus
the presidency, to Bush. The conduct of Ohio's election was formally
debated by Congress in January 2005.

The National Election Data Archive (NEDA) is the first mathematical team
to release a valid scientific analysis of the precinct-level 2004 Ohio
presidential exit poll data "The Gun is Smoking: 2004 Ohio
Precinct-Level Exit Poll Data Show Virtually Irrefutable Evidence of
Vote Miscount" available at
http://electionarchive.org/ucvAnalysis/OH/Ohio-Exit-Polls-2004.pdf.
NEDA's analysis provides significant evidence of an outcome-altering
vote miscount.

The analysis is based on the most accurate statistical method yet
devised for determining whether exit poll error, random variations, or
vote count manipulation cause the discrepancies between exit polls and
official vote tallies. This analysis method was made public recently by
NEDA in "Vote Miscounts or Exit Poll Error? New Mathematical Function
for Analyzing Exit Poll Discrepancy" available at
http://electionarchive.org/ucvAnalysis/US/Exit-Poll-Analysis.pdf

Exit Polls were conducted in 49 of Ohio’s 11,360 precincts. At least
40% of Ohio's polled precincts show statistically significant
differences between Kerry’s exit poll percent and official vote count
percent. 35% of these exit polls overestimated the Kerry official vote
share. This is five times the number expected. Three of the most
glaring examples are:

1. In E/M precinct 27, with an estimated 100 respondents, Kerry’s
official vote count was 29% less than his exit poll share, creating a
58% difference between Kerry and Bush exit poll and official vote
margins. There is less than a one in 867,205,500 chance of this
occurring due to chance.

2. In E/M precinct 25, with an estimated 62 respondents, Kerry’s
official vote count was 28% less than his exit poll share, creating a
56% difference between Kerry and Bush exit poll and official vote
margins. There is less than a one in 234,800 chance of this occurring
due to chance.

3. In E/M precinct 48, with an estimated 100 respondents, Kerry's
official vote was 16% less than his exit poll share, creating a 32%
difference between Kerry and Bush exit poll and official vote margins.
There is less than a one in 17,800 chance of this occurring due to chance.

There are also two precincts where the Bush official vote count is
significantly less than the Bush exit poll share. The number of
significant discrepancies and the pattern of Ohio's discrepancy shown in
the NEDA report provide strong support for the conclusion that vote
count errors converted a Kerry win to a Bush win.

New electronic voting equipment without voter verified paper ballots,
implemented under the 2002 Help America Vote Act, makes it easier for a
small number of people to manipulate vote counts and nearly impossible
to independently audit vote count accuracy. Virtually every county in
America today publicly reports its vote counts in a way that hides
evidence of miscounts. This allows those with access (whether
authorized or not) to manipulate or make mistakes in vote counting with
negligible possibility of detection.

Without accurate elections, America is not a democracy. NEDA urges the
media to publicize the results of this report and its recommendations,
in order to return to the American people their right to determine the
country’s leaders.

The National Election Data Archive is a 501(c)(3) nonprofit organization
whose mission is to scientifically investigate the accuracy of elections
through the creation and analysis of a database containing
precinct-level vote-type election data for the entire United States. By
making detailed election data publicly available NEDA furthers its goal
of providing the means for independent analysts to evaluate the
accuracy of vote counts in time to ensure that properly elected
candidates are sworn into office following future elections.

For further information contact:
Kathy Dopp, MS Mathematics, NEDA, President [email protected]
435-658-4657 or
Ron Baiman, PhD, Economics, NEDA, Vice President, [email protected]

National Election Data Archive (NEDA)
http://electionarchive.org
--------------------------------

To receive our email announcements, please email
[email protected]

Common-sense solutions to ensure accurate vote counts:
http://electionarchive.org/ucvAnalysis/US/election_officials/Audits_Monitoring.pdf

Please donate to our 501(c)(3) nonprofit to help implement NEDA:
http://electionarchive.org/fairelection/donate.html

NEDA brochures are available:
http://electionarchive.net/docs_pdf/info/US/USCVbrochure.pdf

National Election Data Archive
P.O. Box 682556
Park City, UT 84068

[RonB]

60. Matching Algorithm Makes NO Difference to Key Point of Paper

Febble and OTOH,

The Bottom line:

Even if you're correct, and the matching is off and this leads to some changes in levels of significance for some precincts. Not only would these additional discrepancies between the ESI and UMich/Roper exit polls have to be explained but - and this is key:

NONE OF THIS WOULD MAKE ANY DIFFERENCE TO THE MAJOR FINDING OF THE PAPER OF OVERWHELMINGLY ONE-SIDED PRECINCT LEVEL DISCREPANCIES IN OHIO WHICH REVEAL A STRIKING PATTERN THAT MANIFESTLY CANNOT BE THE RESULT OF "PERVASIVE BIAS".

Look at the graphs on p. 14 of the paper. These are of ALL DISCREPANCIES, taken directly from ESI exit poll and blurred official result data - THEY INCLUDE NO MATCHING AND NO SIGNIFICANCE CALCULATIONS.

The patterns are striking (especially in the graph ordered by Kerry exit poll - before "the shift" - if there was one). Very large Kerry discrepancies throughout the sample and much smaller Bush discrepancies at only one end of the sample.

As you know pervasive Kerry bias would produce an "inverted U" pattern of Kerry discrepancies - larger in less partisan districts and smaller in more competitive districts . This data shows no such pattern.

Random non-sampling error would produce more or less equivalent Kerry and Bush discrepancies across the sample. Again - no such pattern.

If there is an "exit poll" error explanation for this, it is has to include a specific linking of exit poll error factors to this pattern. No such explanation has been forthcoming and the data have not been released to any independent analyst for investigation.

In the absence of a serious explanation of this sort, and given this record of withholding data and inconsistent, unexplained, and misleading data releases, and reports, and given the massive amount of on-the-ground evidence of pro-Bush election fraud in Ohio (that goes way beyond simple voter suppression), I think, and I think most reasonable and open minded people who are familiar with these facts would conclude, that the weight of the evidence clearly supports an "exit poll fraud" hypothesis.

[OnTheOtherHand]

61. unsubstantiated allegations of deception do make a difference

On your own account as I understand it, someone has jiggered some precinct exit poll percentages -- and someone has been deceptive about the exit poll sample size -- even though these points of fact have no bearing on your basic findings.

It is fairly common around here to be freehand about inferring fraud from the existence of a motivation for fraud. But you seem to posit deception even in the absence of apparent motivation. Go figure.

One simply cannot have huge exit poll discrepancies favoring Kerry in precincts that are already heavily Kerry in vote share. I don't see how your plots support much stronger an inference than that. Have you attempted a statistical argument as to why the data are more consistent with one causal mechanism than another, or are you stuck at "just look and see"? This is very frustrating, because I basically have to argue your side for you before I even get around to arguing my side -- and I am no longer willing to try to do your work for you.

Suffice it to say that you have rebutted a claim that I doubt anyone ever made, namely that the exit poll bias is equal in every precinct. I think, if anyone had harbored that belief, one glance at the scatterplots from last May probably disabused them. So what? The anecdotal evidence from Cincinnati 4M, and the silent scream of the approximately 17% reported completion rate in precinct #27, tend to confirm the view that the Ohio data are full of noise. You think you have found a pony in them, but my colleagues (rightly or wrongly) continue to write about why Bush won Ohio -- not why Kerry did.

You are welcome to believe that the professional consensus of political scientists who have studied the Ohio 2004 returns is other than reasonable and open minded. Like it or not, most of us will lean more heavily on e.g. Mebane and Herron's analysis of over 11,000 precincts than on your complaints that the exit poll results in 49 precincts have been insufficiently considered or explained. Ergo, like it or not, if you want to get any traction among political scientists, you will need to engage work such as theirs. And I will caution you that merely asserting the existence of countably infinite counterexamples will not score you many points.

(EDIT: But don't we have evidence of wrongdoing in Ohio? Yes, we do, some of it documented by Mebane and Herron. Much of it applies to vote suppression, and so the exit polls will not measure it, with the possible partial exception of uncounted provisional ballots. Could this wrongdoing have swung Ohio? Some of us have been trying to figure that out!!!)

[organik]

62. Enough of Ohio and exit polls!

I'm sick of this focus on Ohio - yes it was THE swing state - but I feel the problem is a much larger "culture of corruption" if you will.

Has anyone here that's read Mark C. Miller's "Fooled Again" not think this election was stolen?

It's the idea among the righties, the christian fundamentalists that the end justifies the means - they would do absolutely anything to win, and did. 10,000 frauds in every state of the union. Anything possible, suppression, phone jamming, stickers on ballots, vote switching, "lost" registrations, etc. They stole 2000, this is fact - and with four years to work on the next one with the help of HAVA, why wouldn't they steal it too? To me, it's obvious.

How about instead of bickering about exit polls we work for reform, perhaps parallel elections in 2006.

It's easy, efficient, and accurate. Small precincts with hand counted paper ballots, results made public immediately (locally), then reported, random audits. Sound Ok?

[OnTheOtherHand]

63. umm, I agree! I disagree! auuugggghhhhh!

No one would be happier than I if NEDA stopped churning out bad exit poll papers. And the discussion has to go beyond Ohio.

Yes, I read Mark Crispin Miller's book and do not believe that the election was stolen. I do agree with a lot of what he has to say, and what you say, about e.g. vote suppression and "lost" registrations, although I can't tell whether any of that was decisive. I could rattle off the reasons why the book doesn't convince me that the election was stolen, but in a way I don't think it matters much, because infringements on voting rights are intolerable whether they were decisive or not. So, there's another thing we could choose not to bicker about.

A confession, however: sometimes we need to "bicker," because facts and beliefs guide actions. I do think it matters whether there was massive electronic vote-switching in 2004, and I don't think it happened. That doesn't mean I won't work with people who disagree with me about that, and it doesn't mean that I am telling anyone to trust e-voting. But I won't pretend to think that American democracy is dead until we get all the machines out of elections, because I don't believe that.

People can work on registering voters now, and on following up to make sure that the registrations get processed, and encouraging folks to follow through and stay engaged. People can work now on making sure that voters aren't getting "purged" from the registration rolls for no good reason. People can work now on influencing voting technology choices. (All these things are actually happening.) I personally do not love parallel elections, although I am very fond of some of the people who work on parallel elections -- hey, whatever. As Pete Seeger says, "We don't always have to agree, you know -- if I'm sailing up, you can be sailing down." It's all the same river.

"Small precincts with hand counted paper ballots" is at best a long-term project, but I don't object to long-term projects as long as they don't strangle the short-term ones. Me, I'm OK with op-scan with random audits, if the audits are good. I would even be OK with DREs if all the computer scientists told me they were copacetic, but that sure hasn't happened yet.

[organik]

69. Don't believe the election was stolen?

Think about Gore's popular vote win, then about Bush's 2000 theft, 4 years of a terrible presidency in which an illegal war was started and a terrorist attack came on his watch. Think about prominent repubs. breaking the mind control to say NOT to vote for Bush, and the newspapers that endorsed him in 2000 that wouldn't in 2004.

I think it's nearly impossible that Bush won. I'd like somebody to prove it to me. If it wasn't stolen, they were certainly trying their best to steal it. What evidence (besides the count) is there that Bush actually won? I personally know several Repubs that voted for Kerry in '04, and NO dems that voted for bush (although I am in California, to be fair).

I DO believe American democracy is dead unless we get rid of the machines, because there's a definite pattern of odd results since they were introduced.....starting with the 2002 midterms, in '04, '05, probably '06 and beyond.

[OnTheOtherHand]

71. forgive me if I don't try to prove anything to you

Trying to "prove" Bush won in 2004 seems like a waste of time to me. I'm happy to explain why I think he did. But my training is actually to try to refute my own opinions, more than to try to prove them.

I might be able at least to explain to you why a huge majority (as far as I can tell) of political scientists think it is very possible that Bush won, and why we aren't convinced by Miller.

Larry Bartels estimated some years back that incumbent presidents get something like a five-point advantage attributable basically to the fact that many people aren't paying much attention to politics. (There are other ways of thinking about the incumbency advantage -- and of course no one claims that it is always and everywhere exactly five points.) So, in that sense, even though Candidate Bush ran slightly behind in 2000, President Bush has an advantage headed into 2004 just because he is the incumbent. Of course, that advantage could be more than outweighed by other factors.

Frankly, there just isn't much reason to think that a lot of Americans were reading newspaper editorials, or op-eds by Dwight Eisenhower's son &c., to figure out who to vote for. Really, I just don't know anyone (other than Mark Crispin Miller?) who thought, "Migosh, Doug Bandow is attacking W. -- he's toast!"

Most Americans aren't especially convinced that the war was "illegal" -- they aren't thinking about it that way. And whether or not they thought the war was a good idea, many people weren't especially excited about handing it over to someone who thought it was a bad idea. Does that make me want to pound my head against a wall? yes, it does. I think it was straight out of Wag the Dog -- no last-minute crisis required, just "don't change horses in mainstream."

Could this have happened? Certainly. TIA insists that Kerry was running ahead in the polls properly understood, but the people who actually conduct the polls never agreed. I've linked before to a roundup of various predictive models, most of which gave Bush the advantage (one favored Kerry by a fraction of a point). The Pew poll immediately after the election also gave Bush the win -- and I have a hard time interpreting that as a "honeymoon effect"! It's nice to think that millions of young cell-phone users stormed out to vote for Kerry, then disappeared again into the mists, but I would not stake my reputation on it.

I dunno, what evidence is there (other than the exit polls) that Kerry won? Do not rattle off evidence of fraud and possible fraud, I have probably heard it all already -- but what convinces you that all this actually flipped an election that Kerry otherwise would have won? The fact that there was a terrorist attack? (But Bush's approval ratings were sky-high after that attack.) The war? (Come to the mid-Hudson and count the "Support Our Troops" car magnets -- and Woodstock isn't exactly as red as Wyoming.) The newspaper editorials? (Really?) The fact that you don't know (or don't think you know) any Democrats who voted for Bush? I mean, there is a huge selection bias here. I dunno about you, but I know that anyone -- Democratic or Republican -- who knows me wasn't likely to go out of his or her way to talk about voting for Bush. Political partisans tend mostly to talk with people who mostly agree with them. And political partisans were exceptionally polarized about Bush. It's no wonder that lots of people on both sides thought they were sure to win.

(EDIT: I meant to finish as I had started. I don't think we have to agree about this. And we don't.)

[organik]

72. Hmm.

Good points all, but I really do feel from all that I've seen, that Kerry was likely the winner. I feel it in my heart.

I just remember hearing about Bush's "loyalty oath" that people had to sign at his campaign stops. All these screened appearances because the opposition was so outspoken. Kerry had droves of hardcore supporters. Bush's hardcore supporters - a handfull of religious fanatics? There really aren't that many fundamentalist christians. Where did Bush's new voters (that didn't vote for him in 2000) come from? Haven't seen that one explained very well.

I just think it's absurd to think that quite possibly the worst president in our history, with a mediocre approval rating, was re-elected.

I would have voted for a turd on a stick before W.

It's good to disagree, but I think it's hard to motivate people to change the system if they don't think it's broken. Even if you don't believe it was stolen, you certainly believe the election system is broken, no? And if it wasn't stolen, at least admit they were trying - there is ample evidence of that. Nathan Sproul for instance...made tons of cash from the Rupubs registering Repubs and shredding Dem registrations....

[OnTheOtherHand]

73. OK

I think I understand your point of view, because I felt the same way. What was it that Kerry was quoted as saying? "I can't believe I'm losing to this idiot"? Well, "idiot" was imprecise. As far as I'm concerned, the Bush administration has declared a multiple-front war on reality. That doesn't just horrify me as a partisan, it offends me in just about every way I could be offended -- and many principled conservatives feel the same way.

I just can't pretend to trust my heart in this matter. But that doesn't mean that I will pound the table and scream that you are wrong. How do I know?

A thought on hardcore supporters. One way of thinking about this issue is to look at "thermometer" ratings. Let me quote the National Election Study question wording, so there is no needless ambiguity:

I'll read the name of a person and I'd like you to rate that person using something we call the feeling thermometer. Ratings between 50 degrees and 100 degrees mean that you feel favorable and warm toward the person. Ratings between 0 degrees and 50 degrees mean that you don't feel favorable toward the person and that you don't care too much for that person. You would rate the person at the 50 degree mark if you don't feel particularly warm or cold toward the person.

OK. It's weird, but years of experience indicate that it sort of works. Onward....

Bush got about 12% 0 ratings, and about 17% 100 ratings. Kerry got about 7% 0 ratings, and also about 7% 100 ratings. Ergo: Bush was a polarizing figure, and a lot of the hardcore "Kerry supporters" were really Bush opponents. However, the election wasn't necessarily decided by the "hardcore" who rated Bush 100, whether they were Christian fundamentalists or whatever they were (I haven't looked). It might have been, if those folks turned out in much larger numbers than would otherwise have been expected -- I haven't looked at that, either, and that sort of issue is very hard to pin down. But probably the election was decided by the muddled middle. Bush's average rating was a 57; Kerry's was a 55. Now, most individual respondents leaned distinctly one way or the other, but even among voters, about 20% scored it within 20 points, and about 10% scored it as a 10-point difference or less. About 5% of voters scored it a tie. If you think of the election as a battle of thermometer ratings -- which is kind of stupid, but probably less wrong than thinking of it as a contest of ideas! -- then Bush won not so much by mobilizing hardcore supporters, but by getting a few more people to lean his way. How hard they leaned may not have mattered.

A thought on the system being broken. It's multiple systems, broken and gamed in multiple ways. I am less worried than many around here about e-voting in 2004. But anyone who thinks that trashed registration cards don't matter unless Kerry actually won, well.... Anyone who thinks it's fine for thousands of Ohioans to get stuck standing in the rain for hours, and thousands more to turn away, because Blackwell wouldn't grant an administrative waiver that would have allowed them to cast paper ballots -- just as long as it maybe didn't actually determine the outcome of the election -- well.... For that matter, anyone who trusts any part of the election system -- hardware, software, or wetware -- to Just Work, well....

I think there is plenty of stuff that reasonable and decent people can agree is intolerable.

(Edit to remove confusing lapse into metaphor.)

[Febble]

74. Just chipping in here:

I agree with you about the exit polls in a way, because they have tended to polarise the whole story into an argument that goes:

"The election was stolen! Kerry won! By 7 million votes!
"Well, probably not, maybe Ohio...."
"Naysayer!"

(or worse.)

To me, although I desperately hoped for a while that there had been massive theft, which would be revealed in time to inaugurate President Kerry, the important thing was not whether he really "won" or not (actually if "winning" means "would have won if the other side hadn't stacked the deck, then I think he did - telling lies about 9/11, Saddam, and WMD is stacking the deck IMO) but the fact that so many people, disproportionately black/Hispanic, disproportionately poor, and disproportionately Kerry voters, were systematically disenfranchised, as they always are, whether it would have swung the election or not. Heck some of those voters standing in long lines in Ohio were standing in line to vote for Bush. They were disenfranchised too, just not as many of them.

So the emphasis on wholescale theft of white votes (which I don't think is supported by the evidence) really bugs me, because the much of real reform needs to go on at a level of "theft" that wouldn't have registered on the exit polls at all - those people who didn't even get as far as the precinct.

And the other reason it bugs me is that it makes people think (and I read it often on DU) that it isn't worth voting unless you get all HCPB or whatever. Yes it is. It is desperately important. And it's desperately important to get as many people properly registered in good time as possible.

AFAICT, the only good evidence for massive theft is the exit polls, and that isn't good evidence IMO. And I fear it is stopping us looking for the real evidence of a corrupted democracy, which is all over the place - Ohio, New Mexico, Florida, and elsewhere. I've done a bit of looking myself, and it wasn't hard to find - and I live in the UK.

Yes, let's end this stupid exit poll debate. It's over. There is no evidence in the exit polls of massive electronic fraud, and I haven't seen much anywhere else either. Let's find what there is for what opportunistic fraud there was, plug the loopholes for future electronic and other fraud, and work to end voter disenfranchisement, for which there is, shamefully, only too much evidence.

[organik]

75. They stole it - the media failed us.

Even if more people actually voted for Bush, which I don't think they did - here's why I STILL believe it was stolen:

Terror alerts to prop up the Presiden't approval rating.

People were much more likely to support Bush when they were shown images relating to terrorism (inducing fear).

This fear mongering, and the media supporting it - could very well be why reasonable people voted for Bush.

Had the truth about Bush's first term been widespread knowledge, he would not have won (if he did), not a chance. So very many things were withheld until AFTER the election, that should not have been.

Fraud by suppression of information, but fraud indeed. Bush IS the corporate president, and the corporate media made sure that he "won" and that any irregularities of his "win" were not reported.

This arguing of whether or not Bush actually won is not my biggest concern, and I do not label "naysayers" - I welcome debate, it's educational for both sides. I am concerned that our election system(s) needs to be fixed ASAP, and that promoting the idea that Bush's win was legitimate will discourage people from wanting to fix the system.

Besides, it is now common knowledge that Gore won in 2000, so everything since is irrelevant. You cannot re-elect a president that wasn't elected in the first place. Six years this un-elected clown has been in the white house (except for the year he was on vacation). It disgusts me to the core.

[Febble]

76. Well, I have a lot of sympathy with all that

it is because of Florida 2000 (which I watched in agony from Vancouver BC) that I got obsessed with Ohio 2004.

Good luck!

[RonB]

64. Repost and Response

Dear DU Readers,

I apologize for my many typos and other errors. This is a corrected post with some additional commentary.


Even if you're correct, and the matching is off and this leads to some changes in levels of significance for some precincts, not only would these additional descrepancies between the two exit polls have to be explained, but - and this is key:

NONE OF THIS WOULD MAKE ANY DIFFERENCE TO THE MAJOR FINDING OF THE PAPER OF OVERWHELMINGLY ONE-SIDED PRECENT LEVEL DISCREPANCIES IN OHIO WHICH REVEAL A STRIKING PATTERN THAT MANIFESTLY CANNOT BE THE RESULT OF "PERVASIVE BIAS".

Look at the graph on p. 13 of the paper. These are ALL DISCREPANCIES, taken direcly from ESI exit poll and blurred official result data. THEY INCLUDE NO MATCHING AND NO SIGNIFICANCE CALCULATIONS.

The pattern is striking. Very large Kerry discrepancies throughout the sample with much smaller Bush discrepancies at only one end of
the sample.

As you know pervasive Kerry bias would produce an "inverted U" pattern of Kerry discrepancies - larger in less partisan districts and smaller in more competitive districts. This data shows no such pattern.

Random non-sampling error would produce more or less equivalent Kerry and Bush discrepancies across the sample. Again - no such pattern.

If there is an "exit poll" error explanation for this, it is has to include a specific linking of exit poll error factors to this pattern. No such explanation has been forthcoming, and the data have not been released to any independent analyst for investigation.

In the absence of a serious explanation of this sort, and given this record of withholding data and inconsistent, unexplained, and misleading data releases/reports, and given the massive amount of on-the-ground evidence of pro-Bush election fraud in Ohio (that goes way beyond simple voter suppression), I think, and I< think> believe most reasonable and open minded people who are familiar with these facts would conclude, that the weight of the evidence clearly supports a "vote miscount" hypothesis.

Finally, the inconsistency between UMich/Roper and ESI exit polls is an incontrovertible fact and matter of public record. A 9% reduction in Kerry exit poll share is not "small" by any measure. Our data is based on ESI exit polls. If we added back 9% this discrepancy would be worse. These results are based on E/M and ESI reported numbers without taking into account exit poll inconsistencies.

[Febble]

67. Ron, please see my response

to your other post.

But there are a few points worth mopping up here too.

A "pervasive" Kerry bias would not necessarily produce an "inverted U" with respect to Kerry's vote share. A uniform Kerry bias would. A distribution of biases, including some pro Bush bias, would produce a distribution of precincts with a tendency (absent too much other variance) to have a positive linear correlation with Kerry's vote share. In a noisy plot, it won't tend to be significant, and in fact it isn't. But it means that we can further discount the tendency, as the predicted relationship for a given mean alpha is going to be a slight positive linear correlation.

Your statement that "random non-sampling error would produce more or less equivalent Kerry and Bush descrepancies across the sample" is simply not true, as you MUST know if you read my paper - maybe you didn't. But if you did, don't you remember the sigmoid?

So in fact there is no "absence" of a serious explanation, I've just given you a purely mathematical one.

As for your last statement - there is no "inconsistency" that has not been clearly explained, and can be googled by anyone interested. The Roper set will have a greater standard error because the samples are smaller. Because they are a subset. They are the subset used for the crosstabs.

Lizzie

[Febble]

68. Ah, just noticed the other correction

So we aren't talking about the exit-poll share plot anymore?

OK, then we'll draw a veil over page 14.

I will agree with you on one conclusion: The exit poll discrepancy in Ohio was significantly outside the Margin of Error.

It wasn't due to chance.

[Febble]

70. Actually, let me be more specific:

On the most conservative test I can think of, making no assumptions about distributions, I make the probability of the Ohio exit poll discrepancy being due to chance as about 1 in 16,500. That’s from a chi square, assuming, from the ESI chart, that there were 9 blue shifted precincts and 40 redshifted ones.

So, yes, the discrepancy was not due to chance, whatever the MoE on the individual precincts.

But that is precisely why we are having this debate, and why so many of us have spent so much time trying to figure out what caused the discrepancy. ESI concluded:

that the non-response rate theory is much more likely than the fraud accusation theory to account for most, if not all, of the observed discrepancy between the exit polls and the actual results. The more detailed information allowed us to see that voting patterns were consistent with past results and consistent with exit poll results across precincts.

NEDA takes issue with this conclusion, and the present study claims to offer


Virtually Irrefutable Evidence of Vote Miscount.

So how does it do so?

Well, it offers evidence that the discrepancy cannot have been due to chance. Fair enough. But what else?

The paper claims that

In an October 31st paper, NEDA mathematically proved that ESI’s and Mitofsky's analyses were incorrect because many counterexamples exist to its basic premise. In other words, NEDA proved mathematically that ESI's and Mitofsky's analysis of Ohio's and national exit poll data is of no analytical value and no conclusions about the presence or absence of vote fraud can be drawn from them.

This “proof” appears to me to be invalid. Indeed if valid, it would also invalidate the conclusions drawn in the smoking gun paper, but actually, it is not, so it doesn’t, although a number of other things do. So we’ll move on.

The authors then claim:

The ESI report had made no attempt to explain or mathematically analyze the actual 2004 exit poll discrepancies and the ESI report was missing key data. To date, Mitofsky and ESI have provided no explanation for the exit poll discrepancy that is supported by data and analysis.

Which is not true. The ESI report does analyze the 2004 exit poll discrepancies in considerable detail, and while what we have is a “brief” report, I understand a full report is currently undergoing peer-review, as such analyses should. I am not clear what “key data” is supposed to be missing from the ESI study, but they studied 49 precincts, and issued tables with data on those 49. The ESI study does not, indeed, provide an explanation, they merely conclude that the discrepancy is consistent with the poll-bias explanation and does not support the fraud explanation. They no-where say that it “rules it out”. On the other hand, the January E-M report does provide a fairly full explanation in the forms of crosstabs showing that WPE/WPD was greater where a number of factors likely to affect random sampling, notably low interviewing rate. Like NEDA, I had issues with that report, as no statistical details were given, plus, unlike NEDA, I do not consider WPE a good measure of precinct-level discrepancy. But it is simply not true that “no explanation” was given.

Now things get difficult as the smoking gun paper is not couched as a scientific report and it is difficult to find the hypothesis, the methods or the results in any clear order. But picking through the rubble, it appears that the authors attempted to match the ESI precincts with the Roper precincts, and found that they didn’t match; they nonetheless matched them “conservatively” – and used those matches to deduce the sample size on the assumption that the total sample was double that in the Roper sample. Well, firstly their matches are almost certainly wrong, because if the Roper samples were half the size, in many cases, sampling error in the subsampling process would generate greater discrepancies between the two sets than the matching algorithm assumes. So even if their “doubling” heuristic was justified, the wrong sample sizes would be assigned to the wrong precincts. But in fact the “doubling” heuristic is also wrong; because the subsampling process was designed to net about 50 responses from each precinct for the crosstab analyses. If a total sample was small, more than half might be subsampled – in fact the Roper set might comprise the entire sample. But the point is that the NEDA authors don’t know which. So any analysis by precinct is based on two faulty guesses. However, the authors do not include this error source in their probability calculations.

So what follows is based on faulty analysis. However, this doesn’t actually matter, because you can reach the same conclusion on the basis of the chi square I just did. We know that the discrepancy can’t have been chance.

NEDA then presents a plot of WPE against Kerry’s vote share. Bizarrely, they plot this as a bar graph rather than a scatterplot, which leads them into trouble later on as there are precincts that share the same value on the x axis, and when this happens, they even more bizarrely take the mean. They plot a regression line through their bar chart, but do not give the regression coefficient. They then ignore the bar chart for a bit.

Their next plot, page 10, appears to plot Kerry’s official vote values against Kerry’s official vote ranking (although the axes are poorly labelled so I may have got this wrong) as well as Kerry’s exit poll share against Kerry’s official vote ranking. They then fit a poly curve (polynomial unspecified, but it looks like a quadratic; they don’t give the equation but it looks as though only the linear term would be significant) – from which, I assume we are to deduce – dada! – that Kerry’s exit poll’s share tends to be higher than his vote share.

OK, my chi square did that.

Let’s move on.

At this point they say:

Ohio's exit poll discrepancy pattern is statistically implausible and has not been supportably explained in terms of any factors that cause exit poll error. Edison/Mitofsky and their NEP media clients have not publicly released information on the exact sample sizes, type of voting system, locations of precinct, or other exit poll factors to allow investigation or independent analysis.

Quite. So you can’t do the analysis that they claim to have done.

Next, page 11: WPD plotted against vote share. Again it’s a bar graph, so they have to average WPD where precincts share a vote share (which happens more often in this dataset than in actuality as a result of the blurring procedure). Given that they’ve done this it is just as well they don’t offer a correlation coefficient – they just point out that the regression line is negative, which it clearly appears to be. No probability value is given either, which again is just as well, because as I’ve pointed out in my other responses to Ron, the plot is completely meaningless. It just shows that Kerry’s exit poll share will tend to be greater where the exit polls were in his favour, and less where the exit polls were in Bush’s favour. Big people are significantly bigger than little people.

They then assert:

If vote miscounts cause discrepancies, then the trend line is negatively sloped when WPD is ordered by exit poll shares. Combined with a trend line with positive slope when WPD are ordered by official vote and the fact that the discrepancies shift across the 50/50 line when ordered by exit polls, Ohio's WPD is consistent with vote miscounts that altered the outcome of Ohio's presidential election.

So this is the smoking gun! They show an apparently positive regression line (without any statistical backup) between WPD and Kerry’s vote share, and a negative regression line (without any statistical backup) between WPD and Kerry’s exit poll share – and assert that this is “consistent with vote miscounts that altered the outcome of Ohio’s presidential election”.

Well, no.

The first is not statistically significant – and even if it was “approaching significance” (as my statistics tutor used to say “how do you know it was approaching? How do you know it’s not leaving?”) – the WPD itself tends to have a positive correlation with the Democratic candidate’s vote share if there is any bias at all, in either direction, even if it was "uniformly" distributed. So that isn’t a smoking gun.

And the second is meaningless, as I’ve just explained.

So no, the combination of the two regression lines is not suggestive of vote miscounts at all. It's what the math will produce in the presence of bias (which we know there was, whether in count or in poll) and sampling error (which, clearly, there was).

The fact that the discrepancies “shift across the 50:50 line” is, of course suggestive of vote miscounts because there is a significant discrepancy. But it is equally suggestive of poll bias

And in any case, it's exactly what I demonstrated with my chi square.

This response is already too long. The rest of the paper tiddles about with more regression lines through more bar charts (averaged precincts and all) again, with no statistical tests (they might have tested the significance of the difference between their two regression lines, but they don’t – it wouldn’t have been valid, for the reasons given above – but hey, it would have been a hypothesis, with a test, and a result.)

And they continue to accuse ESI of inadequate work.

As I said, elsewhere, the election reform movement deserves better than this. Good, hardworking, honest people are treading Ohio trying to find good evidence of the fraud we all suspect. Many others rely on “experts” to provide them with good evidence of what they feel they know in their gut – that the election was not fair, and may have been stolen outright. For sure many Kerry voters were either unable to cast their votes, or have their votes counted. For sure the DREs and tabulators are insecure, as the GAO report attested. That’s why we need good analyses from good analysts, to tell us what went wrong and what needs fixing. Because sure as hell something needs fixing.

What we don’t need is shoddy analysis from people purporting to be experts, who won’t even verify the limitations of the data they are working with, who produce an incomprehensible report with no decent hypothesis, methods section or results, yet bill it as:

The Gun is Smoking: 2004 Ohio Precinct-level Exit Poll Data Show Virtually Irrefutable Evidence of Vote Miscount.

No wonder it got 123 votes on GD. It's a great headline.

Unfortunately it means that 123 good people were sold a pup.

[Febble]

65. Ron, there is a serious problem here

Check your email, but I'll post it here too:

Your case seems to be that after subtracting from each precinct some value equivalent to the mean postulated "bias" for the nation (though why it should be uniform across the nation is something I've never understood, even though it has been repeatedly ascribed to me) you are left with some measure of bias that must be attributable to fraud because it is positively correlated with Kerry's vote share (WPE more redshifted where Bush's vote-share is greater) and negatively correlated with his exit poll share (WPE more redshifted where Kerry's exit poll share is greater).

Let's take this in two parts, and I'll take the second one first, as it's novel.

It should be pretty clear, from simple algebra, or even from considering the axes of the second plot, that any error in the poll - including sampling error - will induce a negative correlation between WPE and Kerry's vote share: because there is shared variance - error in the poll. Where, even simply by sampling error, you sample a greater proportion of Bush voters than the the proportion who vote, the WPE will be more negative, and Bush's exit poll share will be higher. Ditto for Kerry voters - if, purely by chance, you sample a greater proportion of Kerry voters than the proportion in the total, then your WPE will be more positive (or less negative) and Kerry's exit poll proportion will be higher.

So your second plot is entirely meaningless - it just says that where there were too many Bush voters in the exit poll, there tended to be more Bush voters in the exit poll than when there were too few Bush voters in the exit poll. See what I mean?

OK, first plot.

Well, as can be ascertained from the ESI data you use, the correlation is not significant. It is a trend only. Not only that, but, as we argued out long long months ago, WPE as a measure, tends to have a positive linear correlation with Kerry's vote share in the presence of any bias, even if the mean bias is zero. It was what that geeky paper was about. Yes, there is an U shaped relationship between any WPE for any given value of alpha, but the U is not symmetrical. Where alpha is greater than 1 (pro Kerry bias inthe poll, or pro Bush bias in the count), WPE will be more less negative at the high Kerry end of the plot than at the high Bush end. Where alpha is less than 1 (pro Bush bias in the poll or pro Kerry fraud in the count) WPE will be more positive at the high Kerry end than at the high Bush end. In both cases you will tend to get a positive correlation between WPE and Kerry's share of the vote. Only precincts with an alpha of 1 will have a flat correlation. There is not a single "U" but a distribution of assymmetrical Us, corresponding to given distribution of alpha. And that distribution looks like a tilted football. The tilt is what gives it the linear correlation.

Sorry if this is a bit geeky for anyone new to the series, but Ron knows what I mean, or if he doesn't he'd better re-read my paper.

In other words, both patterns are exactly what the math will produce in the presence of some kind of bias, whether in poll or count. They are NOT a fingerprint of one or the other. They are a fingerprint of WPE.

Well, one is the fingerprint of WPE, the other is the fingerprint of a tautology.

If that's the bottom line, I think we're in trouble.

[RonB]

77. No Need for Calculations - Just Look at the Data!

Febble,

In my first post I inadvertently referred to the graphs on p. 14 of our paper, when I meant p. 13 as stated in my "Repost".

THIS GRAPH OF ALL OHIO DISCREPANCIES (NOT JUST SIGNIFICANT ONES) INCLUDES NO CALCULATIONS. IT IS SIMPLY A REPRESENTATION OF ESI'S PRECINCT-LEVEL OHIO EXIT POLL DATA.

The graph on p. 13 shows:

a) Pervasive large pro-Kerry discrepancies across the sample.

b) NO "INVERTED U" PATTERN THAT WOULD BE INDICATIVE OF "PERVASIVE PRO-KERRY" EXIT POLL RESPONSE BIAS.

c) No pattern of random pro-Kerry and pro-Bush discrepancies that would indicate random (non-sampling) exit-poll error.

d) An unexplained pattern of mostly small (but a couple of large) pro-Bush discrepancies that are concentrated on the right side (high Kerry precinct) side of the sample.

e) NO PRO-BUSH DISCREPANCIES IN PRECINCTS WITH LESS THAN A 43% OR SO KERRY OFFICIAL VOTE.

No "exit poll error" explanation has been offered for these very striking patterns (that are consistent with WPD trends in the national data). Rather you and E/M have persisted in denying that they exist and ESI has ignored the question of explaining the discrepancy altogether (producing a "proof" that is utterly meaningless - see paper).

Unless an "exit poll explanation" can be produced for this - these patterns, plus the on-the-ground evidence of massive (almost exclusively) pro-Bush fraud, and the fact that both E/M and ESI reports attempting to demonstrate the opposite are methodologically and empirically in error and are based on withheld and inconsistent data, strongly suggest vote miscount that was more than adequate to change the vote in Ohio.

[Febble]

78. OK, Ron, then read my other posts.

But here are some questions:

Why bar charts not scatter plots? And why no correlation coefficient? And if not, why no explanation? And why take the mean of precincts with the same vote share?

The way the blurring was done, BTW, if you haven't read Scheuren's 1986 paper, involved grouping precincts with similar vote-shares and allocating the mean of the group to the selected precinct. So if two NEP precincts shared a group, they would have the same mean vote-share.

But that wouldn't prevent you doing a scatter plot and giving us the R squared.

I notice that you also, in your corrected post, deleted the reference to the plot of WPE against exit-poll share. I take it that you no longer want to draw attention to this plot, which is wise, because, as I argue in my other posts, it is quite meaningless.

I note that you still refer to this "inverted U". Ron, I will say this one more time: each value of alpha will have an assymmetric U relationship with vote share. But a distribution of alphas will not have a U shape. It will be a tilted blob. And if you draw a regression line through that blob you will get a positive slope. And even if, in the vanishingly unlikely event of uniform bias, you plot a linear regression line through the resulting U shape it will have a slope because the sodding thing is ASSYMMETRICAL.

So there is absolutely no reason to expect a U function in a data set with a distribution of alphas, which this dataset clearly has. But there is every reason to expect it to have a positive linear slope, even if the underlying alpha was not correlated with vote share. Which is one of the many reason why WPD is a very poor measure for correlational analysis, which is what you are doing here, even if you do not do it properly, and you do not quote the correlation coefficient.

Re: c) If you see "No pattern of random pro-Kerry and pro-Bush discrepancies that would indicate random (non-sampling) exit-poll error" you had better say what that pattern is, because it looks damn like it to me, with the obvious proviso that overall there was a massively significant preponderance of pro-Kerry bias (or pro-Bush fraud). But you certainly have not demonstrated that it is one and not the other.

In short - no-one has offered an explanation for these "striking patterns" because they are not striking patterns. They are patterns that indicate that either there was pervasive pro-Bush fraud or there was "pervasive" (but certainly not uniform) pro-Kerry polling bias. There is absolutely nothing that I can see in your plots or paper that even start to distinguish between fraud and polling bias. There are several assertions that the patterns are suggestive of fraud, but no demonstration, and indeed, in my other posts, I have demonstrated that they are not.

If you want to continue this debate, please address my concerns regarding the plot of WPD against exit-poll share, as it appears to be (or at least was earlier yesterday) a major plank in your "striking pattern" argument.

But even then "striking" pattern, is not statistically "striking" unless you can provide a clear testable hypothesis and test it, and demonstrate that it has a less than 1 in 20 chance of being a "striking" pattern that has occurred by chance, i.e. random sampling of the population of precinct in Ohio.

While I rail against the tyranny of the of the significant p value in published papers, there is a reason for it. You don't get papers published if you merely assert that the results are "striking", and demand an "explanation".

What is statistically striking is the overall discrepancy which is clearly statistically significant.

But if that's your smoking gun, you are about 15 months late.

(Edited for error).

[OnTheOtherHand]

79. your "valid scientific analysis" is "Just Look at the Data!"?

OK, maybe we're done here.

I agree with Febble that your point (b) is a mess -- you haven't defined "pervasive pro-Kerry," you haven't explained why it is important to anyone's argument other than your own, and you haven't formally tested whether the distribution diverges significantly from what you would regard as an "inverted U." As far as I can tell, you are just eyeballing. It may be good eyeballing, it may be bad eyeballing, but I can't imagine what dignifies it as "valid scientific analysis."

As for point (e), it simply isn't true. There is a pro-Bush discrepancy (albeit small) in precinct #26 at 28% Kerry vote, which you have obliterated by arbitrarily averaging it with another precinct that happens to have the same percentage Kerry vote. If one of my students used a bar graph to depict a small dataset of two interval/ratio variables, I would make him or her do the exercise over. Wouldn't you?

And you once again claim that ESI has "produce(d) a 'proof'" although, as far as I know, only NEDA has purported to produce a "proof" of anything. In what universe is it acceptable scientific practice to, well, invent quotations?

[Febble]

80. self-deleted

it's been said already.

[FogerRox]

81. The on the ground pro-BUSH story is clear- it happened

Huge tabulator errors showed up all over Ohio. I remember Randi Rhodes going on the air about these errors. 10s of thousands of votes, possible over a 100k. I remembver compiling a list of these errors and they quickly topped 200k. including provisional, absentee.

It maybe that the entire state was so screwed up that No Accurate count could occur.

[Febble]

82. Well, if you are right

then I'd credit you with "virtually irrefutable evidence of vote fraud".

Although I'd like to see chapter and verse.

I know it sounds nit-pickety, but even when a crime happens, there is good evidence for it and bad evidence. Ron's evidence is not good evidence - it's not evidence at all, except for the case that the exit poll discrepancy in Ohio was not due to chance, which my chi square upthread shows.

I think it is irresponsible to issue a paper claiming "virtually irrefutable evidence of vote fraud", when it is no such thing. If they'd issued a paper detailing huge tabulator errors, and computing the total votes switched, then I'd be more convinced. (BTW, the vote counts used by ESI and thus by Ron should be precinct vote counts, not tabulator vote counts. So if the fraud was at tabulator level, it can't account for these discrepancies.)


I'd also, while I'm here, like to make another point, not directed at you, but at an assertion made by both Ron and his co-author, which is that I originated the hypothesis that polling bias was "uniform". (I originally used the term "bias" to include both polling and counting bias, but I'll use it in the polling sense here, as that seems to be the sense in which Ron and Kathy are using it.)

I did not ever propose that any sort of bias was uniform, never have, and don't believe it*. I have on file email strings dating from the days before USCV and I parted company, in which I repeatedly, exasperatedly, try to explain that a "uniform" bias was not what I was alleging. I think, in the end, someone's (Ron's?) phrase "constant mean bias" cropped up, and I accepted that it was better** than "uniform". But "constant mean" is an odd phrase to describe two variables that don't covary, which was what I meant: that the underlying "bias" would not necessarily covary with vote-share. I don't think there is any reason to expect that it would, although at one point someone suggested that Bush voters might be "shyer" in Kerry company. But my expectation would be that the two variables would be orthogonal.

So the idea that demonstrating that polling bias can't be uniform means that that the the discrepancy can't be due to polling bias is ludicrous. Of course polling bias will be non-uniform. It is what the E-M report demonstrated - that polling bias covaried with lots of things, including distance of interviewer from precinct, interviewing rate, etc. What it didn't demonstrate, although USCV/NEDA tried to claim it did, was that it covaried with vote-share - at least there was no significant linear correlation. Demonstrating that the discrepancy systematically co-varies linearly with vote share might be an indication of fraud (although it could be something else, but I would certainly want to investigate), as it might indicate that greater discrepancy was associated with more votes (fraudulent) for Bush, but that was not demonstrated on the total dataset, and has not been demonstrated for the Ohio dataset, either by Ron and Kathy's New Scientific Method or by Eyeballing.

If Dopp and Baiman can't demonstrate that discrepancy covaries systematically with voteshare then there is no case. It could still be fraud, but we are no closer to demonstrating that it is than we were with my chi square upthread. And my chi square, simple though it is is, is a lot more "sophisticated" than eyeballing a bar chart (invalidly drawn) and saying "hmmm - doesn't looking like uniform bias to me - must be a Smoking Gun for fraud". As for demonstrating that it covaries with exit poll share - well, hmmmm. Better delete that part, guys. Bit embarassing.

Give me your huge tabulator errors any day, Foger. In fact - give me your huge tabulator errors - where, when, how much?


* and if there is any doubt about this, in the land of the undead, then my paper gives details of what I was actually proposing which was a distribution of alpha. A distribution of a value is not the same as a "uniform" value or even a "constant" value. Duh.

**If two variables don't systematically covary, n a statistical test, the mean value of the discrepancy at any given value of vote share is "constant" - i.e. not significantly different from the mean at any other value.

[Febble]

83. OK, had another look at this

You draw attention to page 13 (though it's a bit odd that your original post particularly asked us to look at the exit poll share plot on 14, and there isn't one on page 13) which has a plot of WPD against Kerry's vote share.


And apparently, you want us to gasp at is the observation (which, as OTOH points out, turns out to be wrong) that there are:

"NO PRO-BUSH DISCREPANCIES IN PRECINCTS WITH LESS THAN A 43% OR SO KERRY OFFICIAL VOTE."

And yet your co-author's paper on the right and wrong way to do exit poll analysis (mine and Lindeman's being, apparently, the wrong way) appears to model, on page 9, a fraud mechanism which appears induce GREATER pro-Bush discrepancies in the higher Kerry vote-share precincts, not the lower ones. Bit odd, don't you think?

Looks like the "fraud" finger print is a negative correlation between WPD and Kerry's vote share, not a positive one, which is what you seem to be inviting us to admire on page 13. I have to say, it had me confused. I did rather assume that Kathy's paper would argue for greater discrepancy at the Bush end of the vote-share plot, seeing as the running battle between me and you guys all those months ago was about greater WPD at the High Bush end. And there is a trendlet in that direction, I admit.

But ah - I see - page 13 of the Dopp paper: yes, it looks a bit like Ohio (as my esteemed colleague and fellow tag-teamster has pointed out). And yes, it does look as though there is a slight positive correlation between voteshare and WPD (we will again draw a veil over the exit poll share plot) in the fraud vote scenario. But get this:

it assumes that the exit polls were biased in favour of Bush.

OK, I'll turn off the schadenfreude right now. Ron, this thing is a mess. You need to withdraw it.

And the reason you need to withdraw it is that it brings perfectly decent arguments for the case that 2004 was corrupt into disrepute. This forum deserves better.

[OnTheOtherHand]

84. I think it's just cherry-picked

I don't think the positive slope especially depends on the direction of exit poll bias at all. I started to explain what I do think it depends on, but maybe we should just see whether they can figure that out on their own. It is their New Scientific Method, after all.

Hey, Kathy, should I offer not to post my critique if you offer a public apology and contribute $200 to a non-profit organization of Febble's choice? Just a thought.

[Bill Bored]

85. Well, there's one good thing about this pissing contest:

at least the folks over on PI are FINALLY talking about AUDITING, instead of just exit polls!

I'm going back to that auditing thread now.

Good day!:party: