Dr. Freeman talks back to "Mystery Pollster" Blumenthal

[Helga Scow Stern]

I got this in an e-mail from Dr. Freeman, in response to my request that he answer Blumenthal's critique of his paper on exit polls, which is here:
http://www.mysterypollster.com/main/2004/11/the_freeman_pap.html

Here is Dr. Freeman's answer:

Hello, Mark:

I’d like to thank you for taking the time for offering this detailed critique of my paper, and more generally for your knowledgeable commentary on polling processes. Since writing the draft you read, I’ve learned a great deal about polling – in large part from reading through your site. I’ll have a revision of my paper out in a few days, which will be much stronger for having read your commentary.

Regarding your post, I’m going to respond to several points and then give a general response to what I see as the big question:


Data. I’m happy to make my CNN data available. I have 49 states & DC (only Virginia missing if anyone has that), although for a few I don’t have sample size. Just tell me where you’d like me to send it to or post. (My own personal and University websites have been going down from too much traffic.)


High degree of certainty (Your point 1). I agree that I overstated the case, should never have cited Hartmann, and did not understand the logistical challenges you explain. NEVERTHELESS, logic and evidence still indicate that exit polls should be a good basis for prediction, and although I can understand why the logistical challenges would increase the margin of error, it’s not at all clear why they should skew the results.


250-million-to-1 (Your point 2). I see that I did put too much faith in stratification counterbalancing the effects of clustering, and will redo the calculations with the 30% increase. That’s a very good citation. NEVERTHELESS, as you point out, it doesn’t change the finding that **random error can be ruled out as an explanation.** This is really the main point of the first draft, because once chance is ruled out, some other explanation needs to be found.


Official “explanations. (Your point 3). My key point about explanations is that all we have, -- at best -- are hypotheses. Perhaps Bush-voter refusal is a better hypothesis than I gave credit for, but it still is only a hypothesis. (Too many women would be irrelevant to the CNN data. Male and female preferences are reported separately and thus automatically weighted appropriately.) On the other hand, there are also creditable hypotheses, some with substantial evidence, which could have effected the tally.


I object most to belittling dismissals by the media of these second set of hypotheses and allegations (e.g, Manuel Roig-Franzia and Dan Keating, “Latest Conspiracy Theory -- Kerry Won -- Hits the Ether” Washington Post, November 11, 2004 and Tom Zeller, Jr. "Vote Fraud Theories, Spread by Blogs, Are Quickly Buried" New York Times November 12, 2004 (Front page) ), along with unquestioning acceptance as “explanations” the hypotheses and allegations about poll error.



In summary, I think that perhaps I biased my paper somewhat unfairly towards suggesting count errors as explanations, but that was probably in response to what I still see as an extreme bias at the press in dismissing them.



When you say that suggesting the possibility of count errors is delusional, perhaps you have done the same? (It seems as though you spend a lot of time on the tin foil hat circuit.)

Thinking coolly and scientifically: Is it delusional to question the Bush-voter-refusal hypothesis as conclusive without independent evidence? On the other hand, considering the scores of allegations, the history (especially in Florida), the lack of safeguards with electronic voting, the conflict-of-interest in election oversight, etc…, etc… (and now the Berkeley study) is it delusional to consider that, just possibly, even part of the discrepancy might be due to the possibility of miscount?

Dr. Freeman's paper is here:
http://www.legjoints.com/AmericanElectionFraud/the-unexplained-exit.pdf

[goldengreek]

1. Thank you for all this.

Both the counter-argument and Freeman's response are excellent. Most importantly, Freeman has shown himself to be a careful scientist. He asked for input in his paper and he certainly got it.

So far, although his case has been weakened, it's still solid. The odds could drop all the way down to a "mere" million to one and we still have a case.

Maybe he could do a quick review of exit polls v. results in other races in this election?

[Helga Scow Stern]

2. Would you write him?

[email protected]
his e-mail address is posted at the top of his paper

[goldengreek]

3. Looks like the question was answered.

I'd just scanned through the counter-argument when I posted, so I see that analysis already been done by others.

But then again, if, for example the election in 2000 were perverted in states other than Florida, that analysis isn't any good, either.

We'll have to wait for Freeman's next revision to see how the dust settles.

It's just too late for me right now and there are so many things for me to pick though. I'll work more on this tomorrow. Waaay past my bedtime. Looks like it's still up in the air, though.

[Helga Scow Stern]

5. Let us know what you think when it comes out.

Night-night.

[goldengreek]

9. Okay, I just emailed him.

I wrote:

"Your paper is a hot topic over at www.democraticunderground.com and if you want to look at a discussion of it you can see it here:

http://www.democraticunderground.com/discuss/duboard.php?az=show_topic&forum=203&topic_id=62361#62484

I post as "goldengreek," and although I haven't thoroughly gone over the "mystery pollster" analysis of your paper, I have reviewed it somewhat.

I'm not any kind of expert, though, and neither are most of us. Just some thoughts getting kicked around here and there. I tend to think the election was stolen, but I'm ready to just shrug my shoulders and move on to France if I'm wrong. Thanks a lot for you answer to Ojai Person, and I can't wait for your revision!!"



Now I'm really, REALLY going to bed!

[truehawk]

4. kick

[truehawk]

6. True 2000 is not a good benchmark.

[goldengreek]

7. Okay, one last thing for now.

From http://www.mysterypollster.com/main/2004/11/the_freeman_pap.html


"An inspection of within-precinct error in the exit poll for senate and governor races in 1990, 1994 and 1998 shows an understatement of the Democratic candidate for 20 percent of the 180 poll in that time period and an overstatement 38 percent of the time...the most likely source of this error is differential non-response rates for Democrats and Republicans (Mitofsky, 2003, p. 51)."

I'd be very interesting in looking at those areas where the Democratic votes are overstated by polls. Could we be looking at long-term vote manipulation here against the Democrats? We know from Pat Robertson, for example, that the Bushes were stealing elections at least as early as 1988, when they stole South Carolina(?) from Pat in the primaries. And as a Democrat I get very uncomfortable voting down here in Dallas. We use electronic ballots without a paper trail. Furthermore, voter fraud in Volusia county and major violations of the Civil Rights Act of 1965 in both Ohio and New Mexico have been documented in this election.

We also need to keep track of exit polls v. tracking polls going into elections. The exit polls for Kerry segued very nicely into the polls leading into the election.

[goldengreek]

8. Dammit! I am vexed!

I work at Texas Instruments, and to keep track of the tools we build chips on we use the WECO rules. They can be found here.

http://www.sixsigmaspc.com/spc/x-bar_and_range_charts.html

Freeman's chart, reproduced on the mystery pollster site, would definitely raise a few eyebrows with us if we were looking at the results of the qualification tests run on tools.

"You ARE OUT OF CONTROL IF . . .

A single point falls outside the 3 Sigma limit, i.e. beyond Zone A

Two out of three successive points fall in Zone A or beyond, the odd point may be anywhere

Four out of Five successive points fall in Zone B or beyond, the odd point may be anywhere

Eight successive points fall in Zone C or beyond"

There are also rules for keeping track of things like trends and shifts. All of these are rules of thumb that are well documented and widely understood.


Now I'm going to bed.

[althecat]

10. Go Ojai... damn cool...

[pauldp]

11. Thanks Ojai

Freeman sure seems like a level headed scientist. Looking forward to the next draft.