Reply #69: I think we can all agree that Febble is funnier than I am

[OnTheOtherHand]

but it is a bit disheartening to have to wonder whether you have gleaned anything else from all our posts taken together.

"I don't necessarily agree that what You assess as a bad argument Is a bad argument."

No, of course not, and I don't expect you to. Honoring the truth doesn't entail agreeing with me; I don't agree with myself half the time, which is probably one reason why you think I am long-winded. But I do think honoring the truth entails either engaging the substance of arguments or else reserving judgment.

Freeman's untrue arguments? Here is a short partial list. I could do it in detail, but I don't know whether any part of it bears further discussion right now.

* Freeman maintains that "logic... and experience" testify to exit polls' high accuracy, but they don't. No "logic" maintains that exit polls should be bias-free, and there are many examples of inaccurate results, including the 1992 U.S. exit polls. (One could argue that the 1992 election must have been riddled with fraud -- but then the argument from "experience" is circular.)

* Freeman cites Ukraine 2004 as evidence of exit poll accuracy, but it isn't; the two exit polls in the run-off differed from each other by more than the U.S. exit polls differed from the official returns. (I think he also implies that the exit poll results were used to overturn the result, which they weren't, but it is hard to keep track of who says what about that. In Montreal he said something off the cuff about 'huge headlines' about the exit polls in Ukraine; actually, so far I haven't found any U.S. news source that headlined the exit polls. CNN did refer to them in a caption at least once.)

* Freeman said in October that there was "no independent evidence" of non-response bias in the 2004 exit poll or other exit polls. There is considerable independent evidence of non-response bias in the 2004 exit poll. Also, there is very considerable evidence from other exit polls, including experimental evidence that is difficult to argue around. (This is a huge, huge problem: Freeman has encouraged a lot of people to buy into the idea that non-response bias is some weird excuse that Mitofsky made up, when actually it is something that survey researchers worry about all the time.)

* Freeman attaches great importance to the lower mean red shift in paper-ballot precincts -- but controlling for location (urban/rural), there is no statistically significant difference between these and other precincts.

* Freeman said that for the non-response bias account to hold, it was "practically a mathematical necessity" that reported completion rates should be higher in Democratic precincts -- but it isn't.

* Freeman cited higher red shift in states with Republican governors, but his finding hinges on states where the election machinery was controlled not by the governors, but by Democratic secretaries of state or (in New York) non-partisan election board.

* Freeman wrote, "There’s no PLD (red shift) at all in the Kerry strongholds, but the discrepancy is highest on the right side of the spectrum.... In fact, the stronger Bush’s support, the greater the disparity.... If fraud were afoot, it would make sense that the president’s men would steal votes in GOP strongholds, where they control the machinery of government...." Actually, there is red shift in "Kerry strongholds" (just eyeballing a graph, it looks like over 20 points in at least five precincts) ; there is no monotonic (one-directional) trend for red shift to increase with Bush support; there is no basis for the assumption that "the president's men... control the machinery of government" in all the "GOP strongholds"; and even if they did, there aren't enough red-shifted "GOP strongholds" to put the outcome in doubt.

* In Philly, Freeman said that for Mitofsky to claim "that the exit polls did not indicate a victory for John Kerry, is stunning. He might as well stand before us and say 'blue is red.'" But Kerry's apparent lead in the Ohio exit poll was within the margin of error, even assuming zero bias, so Ohio was too close to call, and there is nothing "stunning" or Orwellian about Mitofsky saying so.

* In Philly, Freeman said, "When a company cooks the books or when a scientist fudges figures, a contradiction is created.... When an irreconcilable contradiction exists, it is a sure sign that something is amiss.... NEP had to enter into a Wonderland of numbers." The "irreconcilable contradiction" is basically TIA's old argument about too many Bush 2000 voters. But if Freeman had only Googled the 2000 exit poll, he would have found that it had the same problem in the opposite direction: too many Clinton 1996 voters. If he had checked other exit polls, he would have found that they routinely overstate the prior winner's vote share. Given some of the epithets that have been hurled at Mitofsky (I could direct you to some here on the board), I find that combination of polemic and laxity very hard to stomach.

* Freeman has never responded to Mitofsky's observation that red shift doesn't correlate with improvement in Bush's performance from 2000 to 2004 ("swing") -- a very strange non-finding if red shift measures fraud. We have discussed this issue extensively on the board. See also my home page, passim, especially the first two working-paper links. (There is the further problem that red shift doesn't correlate with Bush improvement over the pre-election polls, either.) I don't know whether non-arguments count as untrue arguments, but in academic debate, it is definitely bad form to ignore objections.

I stopped at an even ten. Most of these, in my professional opinion, are big problems -- and I don't think my opinion is at all unusual.* Everyone makes mistakes, but it is uncommon in academic discourse to make so many mistakes, so enthusiastically, and all in the same direction. So, when Freeman presents his work as that of a "survey expert," it poses ethical dilemmas for the entire profession.

*EDIT TO ADD: Actually, I imagine that most of the relevant professionals haven't read the work, so it is probably unusual even to have formed an opinion on, e.g., 'what Freeman says about Republican governors.' What I should say is: I haven't encountered many survey researchers or analysts who would argue Freeman's side of most of these points. (A few are at least open to interpretation.) The people whom I find arguing Freeman's side tend -- like Freeman himself -- to be knowledgeable in other fields.