Reply #297: good questions

[OnTheOtherHand]

I haven't had my coffee yet, so we may have to try it twice. (And the conversation can ramify in lots of directions, anyway.)

Ron B. and I basically agree that it is reasonable to look at the observed distributions and consider whether they are statistically and substantively significantly different from distributions we might expect (some null hypothesis or hypotheses).

We agree that it is important to look at biased error, not just error that nets out to zero.

I think that the way he calculates K and B is fundamentally flawed (which is why it can generate apparent completion rates over 100% or, in fact, under 0%), and that infirms a lot of his argument in this thread. I would consider that a mathematical error, although I don't think there is anything wrong with his algebra.

I think that he underestimates how much variance -- much of it presumptively innocent, since it favors Kerry (maybe I'm sentimental, but I'm not willing to embrace widespread Kerry fraud) -- is in the observed data. This informs tests of statistical significance. Remember how t or z scores are calculated as a difference from the mean, divided by the standard deviation or standard error? If you systematically lowball the standard deviations/errors, you will exaggerate the t scores.

(How to fix that? I've basically adopted and adapted the Liddle approach, which is to simulate varying Kerry and Bush completion rates, draw "samples" from various precincts with varying levels of partisanship, see how the statistics from those samples vary across multiple runs, and compare that with the reported results. O'Dell took a similar approach, hand-tweaked some values to increase the variance, but is still below the E/M reported mean absolute WPEs. Dopp also took a similar approach, but since no one here has discussed it, I will leave it be.)

Ron focuses on four outliers in high-Bush precincts as suggestive of fraud; I see outliers in all sorts of places and in both directions. I'm not saying that those four outliers (or the others in both directions) couldn't be fraud, although I think we already have places to look for fraud. But I do say that substantively, you could assume fraud in those precincts, double it, extrapolate to the whole country, and still come nowhere near changing the election outcome. The fixation on "Bush strongholds" seems unhelpful.

One remaining interesting question is whether the data nonetheless hint at a wider pattern of vote embezzlement. I'm not sure that Ron has actually addressed that question. So far, I think the lack of correlation between partisanship and the Febble function is a real problem for scenarios of massive vote embezzlement. But I agree with Ron that at a minimum, we need more multivariate analysis -- and I would add that we need more sharp thinking about what these scenarios might look like "on the ground" and how they would show up in the data.

(Another question is whether the whole idea of widespread response bias makes any sense. As far as I can tell, most public opinion researchers think that differential response does explain the exit poll discrepancies. But even if one assumes that that is true, there are still important questions that need to be sorted out.)

I joined USCV because I thought (and think) the work is important for the same reasons as Bruce -- but that isn't quite getting at your question. Would it have been intellectually correct to read E/M's January report, look at the mean absolute WPEs, and say, "Beam me up, there's no usable data here"? or to have bailed out before ever signing on? I sort of wish I had, in retrospect, but actually I don't think that's correct. Since then, for instance, we have explored facially plausible models of massive vote embezzlement that could be supported even by very noisy data (but apparently aren't -- but there could be other models). I have no way of anticipating what we might discover next. Even if we don't find massive vote embezzlement in 2004, the work may help us detect it or prevent it in future elections, and that is exactly USCV's mission as I understand it.