Reply #69: USCV's Analysis is not Based on WPE, "Aggregate" Analysis is not "Flawed"

[RonB]

Bruce,

You say:

"The key argument of the USCV Working Paper is that Edison/Mitofsky’s exit poll data cannot be explained without either (1) highly improbable patterns of exit poll participation between Kerry and Bush supporters that vary significantly depending on the partisanship of the precinct in a way that is impossible to explain, or (2) vote fraud. Since they rule out the first explanation, the authors of the Working Paper believe they have made the case that widespread vote fraud must have actually occurred."

I believe what we say is that varying patterns of partisan exit poll response require "further explanation" beyond a "pervasive bias" theory that is somehow inherent to being a Bush voter in that it is independent of precinct partisanship - our interpretation at the time of "rBr". The effort as I see it is to move the analysis beyond these vague and general "explanations" toward more serious substantive explanations that relate to the actual factors that influence response rate.

I don't believe we ever state that implausible invariant mean bias (K/B) is sufficient evidence of vote fraud. Just that a more substantive (so far not offered) explanation is required.

However, a closer look at the data they cite in their report reveals that Kerry and Bush supporter exit poll response rates actually did not vary significantly by precinct partisanship.

My analysis (for calculations see previous post - summary table copied below) demonstrates conclusively (I think) that K/B levels are indeed different at a 5% two-tail level of statistical significance for certain partisan categories of precincts. As you can see, it is particularly hard to see how the K levels calculated from both mean, and median (to eliminate out lier influence) WPE's for high Bush precincts could be obtained in any of the other categories of precincts.

These 95% confidence intervals are based on random sample exit poll selection - the same methodology use to analyze exit polls - without any cluster adjustment as there is no bias in estimating single precincts - and with mean approximates of R and b and k levels. If these later are not significantly correlated with each other (and the data suggest that they are not) these estimates should be quite accurate.

Means and Medians - and not individual data points, are after all, what statistical inference is usually about. Simulation in many ways is a "backward" reconstruction exercise, that cannot really add much (except for the additional hypotheses used to generate the simulations) to the original mean and median "information" about the data.


Table 2: Mean Based Calculations:

95% Confidence Interval for B 95% Confidence Interval for K
Covers all B? -1.96 x SD' B +1.96 x SD' Covers all K? -1.96 x SD' K +1.96 x SD'
Yes 20.9% 53.8% 86.6% No 49.3% 52.9% 56.6%
Yes 36.8% 49.6% 62.4% No 51.8% 57.3% 62.8%
No 43.2% 47.6% 52.0% No 52.0% 56.4% 60.8%
Yes 49.1% 52.6% 56.1% No 52.5% 60.6% 68.7%
Yes 47.1% 52.9% 58.7% Yes 31.9% 84.0% 136.1%


Table 4: Median Based Calculations:

95% Confidence Interval for B 95% Confidence Interval for K
Covers all B? -1.96 x SD' B +1.96 x SD' Covers all K? -1.96 x SD' K +1.96 x SD'
Yes 19.1% 51.9% 84.8% No 49.5% 53.1% 56.8%
Yes 37.1% 50.0% 62.8% No 51.7% 57.2% 62.7%
No 43.3% 47.7% 52.1% No 51.9% 56.3% 60.7%
Yes 49.1% 52.6% 56.1% No 52.5% 60.6% 68.7%
Yes 48.4% 54.2% 60.0% Yes 20.2% 72.2% 124.3%



You say:

"Systematic exit poll bias cannot be ruled out as an explanation of the 2004 Presidential exit poll discrepancy – nor can widespread vote count corruption. The case for fraud is still unproven, and I believe will never be able to be pr oven through exit poll analysis alone."

I agree.


You say:

"The fact that I chose not to endorse the USCV Working Paper should be a clear indication that I do not support its central thesis, and in fact believe that the simulation data they cite refutes the Working Paper’s conclusions.

I am not a statistician, but as a computer systems architect, I create mathematical models to simulate the performance of large-scale computer systems, and mathematical simulation of the cost and efficiency of business processes is a significant part of my consulting practice. My own election simulation results are cited on pp. 9 -10 and in Appendix G of the May 12th Working Paper; as the creator of the only USCV simulation which accurately reproduces aggregate Mean WPE, Median WPE and participation rate data from the E/M January report, I feel an obligation to ensure that my work is correctly interpreted."

Yes, definitely. The O'Dell simulation is an "output" simulation whose purpose is to simulate the E-M outcomes. The Dopp simulation is an "input" simulation whose purpose was to simulate the E-M hypothesis (or "hypothetical") inputs.


You say:

"I can show that several of the USCV election simulation programs are flawed, and that when the Liddle Bias Index is applied to the “USCV O’Dell simulation” data cited in the Working Paper, it produces results consistent with those recently reported by Warren Mitofsky for the E/M data as a whole."

I don't know what you mean by "flawed" but if you mean they don't reproduce the E-M outcomes thats the point! (see previous comment).

The Dopp simulator, in particular, as interpreted in Appendix I shows that the E-M outcomes are not obtainable under (what we then thought was) the E-M constant invariant K=.56, B=.5 hypothesis.


You say:

"I respect Ron's opinion, but his insistence on using aggregate WPE as a tool to interpret poll response bias (or vote fraud) is mistaken."

I think you're over stating your case here. There is nothing "inherently wrong" with "aggregate analysis", i.e. using means and medians. All these different types of analysis serve different purposes. Simulation adds the "information" (beyond the means and medians) that it assumes about the distributions to generate the simulations (see previous comments). This can be useful if there is some plausible "additional" information about these distributions and if there is a concern that "aggregation bias" resulting from parameters being correlated with each other (see previous post) will affect the analysis.

Using aggregate statistical analysis (as I do in the Tables above) is particularly useful to get significant probabilities, something that is particularly hard with simulations as there are so many possible options for distributions etc.


You say:

"His analysis of the Liddle Bias Index is also off-target. Liddle's Bias Index is an inherently superior metric to WPE, and analysis based on aggregate WPE are highly misleading."

I've stated all along that Liddle made an important contribution (that we acknowledged in the report) by focusing on looking at the "overall pattern" of K/B over partisan precincts. USCV had previously focused on calculating K and B in precinct quintiles and looking at the plausibility of these values and their ratios and differences across precinct quintiles, rather than at the overall pattern across precinct partisanship.

However, the basic point is that the calculation of and focus on K and B (the "confounded partisan response rates) was initiated by USCV. Liddle's alpha is simply LN(K/B). She derived it without K and B, apparently independently - its not a big deal to derive any of these formulas - the insight is in the need to do it and how they can be used. Liddle's contribution is in the later.


Unfortunately Liddle chose to frame her analysis as a new breakthrough in using "confounded variables" rather than simply a new way to use K and B to do the analysis. If she had been more clear about where her ideas came from, or at least who had done them before, she would have had to explain how she came to opposite conclusions from the same variables. I don't believe she did any of this deliberately but it has been the source of endless confusion that I think, most unfortunately, has gotten you confused as well.

NO BODY IS BASING ANY ANALYSIS ON WPE. ALL OF US ARE USING K AND B. WPE IS NECESSARY TO DERIVE K AND B BUT THE ANALYSIS IS BASED ON K AND B VALUES NOT ON WPE. WE ARE ALL ANALYZING THE SAME VARIABLES IN SLIGHTLY DIFFERENT FORMS (K, B, K/B, K-B. LN(K/B))

Liddle's other contribution was to point us in the direction of simulations. This is another method of analysis that could show the impact of "aggregation" effects. We used this in the report to show that these aggregation effects to not vitiate the conclusions of the aggregate analysis.


Ultimately this debate cannot be resolved with absolute certainty unless Mitofsky gives us the actual data (or does the calculations form the actual data himself). However, short of this I see no reason to make claims that simulations constitute absolute proof and "aggregate" statistical analysis is "off target" or fundamentally wrong.

The aggregate results show, with high probability, that K/B is not constant across precinct category. Your simulations show that, I guess, that this might not be the case. However, I don't think you (like Liddle) should jump to the conclusion that K/B is unvarying. It seems to me that our report (and the analysis above) makes a very plausible case that "more stuff needs to be explained" - and this is the point of it all!


Best,

Ron