
Printerfriendly format Email this thread to a friend Bookmark this thread 
This topic is archived. 
Home » Discuss » Topic Forums » Election Reform 
TruthIsAll (1000+ posts) Send PM  Profile  Ignore  Sun Dec1904 04:49 PM Original message 
EXIT POLLS: THE LATEST MYSTERY POLLSTER BLOG  AND MY COMMENTS. 
Advertisements [?]
Edited on Sun Dec1904 05:32 PM by TruthIsAll
My comments to his points are in bold. The Exit Poll Numbers are what they are. Very compelling circumstantial evidence of fraud. No amount of spinning can make them go away.
The crux of the spin is that we should use a 99.5% condidence interval, and not the standard 95% level. Of course, using 99.5% GUARANTEES THAT EVERY POLL RESULT WILL FALL WITHIN THE MARGIN OF ERROR. THIS IS TOO, TOO OBVIOUS A CLASSIC DIVERSION TO OBSCURE THESE FACTS: THAT THE BUSH VOTE TALLIES EXCEEDED THE MOE IN AT LEAST SIXTEEN STATES, IF ONE USES THE CALCULATED EXIT POLL MOE IN EACH STATE; OR THAT THEY EXCEEDED THE MOE IN TWENTYTHREE STATES, IF ONE USES THE HISTORICALLY PROVEN 2.0% MOE FOR EXIT POLLS. http://mysterypollster.typepad.com / EXITS: WERE THEY REALLY "WRONG?" Last week's posting of more detailed information on the sampling error of exit polls by the National Election Pool (NEP) allows for a quick review of the now well established conventional wisdom that the "exit polls were wrong." Conventional wisdom. Of whom? Let's first set aside the midday numbers that were widely leaked on the Internet but never intended as the basis for projections (numbers that even the exit pollsters recognized as flawed in some states  see this earlier post for explanations on the differences among the many estimates provided by the exit pollsters on Election Day). Let us also stipulate that at least one state estimate of Hispanic voters (Texas) was obviously wrong, given the correction issued by NEP. The conclusion that the exit polls were wrong is essentially about two possible failings: • That the endofday numbers seemed to predict a Kerry victory. • That the endofday numbers showed Kerry doing consistently better than he actually did. What is the reality given what we now know about the exit poll's sampling error? 1) Did the justbeforepollclosing exit polls show a consistent and statistically significant "error" in Kerry's favor? Yes, but that error has been exaggerated. Here is what we know: • An internal NEP review of 1,400 sample precincts showed Kerry's share of the vote overstated by an average of 1.9 percentage points. As far as I can tell, no one from NEP questions the statistical significance of that overstatement. TIA: MY QUESTION TO THE MYSTERY POLLSTER: IS 1.9% OF 1400 SAMPLE PRECINCTS AN INSIGNIFICANT MARGIN? • The beforepollclosing exit poll results posted by Steven Freeman (and included in the updated report by the Cal Tech / MIT Vote Project) show errors in Kerry's favor in 43 of the 51 surveys (the states plus DC). These overstate Kerry's vote by an average of 1.6 percentage points. If the surveys had been perfectly executed with perfectly random samples (an all but impossible assumption under real world conditions), the pattern of errors should have been the same as if we had flipped a coin 51 times: about half favoring Kerry, about half favoring Bush. The probability of 43 of 51 perfect surveys favoring Kerry by chance alone is less than 0.001%. Of course, this statistic only tells us that the surveys were imperfect. It says nothing about the cause or magnitude of the error. TIA: I HAVE CONSIDERED THE MAGNITUDE OF THE ERROR. IT IS BASED ON THE POLLING MARGIN OF ERROR, WHICH THE BUSH VOTE TALLIES EXCEEDED IN 16 STATES AND WHICH THE KERRY TALLIES EXCEEDED IN NONE. To be clear: Everyone  including the exit pollsters  agrees they "overstated" Kerry's vote. There is some argument about the precise degree of certainty of that overstatement, but if all agree that the difference is statistically significant the degree of certainty has little consequence. The size of the error matters, and the reasons for it matter, but whether our level of confidence about the error's existence is 99.9% or something greater does not. TIA: THE EXIT POLLS OVERSTATED KERRY’S VOTE? OR DID THE VOTES UNDERSTATE KERRY’S EXIT POLL NUMBERS? Having said that, the second draft of the paper, "The Unexplained Exit Poll Discrepancy," continues to needlessly exaggerate the significance of the error, especially within individual states. For example, Freeman claims that there were significant "discrepancies" in Pennsylvania, Ohio and Florida when each state is considered separately. He has a graphic (Figure 1.2) showing a significant error in Florida, assuming a 95% confidence level. However, these assertions are not supported by the margins of error reported by NEP. • I applied the appropriate "confidence intervals" reported by NEP (as distributed to its partner networks on or before Election Day) to each state. Contrary to Freeman's assertions, the separate "discrepancies" in Ohio, Florida and Pennsylvania fail to attain statistical significance even at a 95% confidence level. In fact, I see only four states (New Hampshire, New York, South Carolina and Vermont) with statistically significant errors favoring Kerry at a 95% confidence level. Of course, when using a 95% confidence level, 23 states should be out of range by chance alone. TIA: THAT IS FLAT OUT WRONG. ASSUMING THE EXIT POLL MARGIN OF ERROR FOR EACH STATE, KERRY’S VOTE LOSS WAS SIGNIFICANT WHEREVER HIS VOTE PERCENTAGE DEVIATED TO BUSH BEYOND THE MOE – AND THE MOE IS ALWAYS CALCULATED BASED ON THE 95% CONFIDENCE LEVEL, OR +/ 1.96 STANDARD DEVIATIONS FROM THE MEAN. • I also followed the advice of Nick Panagakis and estimated confidence intervals at a 99.5% level of confidence (the standard used by NEP to make projections) using the actual estimates of the design effect obtained from Warren Mitofsky by blogger Rick Brady. By my calculations, this more demanding test renders the apparent errors in NH, NY, SC and VT nonsignificant. TIA: USE OF THE 99.5% CONFIDENCE LEVEL IS A STRAWMAN ARGUMENT. WHY NOT USE A 99.99% CONFIDENCE LEVEL? THEN YOU CAN BE ASSURED THAT IT WILL NEVER BE EXCEEDED. Then there is the statistic heard round the world  Freeman's original "250 million to one" estimate of the odds against the discrepancy occurring simultaneously in Pennsylvania, Ohio and Florida. In his second draft, Freeman revised his estimate down to a mere 662,000 to one. Given that Freeman continues to understate the "design effect" used to calculate the sampling error in this year's exit polls, his revised estimate also remains too high. Some have asked that I calculate my own estimate of the joint probability of an error in Ohio, Florida and Pennsylvania. I am reluctant to do so for two reasons: First, the rounding error in Freeman's data alone renders this sort of hairsplitting moot. Second, and more important, it really doesn't matter. Everyone concedes there was a small (2%) but significant average error in Kerry's direction. For those concerned about problems with the count, what matters most is why that error occurred. TIA: DON’T JUST LOOK AT FL AND OH AND PA. LOOK AT ALL 51 STATES. AND CALCULATE THE PROBABILITY THAT 16 OUT OF 51 STATES WOULD MOVE BEYOND THE MOE IN FAVOR OF BUSH. I AND OTHERS AT DU HAVE CALCULATED THE ODDS THAT THIS COULD OCCUR DUE TO CHANCE: ONE OUT OF 13.5 TRILLION. IF YOU CONSIDER THAT FOR EXIT POLLS, THE HISTORICAL NORM MOE = +/ 2%, THEN 23 STATES DEVIATED BEYOND THE 2% MOE IN FAVOR OF BUSH. THE PROBABILITY OF THAT IS ZERO. 2) Considering sampling error, did the endofday numbers predict a Kerry victory? This one is easy: No. Not even close. TIA: OH YES THEY DID. It is true that the early numbers fooled a lot of people, including the talking heads on the cable networks, pollsters and advisors of both campaigns and possibly  depending on which accounts you believe  even John Kerry and George Bush. The reason, as the Post's Richard Morin put it, is that the 1.9% average error in Kerry's favor "was just enough to create an entirely wrong impression about the direction of the race in a number of key states and nationally." TIA: ABSOLUTELY UNTRUE. ALL KERRY NEEDED TO WIN WAS EITHER FLORIDA OR OHIO. HE WAS AHEAD IN BOTH. BUSH NEEDED TO WIN BOTH. True. But that error was not big enough to give Kerry statistically significant leads in enough states to indicate a Kerry victory. If we totally ignore sampling error, the exit polls showed Kerry ahead in only four states that he ultimately lost: Ohio, Iowa, Nevada and New Mexico. Obviously, Kerry would have won the election had he prevailed in all four. TIA: AGAIN, NOT TOTALLY ACCURATE. HE JUST NEEDED OHIO. Before considering whether any of those leads were statistically significant, two quick reminders: First, as Nick Panagakis points out, NEP required at least a 99% confidence level for projections on Election Night. Second, the margin between two candidates on a survey involves two estimates; and the margin of error applies separately to each candidate. Statisticians debate which rules of thumb to apply when determining the significance of the margin between two candidates, but the consensus in the case falls somewhere between the 1.7, as recommended by the American Statistical Association and 2.0, as recommended by those who consider it more appropriate in a race where the vote for 3rd party candidates is negligible (this is a great topic for another day's post  thanks to alert reader Bill Kaminsky for guiding MP through the competing arguments). TIA: YOU CONVENIENTLY IGNORE THE DEVIATIONS: OH 3.0% SIGNIFICANTLY BEYOND THE 2.21% MOE (1963 SAMPLE SIZE) IA 1.5% CLOSE TO THE 1.97% MOE (2502 SAMPLE SIZE) NV 2.0% CLOSE TO THE 2.13% MOE (2116 SAMPLE SIZE) NM 2.0% CLOSE TO THE 2.22% MOE (1951 SAMPLE SIZE) Fortunately, in this case, the statistical debate is irrelevant. None of Kerry's apparent exit poll leads in the four states were large enough to attain statistical significance, even if we assume a 95% confidence level and use a cautious multiplier (1.7) on the margin of error. As the preceding table shows, the exit polls had Kerry ahead by 4 percentage points in Ohio, by 3 in New Mexico, by 2 in Iowa and by one point in Nevada. The NEP 95% margin of error for these states multiplied by 1.7 is between 5 and 7 percentage points. At the more appropriate confidence level of 99.5%  the one NEP actually uses to recommend projections  these relatively small margins would fall far short of that needed to call a winner. TIA: AGAIN, YOUR USE OF THE 99.5% CONFIDENCE INTERVAL BASED ON THE NEP CRITERIA TO CALL ELECTIONS IS NOT THE ISSUE. THE ISSUE IS THE CONSISTENT RED SHIFT OF EXIT POLLING RESULTS IN FAVOR OF BUSH – 41 OUT OF 51 STATES, AND 16 ABOVE THE MOE. ******* ONE OUT OF 13.5 TRILLION******* IS THAT A SIGNIFICANT ENOUGH NUMBER? In a yearend review in this week's Roll Call (subscription required), Stuart Rothenberg concluded the following: The problem wasn't the exits  it was the folks who treated the earlyafternoon numbers as if they were a predictor of what would happen after everyone had voted. The exit poll was off by a couple of points, but that's well within the margin of error. If only someone had warned us about those early afternoon numbers on Election Day. Oh, wait... TIA: THOSE "EARLY AFTERNOON" NUMBERS (ACTUALLY 4PM) WERE GOOD ENOUGH TO STAY OUT THERE UNTIL MIDNIGHT  AND THEY COMPRISED THE BULK OF THE EXIT POLL SAMPLES. BUT MITOFSKY WON'T RELEASE THEM. WHY NOT? UPDATE: I mentioned William Kaminski's blog, but not his commentary on this issue nor his elegant graphic  it's worth the click just for the chart. My only remaining quibble with his graphic is that it displays a "safe estimate" of a 95% confidence of +/4%; leaving the impression that the numbers for DE, AL, AK and NE fall outside that interval. However, the appropriate 95% confidence interval provided by NEP for these states is +/5%. TIA: FIRST OF ALL THE CONFIDENCE INTERVAL IS 2.50%, NOT 5.0%. WE ARE ONLY CONCERNED WITH THE BUSH HALF OF THE NORMAL DISTRIBUTION TAIL. AND IT'S 2.50%, JUST AS IT IS FOR KERRY. THIS IS VERY SOUND, ESTABLISHED AND FUNDAMENTAL STATISTICS. IT'S BASED ON THE EXIT POLL SAMPLE SIZE AND THE RESULTING MOE. THE EXIT POLLS HAD ALREADY ACCOUNTED FOR DEMOGRAPHICS IN THEIR DESIGN AND PREPARATION. AFTER ALL, THAT'S WHAT POLLSTERS ARE PAID FOR, ISN'T IT? SO, LET ME SAY IT ONE MORE TIME: IN SIXTEEN STATES, THE BUSH VOTING TALLY EXCEEDED THE CALCULATED MOE BASED ON SAMPLE SIZE. NONE DID FOR KERRY. THE ODDS: ONE IN 13.5 TRILLION. AND IN TWENTYTHREE STATES, THE VOTING TALLY EXCEEDED THE HISTORIC, WORLDWIDE, ESTABLISHED 2.0% EXIT POLL MOE FOR BUSH. JUST TWO STATES FOR KERRY. THE ODDS: IMPOSSIBLE. 
Printer Friendly  Permalink   Top 
Home » Discuss » Topic Forums » Election Reform 
Powered by DCForum+ Version 1.1 Copyright 19972002 DCScripts.com
Software has been extensively modified by the DU administrators
Important Notices: By participating on this discussion board, visitors agree to abide by the rules outlined on our Rules page. Messages posted on the Democratic Underground Discussion Forums are the opinions of the individuals who post them, and do not necessarily represent the opinions of Democratic Underground, LLC.
Home  Discussion Forums  Journals  Store  Donate
About DU  Contact Us  Privacy Policy
Got a message for Democratic Underground? Click here to send us a message.
© 2001  2011 Democratic Underground, LLC