Reply #70: Qualification:

[Febble]

I confess I had not looked at this "final poll" before, but if "the math" gives a 99.9% (or even a 99.8%) probability of a Kerry win, then something about the math is wrong. I was optimistically putting it the odds at about 50:50 myself, and hoping that even if the popular vote was pretty evenly split, the EV would benefit Kerry. Which is sort of what happened (in the official vote anyway), in that a few thousand more (official) votes for Kerry in Ohio would have given him the presidency, but not the (official) popular vote.

(And more generally, when I say I agree with TIA's math, I certainly trust him to perform the computations he does correctly - the issue is usually whether we can legitimately infer from them what he infers from them. But his probability calcs are frequently invalidated by the assumptions that underlie them, and a probability calc based on erroneous assumptions is not a correct calculation).

So what is wrong with this particular probability calc? Well for a start, as OTOH says, a good meta-analysis is a complex thing to do, and one thing it involves is using ALL data you can find (including unpublished data, if you can find that) and weighting it appropriately. "Appropriately" is the hard part of course, and it is difficult to avoid the charge of "cherry-picking". There are, however, systematic ways of weighting studies according to statistical power and cleanliness of methodology. But the fact remains that meta-analyses in general (and meta-analyses of these pre-election polls in particular) are acutely sensitive to the studies you pick and the assumptions you make. In the case of the pre-election polls, some analysts called it for Bush (but with nothing like 99.9% confidence; others sat on the fence; and a few brave souls like Sam Wang (I think) called the EV for Kerry (with a wing and a prayer).

Yet TIA called it for Kerry with 99.9% confidence? From a eighteen polls in which (unweighted) the mean difference between Kerry and Bush was not (on my calcs) significantly different from zero?

Look, I don't think that election was on the up and up, and I think the exit poll stuff was worth investigating. And apart from anything else, it also energised a lot of people (including me) into investigating stuff that brought the election result into serious doubt. But claiming that Kerry's probability of victory, in both the electoral college and popular vote, was 99.9% - well, as we say on our side of the pond, pull the other one, it's got bells on.

Were any bookies offering 100:1 odds on a Bush win?