Reply #68: You don't really know how well the polls generalize...

[Sancho]

Most of us don't have access to all the poll data. We have parts of the polls without precinct ids, little information about the interviewers, and less about the details of the process. The sampling from exit polls may be representative, or they may not...and they may be perfectly representative for some states, races, or other units of analysis. They may also be useful to compare to other data outside the exit polls. Without precinct level data, it's had to say.

Unless E-M or others make the process more transparent, you can't say if the polls are representative and some of the elections are hacked, or if the polls are biased by sampling (reluctant voter) and process (gender/age of the interviewer) variables.

We don't know.

TIA's computations are based on normal distributions, and likely overstate the probabilities, (see Bonferronni), but the hypotheses are very interesting and may, in fact, be evidence that something is unexpected.

Personally, I think the defenders of the polls are too quick with excuses and pretty light on the data. On the other hand, now that election supervisors are posting more data at the precinct level and we can put it with annectdotal evidence for a given precinct, county or race...I suspect you'll see more EDA type reports. I'll looking at a few things now from the 2006 elections, and getting facinating trends....we'll see.

Meanwhile, TIA is interesting to read, and he certainly is creating some possible post hoc ideas that may be researchable by others.

PS. Yes, Virginia, I've studied statistics...