2016 Postmortem
Related: About this forumOne in 20 polls, simply by random chance, will be outliers
Please keep that in mind when we're seeing so many polls reported from so many places.
Completely apart from sampling techniques, various likely voter models, real or imagined biases and agendas of pollsters, most polling is done based on a 95% confidence interval. That means that even when pollsters do everything else right, when they conduct good, honest polls and process the results with good math, one out of twenty poll results will fall outside of the stated margin of error.
To put that another way, when a pollster says their poll results have a margin of error of +/- 3%, that's never meant as a guarantee that actual voter sentiment is within 3% from a reported poll results, only that 19 out of 20 results will be within 3%.
Taking the confidence interval into account doesn't mean, however, that every time you see a poll you don't like you should decide that that particular poll was the 1 out of 20 outlier. Cherry picking the polls you want to believe is going to produce a more distorted view of the world than the expected level of polling error produces.
Understanding this simply means you have to be patient, look at the big picture, watch results from multiple pollsters and trends over time, and not be so quick to assume that every poll you don't like is either a sign of doom or a deliberate Republican-backed distortion.
demwing
(16,916 posts)To absorb outlier results.
Btw- Nate Silver has Obama back at
70% chance of winning. ; )
Shivering Jemmy
(900 posts)Even less sensitive to outliers.
bemildred
(90,061 posts)GeorgeGist
(25,321 posts)bemildred
(90,061 posts)Silent3
(15,211 posts)I'm talking about very basic probability and statistics.
http://en.wikipedia.org/wiki/Confidence_interval
"In applied practice, confidence intervals are typically stated at the 95% confidence level."
bemildred
(90,061 posts)It has no more predictive power than probabilities with dice or cards, it "estimates" certain statistics about the distribution of the population sampled, based on certain assumptions about the population sampled and the sampling methodology, but it means doodly squat about what will happen today or tomorrow.
Much of the babble on this subject is based on the unstated assumption of sequential correlation, that tomorrow will be much like today, which works great only when nothing much is going on.
So you always wind up predicting something close to the status quo, which is comforting but wrong when you really need to think about what "rough beast, its hour come round at last, Slouches towards Bethlehem to be born".
DemocratSinceBirth
(99,710 posts)It's up to us to change reality. When the reality changes the polls change. That's why we have our first African American president.
bemildred
(90,061 posts)There are a few polls I pay attention to, but none of them are commercial operations for obvious reasons.
You are quite right, we can do whatever we like within the range of our powers, and our powers are great for what sort of creatures we are, and there are billions of us, too.
But we have to learn to work together, or it will continue to be that same old Hobbesian dystopia we have now.
He is not President beause of some poll, he is President because WE made him President. He, Obama, at least, seems to understand that. (But then, not born to privilege, he would know that.)
DemocratSinceBirth
(99,710 posts)What do you think the results would be?
And now we are on the verge of electing a gay or should I say a openly gay senator.
My point is a reasonably well constructed poll, free from bias, should give you some idea of what folks think on a potpourri of topics. Now, what these people think will change over time, and the polls or surveys will reflect that.
bemildred
(90,061 posts)And it shows.
Silent3
(15,211 posts)How's that spin? "Confidence" does not mean "absolutely unshakable guarantee", especially in this common, long-standing statistical usage.
bemildred
(90,061 posts)The probability that a roll of your die will be a 6 is 1/6 th, REGARDLESS of any past series of rolls. That is what random MEANS, that there is no correlation from try to try, the past does not predict the future, AT ALL.
It is precisely that which justifies these meta-poll studies, that the polls are each separate tries, and hence (by their randomness) independent as sample data. (This is quite questionable too. Pollsters are poll junkies.)
The technical usage has a technical definition, which is correct. That does not mean the choice of the word "confidence" has the correct non-technical connotations, and as I said, in fact it's wrong, 95% probability is not 95% confidence about any try in particular, each try is ASSUMED independent, non-repeatable, and without replacement, "confidence" suggests "assurance", and that misleads, you have no such thing, you are still gambling. The entire system is built on the premise that each try is independent, that every time you ask the question, it is new.
A 95% confidence interval is a range in which the sample is expected to fall 95% of the time, the other 5% of the time it is expected to fall outside, and you are just as "confident" about the accuracy of the 5% as about the 95%, no more and no less.
At best, what polls provide is a crude heuristic that (done well) a sophisticated observer can interpret with some "confidence". Note that "confidence" is something someone has, an opinion, something one could sample too. I'm pretty "confident" at this point that Barak Obama will be the next President, barring "accidents". But I also know the present does not compel the future, it's just an educated guess based on a structured investigation.
I can go on. This is just scratching the surface, the easy stuff, we can get into Mr. Taleb's bootstrap issues with distributions, which are obvious to anyone with mathematical training and the will to look, for example. They annoyed me when I studied the subject 30 years ago, and they still do. There is nothing normal about the "normal distribution". The real world does not in fact exhibit zeroes or infinities or continuity, it is discrete, quantized, finite, and always something. Even "empty space" is far from empty. Velocity is relative (no absolute rest) and finite (less than the speed of light relative to any particular inertial frame). The uses of such notions in the physical world are all based in the laws of large numbers, probability and combinatorics, and all the physical constants that we know with great precision likewise.
Most of the more responsible pollsters seem to know that some of these issues exist, their rhetoric shows, but all the financial incentives run the other way, toward claims of certainty or some near approximation, or to show some desired "trend", a notion which you will see wanders off completely into the brush of cause and effect thinking.
There is a fundamental contradiction between assuming in one place that people form random populations you can sample randomly, by language no less, and in another, that people's behavior exhibits serial correlation (cause and effect, "confidence" about the future) over time, too. This I have seen mentioned, but I've never seen it really addressed at all.
People want confidence, assurance, security if you will, but that is wishing, we have no such thing and we need to pay attention, not be running around fat, dumb, happy, and full of ourselves because we think we know what's coming next. We don't.
I hope this at least clarifies what I base my views on.
Silent3
(15,211 posts)...and let you get on with whatever argument you're having with yourself or somebody else who isn't me.
bemildred
(90,061 posts)Shivering Jemmy
(900 posts)Stats 101 will teach you that.
davidpdx
(22,000 posts)I have access to The Psychology Research Handbook: A Guide for Graduate Students and Research Assistants. There was an interesting chapter about telephone polling.
bemildred
(90,061 posts)Like trying to run on a pool of buttered ball-bearings.
Objective subjectivity.