Reply #6: 1% changes in polls are meaningless

[Hokie]

This is in what is called the "statistical noise" level.

Let me try to explain. I am not a statistician but I understand basic probability theory:

Let's say there are 100 MILLION voters out there. This is a low number but close enough. Imagine Obama voters are blue gum balls and imagine McCain voters are red gum balls. Now imagine that not all the gum balls are the same. Some are really, really blue or red and will never change colors. Some are pale red and might even turn blue depending on the latest news or whether a guy with a McCain bumper sticker cut them off yesterday. You are Mr. Gallup and you have to figure out the total number of blue and red balls in the gigantic jar (that is opaque so you can't look into it). Since it would take two lifetimes to count all the balls you employ this handy dandy thing called statistics and you pay the money to have someone pull about 1000 balls out of the jar and see what color they are (or make random phone calls). Mr. Bernoulli figured out that for totally random things like coin flips that the the larger the sample size the better chance that the count represents what is really in the jar. He even calculated that for about 1000 samples that you have about a 95% chance that the sample count is within +/- 3.5%. This is called the "Margin of Error". Most polls will tell you what this "margin of error" is and let you think that it could never, never be outside that MOE. Guess what. Even under the best of circumstances where all the balls are pure blue or red the real count is wrong by more than the MOE about 1 time in every 20. That means that, on average, more than once a month any one of these fancy polls is wrong by more than the MOE. It gets even better (or worse depending on how you look at it). Voters are not like balls that never change color. Mr. Gallop has to try to guess which voters are likely to vote because those are the ones that determine who really wins. He does this by asking if they they voted in the last few elections and from other questions. He may not know that even though you haven't voted for years but you are really stirred up about this one and are going to vote come hell or high water. To make things even worse almost everyone has answering machines and caller ID. He has to dial about 8 phone numbers at random to get one valid answer. Mr. Gallup has done this polling for a lot of years and is pretty good at guessing the number of balls. However, he can never get better than the limits than Mr. Bernoulli set for him a long time ago.

This is a graphical representation of margin of error from a great post by RiverStone here a while back: