Democratic Underground Latest Greatest Lobby Journals Search Options Help Login
Google

"Margin of Error" is a meaningless buzz phrase (as commonly used)

Printer-friendly format Printer-friendly format
Printer-friendly format Email this thread to a friend
Printer-friendly format Bookmark this thread
This topic is archived.
Home » Discuss » Archives » General Discussion: Presidential (Through Nov 2009) Donate to DU
 
Kurt_and_Hunter Donating Member (1000+ posts) Send PM | Profile | Ignore Thu Oct-30-08 11:30 AM
Original message
"Margin of Error" is a meaningless buzz phrase (as commonly used)
Edited on Thu Oct-30-08 12:04 PM by Kurt_and_Hunter
Every day talking heads summarily dismiss polls that are likely to be accurate in terms of who is ahead. (While treating polls that are only marginally more certain as irrefutable fact.)

A three point Margin of error means, "Given the sample size there is a 95% chance this number is within three points of where it would be if we had polled everyone."

That is all it means.

A poll result within the margin of error is not "statistically tied." Say you have Obama 48, McCain 46 (MOE=3%) The chance that Obama is ahead is 74% (Thanks to Viking12's chart) Today's out-lier Pennsylvania poll is Obama by 4% (MOE 4%). I have heard that described as "statistically tied" even though the poll is telling us that it is 84% that Obama is ahead in PA.

There is nothing magic about 95%. It's just the bench-mark that is used in the polling industry.

Nobody draws a profound right/wrong distinction between 96% and 94% in other contexts. (If there is a 94% chance your missing car-keys are in the sofa you look in the sofa, rather than dismissing that likelihood as meaningless because it is less than 95%.)
Printer Friendly | Permalink |  | Top
grantcart Donating Member (1000+ posts) Send PM | Profile | Ignore Thu Oct-30-08 11:32 AM
Response to Original message
1. you could post this everyday for a year and people won't get it
hell KO doesn't get it with his 'Keith Number' - although I notice he seems to have stopped using it.
Printer Friendly | Permalink |  | Top
 
OPERATIONMINDCRIME Donating Member (1000+ posts) Send PM | Profile | Ignore Thu Oct-30-08 11:34 AM
Response to Original message
2. Not Sure Why You're Claiming This. Margin Of Error Exists For A Reason.
If the MOE is +-4, then Obama could be as low as 48 and McCain could be as high as 50, with every single bit of likelyhood as the 48-46 number.

What matters is consistency. The fact that the polls remain consistent and that day after day the lead remains the same, is what makes the confidence value in the numbers themselves greater.

But MOE absolutely makes a difference, and a poll within the margin of error is absolutely statistically tied. Only when put one after the other and trends, outliers and other analysis is able to form, does the MOE take on less meaning.
Printer Friendly | Permalink |  | Top
 
Viking12 Donating Member (1000+ posts) Send PM | Profile | Ignore Thu Oct-30-08 11:40 AM
Response to Reply #2
3. A poll within the margin of error is NOT statistically tied and the numbers are NOT equal likely
A poll's MOE does represent a 95% confidence interval for each individual's percentage, but it doesn't represent a 95% confidence for the difference between the two, and that's what we're really interested in.

In fact, what we're really interested in is the probability that the difference is greater than zero in other words, that one candidate is genuinely ahead of the other. But this probability isn't a cutoff, it's a continuum: the bigger the lead, the more likely that someone is ahead and that the result isn't just a polling fluke. So instead of lazily reporting any result within the MOE as a "tie," which is statistically wrong anyway, it would be more informative to just go ahead and tell us how probable it is that a candidate is really ahead. Here's a table that gives you the answer to within a point or two:



http://www.washingtonmonthly.com/archives/individual/20...
Printer Friendly | Permalink |  | Top
 
Kurt_and_Hunter Donating Member (1000+ posts) Send PM | Profile | Ignore Thu Oct-30-08 11:42 AM
Response to Reply #2
4. The tail ends of the MOE are not just as likely
"If the MOE is +-4, then Obama could be as low as 48 and McCain could be as high as 50, with every single bit of likelyhood as the 48-46 number."

That's not true. If Obama is at 48% then 48% is the likeliest number. 46% and 48% are less likely. 45% and 49% even less likely.

At some point extending that number reaches the point where you can say, "There is a 95% chance the accurate figure is within this range."

There is another range where it is 93% likely.

The point is, dismissing a result for being only 93% likely versus 97% likely is arbitrary. It's a sliding scale.

A poll is not magically accurate at 95% and trash at 94%.

The 95% is an arbitrary standard that allows us to compare polls to each other easily.
Printer Friendly | Permalink |  | Top
 
Gman Donating Member (1000+ posts) Send PM | Profile | Ignore Thu Oct-30-08 11:48 AM
Response to Reply #2
5. You've got it
The MOE is everything in polling as you describe. The MOE is a calculation that takes into account the size of the sample versus the population. The larger the sample, the smaller the MOE. I can go out and poll 10 people. If 5 are voting for Obama, 3 for McCain and 2 undecided, my poll says Obama leads McCain by 20% (50% - 30%). But the poll is worthless because the margin of error is huge because of the very small sample.

It's only when a lead is outside the MOE that you can say a candidate is likely winning. Note that you cannot say flat out that a candidate is winning. T

And BTW, consistency can also mean that you're making the same error consistently.
Printer Friendly | Permalink |  | Top
 
On the Road Donating Member (1000+ posts) Send PM | Profile | Ignore Thu Oct-30-08 12:19 PM
Response to Original message
6. Originally, It Was a Convention Used for Academic Publishing
Since to follow the scientific method you needed a quantitative standard that would have a Yes/No answer. In other words, they ARE supposed to distinguish between 94% and 96% in journal articles. In practice, many of them use weasel words if their results are almost but not quite significant.

Doesn't mean everyone else has to do it in casual analysis. I've made the same point on other threads, but a lot of people still seem tied to the MOE as a magic number.
Printer Friendly | Permalink |  | Top
 
ddeclue Donating Member (1000+ posts) Send PM | Profile | Ignore Thu Oct-30-08 12:21 PM
Response to Original message
7. It comes from a Gaussian (bell curve) distribution curve and assuming 2 sigma..
but there really is not guarantee that these polls are Gaussian in nature of distribution.
Printer Friendly | Permalink |  | Top
 
DU AdBot (1000+ posts) Click to send private message to this author Click to view 
this author's profile Click to add 
this author to your buddy list Click to add 
this author to your Ignore list Thu Jul 10th 2014, 03:33 AM
Response to Original message
Advertisements [?]
 Top

Home » Discuss » Archives » General Discussion: Presidential (Through Nov 2009) Donate to DU

Powered by DCForum+ Version 1.1 Copyright 1997-2002 DCScripts.com
Software has been extensively modified by the DU administrators


Important Notices: By participating on this discussion board, visitors agree to abide by the rules outlined on our Rules page. Messages posted on the Democratic Underground Discussion Forums are the opinions of the individuals who post them, and do not necessarily represent the opinions of Democratic Underground, LLC.

Home  |  Discussion Forums  |  Journals |  Store  |  Donate

About DU  |  Contact Us  |  Privacy Policy

Got a message for Democratic Underground? Click here to send us a message.

© 2001 - 2011 Democratic Underground, LLC