Democratic Underground Latest Greatest Lobby Journals Search Options Help Login
Google

Kos: Zogby "interactive" polls are junk

Printer-friendly format Printer-friendly format
Printer-friendly format Email this thread to a friend
Printer-friendly format Bookmark this thread
This topic is archived.
Home » Discuss » Archives » General Discussion: Presidential (Through Nov 2009) Donate to DU
 
SaveElmer Donating Member (1000+ posts) Send PM | Profile | Ignore Wed Nov-28-07 10:51 AM
Original message
Kos: Zogby "interactive" polls are junk
In a refreshing bit of clarity from Markos...


I know Hillary's opponents are jumping on Zogby Interactive's latest poll showing Hillary doing substantially poorer than her opponents in head-to-head matchups than her opponents.

Let me make this as clear as possible: Zogby interactive polls are JUNK. They are about as solid as the Daily Kos cattle call polls would be if we were trying to claim the community represented all Democrats.

Witness this little bit of disclaimer:

The poll of 9,355 people had a margin of error of plus or minus one percentage point. The interactive poll surveys individuals who have registered to take part in online polls.


How a poll that is essentially a web poll can be considered credible is beyond me. But you don't have to take my word for it. Look at how poorly the Zogby interactive poll performed in 2006 (after a disastrous debut in 2004):

I opted to use Mr. Mitofsky's method in my own number crunching. I looked at five pollsters that were among the most prolific: Rasmussen, SurveyUSA, Zogby (which releases separate telephone and online polls) and Washington, D.C.-based Mason-Dixon. For all but the latter, I used the numbers posted on the organizations' own Web sites. For Mason-Dixon, which keeps some of its poll data behind a subscriber wall, I used Pollster.com to find polls from the two weeks before the election. I checked the results against vote counts as of this Tuesday <...>

On to the results: In the Senate races, the average error on the margin of victory was tightly bunched for all the phone polls. Rasmussen (25 races) and Mason-Dixon (15) each were off by an average of fewer than four points on the margin. Zogby's phone polls (10) and SurveyUSA (18) each missed by slightly more than four points. Just four of the 68 phone polls missed by 10 points or more, with the widest miss at 18 points.

But the performance of Zogby Interactive, the unit that conducts surveys online, demonstrates the dubious value of judging polls only by whether they pick winners correctly. As Zogby noted in a press release, its online polls identified 18 of 19 Senate winners correctly. But its predictions missed by an average of 8.6 percentage points in those polls -- at least twice the average miss of four other polling operations I examined. Zogby predicted a nine-point win for Democrat Herb Kohl in Wisconsin; he won by 37 points. Democrat Maria Cantwell was expected to win by four points in Washington; she won by 17 <...>

The picture was similar in the gubernatorial races (where Zogby polled only online, not by phone). Mason-Dixon's average error was under 3.4 points in 14 races. Rasmussen missed by an average of 3.8 points in 30 races; SurveyUSA was off by 4.4 points, on average, in 18 races. But Zogby's online poll missed by an average of 8.3 points, erring on six races by more than 15 points.


Seriously, Zogby polls suck. Yet according to Google News right now, the "Hillary loses against all Republicans in the general" poll has been cited by over 200 media, while the far more respectable Gallup effort which shows that Hillary in fact beats them all has been far less reported.

The Zogby survey was covered repeatedly on CNN, earned coverage from MSNBC, Fox News, and Reuters and was covered by multiple other smaller outlets.

By contrast, I can't find a single example of any reporter or commentator on the major networks or news outlets referring to the Gallup poll at all, with the lone exception of UPI. While the Zogby poll was mentioned by multiple reporters and pundits, the only mentions the Gallup poll got on TV were from Hillary advisers who had to bring it up themselves on the air in order to inject it into the conversation.

You could argue that the Zogby poll got all the coverage it did precisely because it is out of sync with multiple other polls, and thus is news. But the truth is that the reporters and editors at the major nets know full well that the Zogby poll is bunk -- yet they breathlessly covered it anyway.

Worse, the Zogby poll was covered with few mentions either of its dubious methodology or of the degree to which its findings don't jibe with other surveys. Bottom line: The Zogby poll was considered big news because many in the political press are heavily invested in the Hillary-is-unelectable narrative for all kinds of reasons that have little to do with a desire to, you know, practice journalism.


The media has its agenda, which right now is the "Hillary is fading" narrative. The hard core supporters of the other Democratic primary candidates have their agenda -- to raise bullshit "electability" arguments against Hillary.

And those of us who remain reality-based and dispassionate throughout this all can only shake our heads at the credence being put in a discredited shill of a "pollster".


http://www.dailykos.com/storyonly/2007/11/28/95347/662
Printer Friendly | Permalink |  | Top
wyldwolf Donating Member (1000+ posts) Send PM | Profile | Ignore Wed Nov-28-07 10:53 AM
Response to Original message
1. yep. The money quote: "bullshit "electability" arguments against Hillary.
Printer Friendly | Permalink |  | Top
 
weeve Donating Member (427 posts) Send PM | Profile | Ignore Wed Nov-28-07 12:04 PM
Response to Reply #1
25. Agreed
By mocking and minimalizing the issue of Hillary's (un)electability, this writer shows his bias and undercuts his entire arguement.

As should be expected. Hillary's fans have used the sacrosanct POLLING arguement as their way to avoid talking substantially about Hillary's hawkish, corporate policies. So it's natural they'd go balistic when one of their beloved POLLS comes up the wrong way.
Printer Friendly | Permalink |  | Top
 
wyldwolf Donating Member (1000+ posts) Send PM | Profile | Ignore Wed Nov-28-07 12:14 PM
Response to Reply #25
29. disagreed
By using "progressive"* buzzwords like "hawkish, corporate policies," the writer fails to acknowledge Hillary supporters will always engage in a fact-based non-hypocritical policy discussion that doesn't erupt into regurgitated ad hominem attacks by the "progressive"* on the other end of the discussion.
Printer Friendly | Permalink |  | Top
 
Texas_Kat Donating Member (1000+ posts) Send PM | Profile | Ignore Wed Nov-28-07 10:56 AM
Response to Original message
2. I don't always agree with Kos
but this time he's dead-on right.
Printer Friendly | Permalink |  | Top
 
Colobo Donating Member (1000+ posts) Send PM | Profile | Ignore Wed Nov-28-07 10:56 AM
Response to Original message
3. It's only bullshit because you guys don't like the results.
Printer Friendly | Permalink |  | Top
 
SaveElmer Donating Member (1000+ posts) Send PM | Profile | Ignore Wed Nov-28-07 10:58 AM
Response to Reply #3
5. C'mon Katz..you weren't always so adverse to facts...
Even you must see how poor this poll is...

I don't generally complain about polls being rigged etc...

but the methodology on this stinks...and you know it...

Any poll where you have to take an active step to be involved cannot possible be representative...
Printer Friendly | Permalink |  | Top
 
DemocratSinceBirth Donating Member (1000+ posts) Send PM | Profile | Ignore Wed Nov-28-07 11:05 AM
Response to Reply #3
9. Do You Know The Difference Between A Probability And A Non Probabilty Sample?
Edited on Wed Nov-28-07 11:05 AM by DemocratSinceBirth
DSB
Printer Friendly | Permalink |  | Top
 
Teaser Donating Member (1000+ posts) Send PM | Profile | Ignore Wed Nov-28-07 11:18 AM
Response to Reply #3
13. I liked the results, but the poll is bullshit
Edited on Wed Nov-28-07 11:27 AM by Teaser
I am an Obama supporter, although he's made it harder of late.

The Zogby poll was crap. It is a self selected sample of internet users.

And we all know that Obama (and to some extant, Edwards) base is stronger among frequent net users than is Hilary's.
The bias is so clear as to be unavoidable.

Until Zogby figures out a fudge factor for this self-selection bias, polls like this will be useless.
Printer Friendly | Permalink |  | Top
 
KingFlorez Donating Member (1000+ posts) Send PM | Profile | Ignore Wed Nov-28-07 12:14 PM
Response to Reply #3
30. It's not accurate
I don't buy that any Republican could be leading over any of the Democratic front runners.
Printer Friendly | Permalink |  | Top
 
Mutineer Donating Member (659 posts) Send PM | Profile | Ignore Wed Nov-28-07 02:11 PM
Response to Reply #3
35. If they showed HRC beating the GOP she would be on the campaign trail
talking smack about how great the poll was and what wonderful proof it was she could win the general election. I see this "Zogby Polls are bogus" meme as the latest in the ongoing HRC propaganda war.
Printer Friendly | Permalink |  | Top
 
DemocratSinceBirth Donating Member (1000+ posts) Send PM | Profile | Ignore Wed Nov-28-07 02:36 PM
Response to Reply #35
37. The Poll Is A Major Outlier
Here's a poll showing her beating The Ghoul by nine points in (Florida)...


http://www.democraticunderground.com/discuss/duboard.ph...


There's no way Clinton could be beating Ghouliani in Florida by nine points and losing to him nationally by four points... It's just not how these things work...
Printer Friendly | Permalink |  | Top
 
wyldwolf Donating Member (1000+ posts) Send PM | Profile | Ignore Wed Nov-28-07 10:57 AM
Response to Original message
4. ...I'm off to read the comments at Kos on this which are sure to sound like pigs squealing
Printer Friendly | Permalink |  | Top
 
durrrty libby Donating Member (1000+ posts) Send PM | Profile | Ignore Wed Nov-28-07 10:59 AM
Response to Original message
6. Zogby went on Joe in the Morning to say how awesome and reliable his polls are
Printer Friendly | Permalink |  | Top
 
Texas_Kat Donating Member (1000+ posts) Send PM | Profile | Ignore Wed Nov-28-07 11:03 AM
Response to Reply #6
8. Since he uses these to 'sell' his services
to retailers, you can depend on him to defend his pocketbook.

Junk poll.... yeah, it is.
Printer Friendly | Permalink |  | Top
 
TwilightGardener Donating Member (1000+ posts) Send PM | Profile | Ignore Wed Nov-28-07 11:02 AM
Response to Original message
7. Online polls are always crap. I didn't believe the results, even though they
favored my candidate. Romney beating Hillary? Come on, people, he couldn't beat a dead man on roller skates.
Printer Friendly | Permalink |  | Top
 
Crabby Appleton Donating Member (1000+ posts) Send PM | Profile | Ignore Wed Nov-28-07 11:55 AM
Response to Reply #7
23. You are correct, dog nt
Printer Friendly | Permalink |  | Top
 
aquart Donating Member (1000+ posts) Send PM | Profile | Ignore Wed Nov-28-07 11:06 AM
Response to Original message
10. I'm in the middle of a Zogby poll right now.
It's asking about local services and infrastructure. I'm here because once again the Zogby server is making me wait. This may turn out to be the third poll I couldn't finish or submit. Yup. Still "loading." I don't think their polls are crap. But I do think they are having serious technical problems that should have been corrected by now.
Printer Friendly | Permalink |  | Top
 
Tellurian Donating Member (1000+ posts) Send PM | Profile | Ignore Wed Nov-28-07 11:15 AM
Response to Reply #10
12. Could be a method used for skewing the polls..
using your computer's cookie then when you freeze, fill in the rest of the poll manually for the desired result.
Printer Friendly | Permalink |  | Top
 
Teaser Donating Member (1000+ posts) Send PM | Profile | Ignore Wed Nov-28-07 11:21 AM
Response to Reply #12
15. You're insane if you believe that.
Zogby's aim is to make money. And he makes money by generating polls that allow his clients to plan their campaign strategies. And to do that, his clients have to have accurate information.

There is no percentage in manipulating the results.
Printer Friendly | Permalink |  | Top
 
Texas_Kat Donating Member (1000+ posts) Send PM | Profile | Ignore Wed Nov-28-07 11:31 AM
Response to Reply #15
17. Multiple usernames... multiple email addresses... multiple responses
from single individuals......

"Playing" with the demographic questions will get you invited to participate in his polls more frequently. Swarming (requires signing up ahead of time) by partisan 'groups' skews the thing too. This isn't the first -- or the last -- junk poll he'll post.

Sure there is a "percentage in manipulating the results' -- both from partisans and from Zogby. He uses them as a cheap way to convince retailers that they can get valid 'marketing' info. Embedded in this poll were questions about the attractiveness of purchasing "Red*" products -- Bono's Global Fund to Fight AIDS, Tuberculosis and Malaria.

They've asked questions (in the past) about everything from handheld PDA's to cars to soap. If Zogby wants to keep raking in cash for these junk polls, he has to defend them.

The project, called "Red," will market red-themed products from Converse footwear, Gap clothing and Giorgio Armani, as well as a red American Express card to be offered initially only in Britain, as of next month.

Products branded "Red" will include sports shoes, T-shirts and sunglasses some produced in Africa, some with African materials. A slice of the revenue numbers vary by company and product will go to the Global Fund to Fight AIDS, Tuberculosis and Malaria.

Printer Friendly | Permalink |  | Top
 
Tellurian Donating Member (1000+ posts) Send PM | Profile | Ignore Wed Nov-28-07 12:09 PM
Response to Reply #17
27. Thanks, TK..I didn't see your response until after I posted mine..
Printer Friendly | Permalink |  | Top
 
Teaser Donating Member (1000+ posts) Send PM | Profile | Ignore Wed Nov-28-07 01:11 PM
Response to Reply #17
32. Incorrect
First off, you still have failed to demonstrate a reason why an opinion firm would manipulate data like this for a client (in this case a media organization).

Secondly, those weird demographic questions in Zogby's surveys have not as much to do with advertising, as they have to do with his "special sauce."

Zogby is different from almost all other pollsters in the extent to which he relies on population models to weight his data.
This is done, in order to more accurately gauge his "likely voter" models. Presumably Z has found that various other factors correlate strongly with terms
in his turnout models.

Zogby's online polls are an attempt to take this mathematicalization of opinion to a whole new level. He asks all those pop culture and marketing questions because he's trying to find, buried within the opinion noise, a means of excluding self-selector bias.

It's doubtful he'll succeed, and they are the reason this kind of poll delivers results that fly in the face of other polling.

You are, of course, entitled to your tin foil. But leaving it lying around for the rest of us is just litter.
Printer Friendly | Permalink |  | Top
 
Texas_Kat Donating Member (1000+ posts) Send PM | Profile | Ignore Wed Nov-28-07 01:35 PM
Response to Reply #32
33. Sorry you misunderstood
Edited on Wed Nov-28-07 01:36 PM by Texas_Kat
Partisans can manipulate these online polls (and do) -- Zogby is just interested in defending them as 'valid' because he 'sells' their validity to commercial interests.

Those his data wonks choose to send the polls to are 'selected' based on the answers to demographic questions he asks (his 'population model'. All of which can be manipulated easily -- and in multiples. GARBAGE IN -- GARBAGE OUT.

I don't have a problem with Zogby's traditional polling, just these online junk polls. You cannot possibly be arguing that a self-selected, unverifiable universe can IN ANY WAY yield valid results. His "results that fly in the face of other polling" because no matter how you manipulate data AFTER it's been poorly collected, you still wind up with 2+2=5.

Personally, I hope the Obama folks believe this poll.

Printer Friendly | Permalink |  | Top
 
Tellurian Donating Member (1000+ posts) Send PM | Profile | Ignore Wed Nov-28-07 12:07 PM
Response to Reply #15
26. So, convince me..
By your assessment of how they make money. Who hires them? Would a presidential candidates campaign qualify as a good client? Or a news media outlet?
Printer Friendly | Permalink |  | Top
 
DemocratSinceBirth Donating Member (1000+ posts) Send PM | Profile | Ignore Wed Nov-28-07 12:11 PM
Response to Reply #26
28. Doing A Poll Correctly Is Not Rocket Science
You could probably find a grad student in any of the social sciences or finance who could conduct a poll...
Printer Friendly | Permalink |  | Top
 
Tellurian Donating Member (1000+ posts) Send PM | Profile | Ignore Wed Nov-28-07 07:05 PM
Response to Reply #28
40. It's not the poll, per se..
It's the exact moment when the pollee is most susceptible to the skewing of results, that interests me.
As when the notable freeze occurs...and the pollee has no choice then to vacate the poll after all the pertinent ID information has been recorded.
Printer Friendly | Permalink |  | Top
 
DemocratSinceBirth Donating Member (1000+ posts) Send PM | Profile | Ignore Wed Nov-28-07 11:19 AM
Response to Reply #10
14. The Problem Is It's A Non Probability Sample
You have to register to be polled and the "randomness" which makes a poll legitimate instantly disappears... There's no technological fix...
Printer Friendly | Permalink |  | Top
 
Teaser Donating Member (1000+ posts) Send PM | Profile | Ignore Wed Nov-28-07 11:26 AM
Response to Reply #14
16. Not entirely true.
Edited on Wed Nov-28-07 11:27 AM by Teaser
Any sample is based on a probability distribution. This kind of sample, though, is not a random sample, and, as such, we can't use the central limit theorem to calculate population statistics (like average percent support). However, the biggest problem with this polls is that the distribution it draws from is biased.

What Zogby is looking for, when he does these polls, though is other correlates of that bias. For example, do certain opinions about local government or services or one's shopping habits correlate strongly with an increase or decrease in support for a particular candidate/party?

He's trying to quantify the number of points that gets added to a candidates number based on regressions against these other variables. Then when he finds the adjustment, he can re-weight the data and find the optimal estimate of the "true" random variable (the right percent of people supporting Hilary, for example). Essentially he's trying to find a way around the self selection bias.

Will it work? I'm dubious, but its possible. Indeed, I hope it does work, as it will make opinion research vastly more efficient.

But right now, it's not ready for prime time, and Z should not be releasing this data.
Printer Friendly | Permalink |  | Top
 
DemocratSinceBirth Donating Member (1000+ posts) Send PM | Profile | Ignore Wed Nov-28-07 11:48 AM
Response to Reply #16
22. I Agree
Two points

1) Political polls demand a greater degree of specificity...Every pollster wants to predict the winner and margin...

2) I don't know how polling will be affected as land line use continues to decrease... That's why pollsters are trying to use the net but a non probability sample is still a non probability sample...
Printer Friendly | Permalink |  | Top
 
DemocratSinceBirth Donating Member (1000+ posts) Send PM | Profile | Ignore Wed Nov-28-07 02:50 PM
Response to Reply #16
39. I Thought About It
Technically it is a probability sample... Everybody that (registered) has an equal probability of being sampled...It's just not random...But that's almost a semantic difference...
Printer Friendly | Permalink |  | Top
 
sufrommich Donating Member (1000+ posts) Send PM | Profile | Ignore Wed Nov-28-07 11:40 AM
Response to Reply #14
20. Zogby also invites participants to ask a friend to sign up for
the polls.How many of us would invite a friend likely to cancel out our "vote". I do the Zogby polls once and awhile,(the last one froze up on me while I was in the middle of taking it)but quit taking them seriously a long time ago.
Printer Friendly | Permalink |  | Top
 
aquart Donating Member (1000+ posts) Send PM | Profile | Ignore Wed Nov-28-07 12:58 PM
Response to Reply #14
31. But culled phone lists are acceptable?
Statistically, email and phone polls match up. That's why businesses and politicians who need them to make winning decisions feel confident in using them.

It's true that there were objections to a rightwing skew on Zogby. Zogby responded by working with Buzzflash to encourage liberals to sign up. We did. That's why so many here make mention of having done a Zogby poll.

Printer Friendly | Permalink |  | Top
 
DemocratSinceBirth Donating Member (1000+ posts) Send PM | Profile | Ignore Wed Nov-28-07 02:34 PM
Response to Reply #31
36. No
Most pollsters use random dialing not phone books...

In the Zogby poll you have to be registered to be polled... The self selection destroys the very notion of "randomness" which is elemental to any poll...
Printer Friendly | Permalink |  | Top
 
Bo Bike Donating Member (89 posts) Send PM | Profile | Ignore Wed Nov-28-07 11:14 AM
Response to Original message
11. Hillarys is targeted by the Corporate Media
because she is in the lead. Thats boring to report day after day. A Hillary blow out will not generate ratings but a CM trash job will.
Printer Friendly | Permalink |  | Top
 
TeamJordan23 Donating Member (1000+ posts) Send PM | Profile | Ignore Wed Nov-28-07 11:32 AM
Response to Original message
18. But if they are junk, why has Penn requested most Zogby interactive polls than any other campaign???
Edited on Wed Nov-28-07 11:33 AM by TeamJordan23
Printer Friendly | Permalink |  | Top
 
wyldwolf Donating Member (1000+ posts) Send PM | Profile | Ignore Wed Nov-28-07 11:34 AM
Response to Reply #18
19. ... according to John Zogby
Printer Friendly | Permalink |  | Top
 
TeamJordan23 Donating Member (1000+ posts) Send PM | Profile | Ignore Wed Nov-28-07 02:45 PM
Response to Reply #19
38. You would think Penn would deny it if Zogby was lying. nm
Printer Friendly | Permalink |  | Top
 
sufrommich Donating Member (1000+ posts) Send PM | Profile | Ignore Wed Nov-28-07 11:44 AM
Response to Reply #18
21. it's entirely possible that he
is making the same request of all the pollsters more than any other campaigns are.
Printer Friendly | Permalink |  | Top
 
durrrty libby Donating Member (1000+ posts) Send PM | Profile | Ignore Wed Nov-28-07 11:59 AM
Response to Reply #18
24. umm campaigns collect all manner of info? How's that for a reason?
Printer Friendly | Permalink |  | Top
 
Texas_Kat Donating Member (1000+ posts) Send PM | Profile | Ignore Wed Nov-28-07 01:38 PM
Response to Reply #18
34. Because data wonks just can't help themselves
Kos does a cattle call once a month. You can bet your bottom dollar every pollster 'looks' at it too.
Printer Friendly | Permalink |  | Top
 
ClarkUSA Donating Member (1000+ posts) Send PM | Profile | Ignore Wed Nov-28-07 07:13 PM
Response to Reply #18
41. That's a good question. Kos didn't answer that question, did he?
Edited on Wed Nov-28-07 07:19 PM by ClarkUSA
I am not aware that Kos has a background as a professional pollster. I wouldn't trust his opinion on this. And like you pointed out, Penn sure seems
to trust Zogby's data enough to request it repeatedly and often. Kos' comparing Zogby's polls with his own are laughable. I don't see Penn or any other
campaign asking for the results of Kos' polls.

Also, it's funny how the same Clintonian DLCers who denigrate all things Kos suddenly think his opinion matters.


Printer Friendly | Permalink |  | Top
 
antiimperialist Donating Member (1000+ posts) Send PM | Profile | Ignore Wed Nov-28-07 08:57 PM
Response to Reply #41
42. Clark, Zogby Interactive polls suck and you are on denial
Would you like to debate the accuracy or Zogby Interactive polls throughout its history? Would you like to discuss what experts say about its methodology? Or would you like to stop being in denial and stop believing crap like this?

It's a challenge.
Printer Friendly | Permalink |  | Top
 
antiimperialist Donating Member (1000+ posts) Send PM | Profile | Ignore Wed Nov-28-07 09:00 PM
Response to Reply #18
43. Teamjordan, are you capable of thinking critically on your own?
If you are going to make a case that Zogby Interactive polls are accurate, find the evidence. There is plenty of data available showing that ZI polls are terrible. To say that because Mark Penn often requests these polls is not evidence.
Printer Friendly | Permalink |  | Top
 
DeepModem Mom Donating Member (1000+ posts) Send PM | Profile | Ignore Wed Nov-28-07 09:04 PM
Response to Original message
44. K&R
Printer Friendly | Permalink |  | Top
 
DU AdBot (1000+ posts) Click to send private message to this author Click to view 
this author's profile Click to add 
this author to your buddy list Click to add 
this author to your Ignore list Thu Aug 28th 2014, 06:41 PM
Response to Original message
Advertisements [?]
 Top

Home » Discuss » Archives » General Discussion: Presidential (Through Nov 2009) Donate to DU

Powered by DCForum+ Version 1.1 Copyright 1997-2002 DCScripts.com
Software has been extensively modified by the DU administrators


Important Notices: By participating on this discussion board, visitors agree to abide by the rules outlined on our Rules page. Messages posted on the Democratic Underground Discussion Forums are the opinions of the individuals who post them, and do not necessarily represent the opinions of Democratic Underground, LLC.

Home  |  Discussion Forums  |  Journals |  Store  |  Donate

About DU  |  Contact Us  |  Privacy Policy

Got a message for Democratic Underground? Click here to send us a message.

© 2001 - 2011 Democratic Underground, LLC