EU referendum polls - a consistent difference between phone and online polls

[muriel_volestrangler]

This Wikipedia page is, I think, getting updated with all the latest polls.

https://en.wikipedia.org/wiki/Opinion_polling_for_the_United_Kingdom_European_Union_membership_referendum

There's a notable difference in the margins of phone and online polls. You can sort the table for the 2016 polls (near the top of the article) by poll type. Online polls are very close - most recently, slightly more giving 'leave' a lead of a percentage point or two. But the phone polls have given 'remain' a lead of 7 points or so.

This is quite a difference. I can't tell what might be causing it, or which might turn out to be more accurate; there's a good case for saying this shows no polls have much meaning for the referendum. But, if polls come up, I think it's a point it's good to bear in mind.

[Denzil_DC]

1. I don't see any mention of margin of error on that page.

Aside from individual polls' methodological inaccuracies and biases, they may be weighted differently, so aggregating them is going to result in a statistical mess.

Standard MoE for single polls of 1,000 or so is usually around 3 points. I haven't seen much that convinces me that any variations are more than statistical noise.

On sheer gutfeel, I reckon it's running neck-and-neck, and has done for a long time. Turnout will be crucial. If Remain gets out its supporters, it'll win. If not ...

[muriel_volestrangler]

2. Yeah, it's usually around 3 points - a bit less for samples of 2000

of which there are a few. Which is why a consistent difference of this size is notable. Here are the percentage points for a 'remain' lead, plotted against the end date of the poll, with trend lines:



I think that's more than statistical noise. Yes, it might be biases or methodological inaccuracies, but you'd expect each company to try to get the same demographics for online and phone polls.

Not many companies do both phone and internet polling, but ICM did, both ending on the 17th April, and the remain lead for the phone poll of 1,000 was +7, while for the 2,000 online poll it was -1. ORB did a phone poll (800, 24th Apr) which gave +8, and online (2,000, 29th Apr) -2. Populus phone (1,000, 6th Mar) +12; online (4,000, 10th Mar) +3.

So, each time, we see a company getting a significantly (ie outside the margin of error) larger lead for Remain by phone than online.

[Denzil_DC]

3. I'd always be more skeptical of online polls.

What's the methodology? To what extent is it self-selecting? Is there consistent polling of respondents who are re-polled? All that sort of thing. You'd expect all sorts of professional things of these polling bodies, but they haven't done us proud in the past.

Same sorts of questions apply to phone polling. We've seen over successive US electoral cycles how skewed they can be in a different country. Was this landline-only?

If so, that would usually indicate an older demographic as households abandon landlines, at least as their main voice communication means, and younger people tend to rely much more on mobiles as their main platform - but how does that tie in with conventional wisdom about generational effects in terms of Remain/Leave preferences?

If not, that raises its own questions.