HomeLatest ThreadsGreatest ThreadsForums & GroupsMy SubscriptionsMy Posts
DU Home » Latest Threads » Forums & Groups » Places » International » United Kingdom (Group) » Revealed: Secret Labour r... » Reply #3

Response to Ironing Man (Reply #1)

Tue Jan 26, 2016, 12:05 PM

3. I'm glad (I think) that you picked up on this.

I think the criticism of focus groups is a foolish diversion - focus groups is just asking people what they think, i'm somewhat mystified by what the genuine difference between the use of a focus group, and the 'golden age' version of the Labour MP or PPC walking into half a dozen different pubs over a weekend to get a flavour of views on political issues within the constituancy, actually is..


I don't think it's a diversion at all. Focus groups, inasmuch as they're useful at all, are a qualitative measure of opinion, as opposed to polls, which are quantitative measures. They have their uses - sometimes if an organization is short of ideas or doesn't have the faintest clue what's going on, it can provide some leads, for instance. But so often their results are presented as "proof."

I'm certainly not claiming this set of focus groups was the be-all and end-all of the research going on, though the media (and no doubt some politicos who find their preconceptions confirmed) often seize on their findings as gospel. But if it reflects the quality of the rest of the research, there are serious grounds for worry if you're a Labour supporter.

The report reveals some of the flaws of the methodology, and the firm drawing it up has violated what I believe are some basic groundrules.

For starters, look at the size and composition of the sample:



That amounts to 70-80 participants, from 5 areas (a tiny sample compared to any credible polling - but so often these two tools are conflated by the media, and I look forward to you telling me that politicos don't tend to fall into the same trap when presented with results like these), the closest to the north of England being Nuneaton, and the south-east where most voters live being represented by just Watford and Croydon, restricted to the 30-50 age group (I can't argue with the choice of Glasgow and Edinburgh as the two largest Scottish conurbations, though I'd be interested to know how their samples reflected the extremely broad demographic mix in both cities).

That's quite a rarified sample to start with, before we start getting into how participants were enlisted, what inducements had to offered to get them to attend, how issues for discussion were drafted and presented etc.

Then there's the cherrypicking of comments as supposedly representative. Even basic theses or reports may have a preamble pointing out the pitfalls and limitations of focus groups. Here are some excerpts from just one such example (http://muskie.usm.maine.edu/helpkids/pubstext/focus.htm ):

Goals of a focus group

The structured discussion that takes place in a focus group is an effective means of gathering information about why people think or feel the way they do. The goal is not to confirm a preconceived hypothesis; instead, it is to gather perceptions, information, experiences, and understanding through discussion of a specified topic.

Neither is the goal to reach consensus, provide recommendations, or make decisions among alternatives. Information is obtained through open-ended questions and also from observations of the group discussion. To the participants, the questions appear spontaneous but in fact are carefully developed and put into a sequence only after considerable reflection.

...

Drawbacks of focus groups

Focus group results generally have what is known as high "face-value" validity because participants' comments sound so believable. In fact, the results may appear so believable that the researcher's tendency is to rush out and put the focus group's recommendations into action without further investigation. The best plan is to wait and, before adopting specific programmatic strategies, take the time to do additional research and look for back-up of the results in the literature.

Another risk is using focus group data to generalize to a larger population. Remember that the group of participants, also known as the sample, does not necessarily reflect the entire population. For that reason, critics say that focus groups are merely an exploratory research tool. For an organization, the real question is: are the data credible enough to use in making decisions?


I'll leave out this report's summary of the method's advantages as you can click through and read it if you want. But here are what it identifies as its pitfalls:

Disadvantages

* The moderator has less control in the group than in an individual interview. Because individual group members can influence the course of discussion, it is not always possible to know if data reflect the behavior and perceptions of individual participants or the group.
* Because focus groups are held in an arranged or controlled setting, participants may behave differently than in private interviews, and this may affect what is expressed.
* Data are difficult to analyze. Be careful not to lift comments out of context or sequence or come to premature conclusions.
* The technique requires carefully trained interviewers. While an untrained moderator can achieve remarkable results it is far better to influence the odds by using skilled people.
* Because each focus group is different and has its own characteristic, you must take each group on its own terms; there is no cookbook method for working with the groups.
* Depending on the available staff support, recruiting participants can be a difficult and time consuming process.
* The discussion must be conducted in an environment that is conducive to conversation. To attract the types of people you want, you may need to provide incentives to ensure that participants will come to the meeting.


My bolded bit up there is exactly what this report has done, if you care to look. It also generalizes ridiculously: "Edinburgh said ..." etc. No, it frikkin didn't!

Whatever, your later comments come close to some of what I was getting round to driving at: "the 'golden age' version of the Labour MP or PPC walking into half a dozen different pubs over a weekend to get a flavour of views on political issues within the constituancy, actually is."

It is indeed a rare MP nowadays who does this (not that a sample of weekend pubgoers inclined to chat to an MP is great methodology anyway). In the light of anecdotal public comments on the likes of Twitter about Labour's canvassing in Scotland (beyond the usual "Great evening's canvassing - good response on the doorsteps", even that opportunity for face-to-face conversations at an individual level wasn't fruitful or exploited successfuly - there are a number of post-election media quotes I could find if driven to, to the effect that canvassers were finding the electorate in large areas, if not uniformly, totally unreceptive and alienated: "There's no talking to them." Coupled with the drastic depletion of activists willing to voluntarily do some doorknocking, this reflected the distance that's grown between the party and those it's claimed to represent until the last election, in Scotland at least.

How you overcome that problem risks being a chicken-and-egg question. The party will have difficulty re-wooing voters while it remains so centralized and publicly hostile toward the unwelcome proles who've had the cheek to take up the invitation to join it, let alone those whose vote it's going to contort itself to court.

If it relies on the media, well, it has to play their game, and it'll get shafted on a regular basis. To talk about Scotland again for a moment, the SNP did and does regularly get shafted by the media - that's not me being unduly partisan, there are numerous examples. The thing is, it doesn't seem to make any difference. The media are pernicious, but unpopular enough among enough of the electorate that most of the time their usual strategies just don't work any more up here. They're generally out of touch with the populace as Labour is.

So it comes back to various types of word of mouth to bypass the traditional media. But if the old activists are discredited, except in certain notable constituencies, how can new activists canvass for a party successfully when numerous of its luminaries have been queuing up in the media on a daily basis to condemn and dismiss them as loons, entryists, or worse?

Reply to this post

Back to OP Alert abuse Link to post in-thread

Always highlight: 10 newest replies | Replies posted after I mark a forum
Replies to this discussion thread
Arrow 4 replies Author Time Post
Denzil_DC Jan 2016 OP
Ironing Man Jan 2016 #1
LineLineNew Reply I'm glad (I think) that you picked up on this.
Denzil_DC Jan 2016 #3
Bad Dog Jan 2016 #2
T_i_B Jan 2016 #4
Please login to view edit histories.