Why we can’t trust recent polling showing a ten point lead for Remain

Sorry, Remainers. The BMG poll for the Independent, which caused such a stir over the weekend, looks wrong. The Remain lead, which has hovered around 1-4 per cent in recent weeks, has NOT suddenly widened to ten points in the wake of the government’s embarrassing defeat last week in the House of Commons.

Here are four reasons to be wary of BMG’s figures.

First, they are mainly driven by a seemingly huge shift in people who did not vote in last year’s referendum. Sure, the Remain camp is being swelled by young adults who abstained last time or were not old enough to vote. But previous BMG polls included this phenomenon.

During the summer, when previous non-voters were asked how they would vote in a fresh referendum, they divided fairly steadily: around 45 per cent Remain, 25 per cent Leave. Now, suddenly, BMG say the divide is 67 per cent—16 per cent. As this group comprises more than a fifth of BMG’s total weighted sample (299 out of 1,363), this 51-point Remain lead within this group accounts the reported overall Remain lead.

However, BMG did not actually interview 299 previous non-voters. Its unweighted subsample was barely half that: 156. The margin of error on such a subsample is large—and the very fact that BMG could not track down as many non-voters as it wanted, provides a clear warning (as I know from my experience at YouGov) that the sub-sample may not be as representative as one would like.

Secondly, if it were true that, despite these sampling issues, there had been a sharp shift among non-voters to Remain, there would be some echo of this among other groups. In particular, one would expect to see clear signs of growing buyers’ remorse among Leave voters.

That wouldn’t apply to hardline anti-EU voters—something like two thirds of those who voted Leave 18 months ago. But it would be likely to have some effect on the one-third who were “instrumental” voters: people who are not viscerally anti-EU, but believed that Brexit held the best hope of more jobs, higher pay, less crime, a better-funded NHS and improved access to public services such as local schools and social housing.

The point is that any weakening of these pro-Brexit arguments that is liable to shift large numbers of non-voters into the Remain camp, should also produce some shift among the “instrumental” Leave voters. But BMG’s figures produce no evidence of this. There is no statistically significant rise in buyers’ remorse among Leave voters in this poll compared with previous BMG polls.

The third reason to doubt that BMG’s figures reflect a public reaction to last week’s government defeat in the House of Commons is that the poll was conducted before the vote. Its fieldwork dates were 5^th – 8^th December—that is the week before last.

Fourthly, other polls within the past fortnight show no indication of a lurch to Remain. Two YouGov surveys and one ICM poll indicate that nothing much has changed in recent weeks, despite the turbulence surrounding the events leading to the EU’s decision to allow Brexit negotiations to move to stage two.

They are consistent with a narrow Remain lead, compared with a narrow Leave lead before this year’s general election. Over the past six months, as I discussed in a recent blog, there has been a small shift to Remain, but only a small shift.

To say this is not in any way to denigrate the quality of BMG’s work. Small subsamples of hard-to-reach groups must always be examined with care, for they are liable to trip up even the best research companies. BMG seem to have been unlucky, not culpable.

The larger point is one that applies to enthusiasts on any side of any issue. It is important to resist the temptation to cherry-pick those polls and findings that support one’s case, and ignore those that don’t. Remember the warning of those great 20^th century philosophers, Simon and Garfunkel, in “The Boxer”: “All lies and jest: a man hears what he wants to hear, and disregards the rest.”