Driving Us Up The Poll.

Rubbish In, Rubbish Out: Put all this together, and it’s difficult to avoid the conclusion that anyone who responds positively to a pollster’s request to “answer a few questions” is just ever-so-slightly weird. Desperately lonely? Some sort of psephological train-spotter? Political party member primed to skew the poll for or against her opponents? All of the above?

THE INCONVENIENT TRUTH about opinion polls is that the people who participate in them are not really typical Kiwis. Agreeing to participate in anything remotely public-spirited is something fewer and fewer New Zealanders are willing to do. Charities struggle to attract volunteers. Sports teams can’t get enough players. Political parties have long since ceased to be mass organisations. Just finding enough people to satisfy the statistical requirements for an accurate public opinion survey gets harder and harder with every passing year.

It wasn’t always this way. Back in the era when nearly every New Zealand household had an old-fashioned land-line telephone; and the easiest way to locate somebody was to simply ‘look them up’ in the phonebook; polling was a breeze.

It was a time when community action and political debate was engaged in by an extremely broad cross-section of the population. Indeed, New Zealanders were gently chided by Austin Mitchell, the best-selling author of The Half-Gallon, Quarter-Acre, Pavlova Paradise, for being inveterate committee formers. “Pressure groups” were studied forensically by political scientists. Overseas visitors marvelled at a nation of joiners.

The late Professor Keith Jackson, in his book New Zealand: Politics of Change, confirms the strongly participatory character of our democracy by citing the research of R.S. Milne:

“Membership of the New Zealand Labour Party which had peaked in the year 1939-40 at 235,605 remained high after the war at 201,765. By 1960, however, this figure was down to 180,000 distributed through more than 600 branches.”

National’s engagement with New Zealanders was no less impressive: “Much the same pattern appears to have developed within the National Party. Speaking in 1956 the President of the National Party claimed that membership varied from 143,000 in a non-election year to 250,000 in an election year.”

In a nation this politicised, the opinion polling companies of the 1960s and 70s easily assembled the requisite number of participants.

The contrast between those times and our own could hardly be sharper. Who uses the land-line-generated phone book anymore? Asked to do so, most younger Kiwis would probably look at you blankly. The ubiquitous cell-phone presents the pollsters with endless difficulties. There’s no “phone book” for a start, and caller ID allows us all to screen our incoming calls. Many people simply don’t answer unidentified callers – justifiably fearing tele-marketers and scammers.

These latter miscreants have become the bane of land-line subscribers’ lives. For many citizens – especially the elderly – it is considered foolhardy to converse with anyone whose voice isn’t instantly recognisable. Someone can say they’re calling from Colmar Brunton or Reid Research – but how do you know? Better to politely decline and hang-up the receiver.

Put all this together, and it’s difficult to avoid the conclusion that anyone who responds positively to a pollster’s request to “answer a few questions” is just ever-so-slightly weird. Desperately lonely? Some sort of psephological train-spotter? Political party member primed to skew the poll for or against her opponents? All of the above?

These distorting possibilities are only increased when the fact that landlines tend to be attached to owner-occupied dwellings is factored into the polling equation. Just ask any Gen-Xer or Millennial what sort of person is likely to pick up the phone in their own home and they will hiss “Baby Boomer!” Quite correctly. Which way, do you suppose, a voter sitting on a million dollars-plus of tax-free capital gain is more likely to vote – Left or Right? No wonder, really, that about 45 percent of the Party Vote appears to be welded-on to the National Party!

So, what do the pollsters do? Basically, they innovate. They try to assemble a representative number of cell-phone-using voters to offset the encrusted biases of land-liners. Or, like the new kid on the New Zealand polling block – YouGov – they step away from phones altogether in favour of a “panel” of potential online participants many thousands strong.

Trouble is, these innovations require the pollsters to run the raw data through all manner of algorithms to make sure their samples remain representative. They then have to make some, frankly, subjective assumptions about voter behaviour. That’s when things can turn very seriously pear-shaped.

The highly-experienced pollster advising the campaigners for “Remain” in 2016 assumed those who didn’t vote in the 2015 UK General Election would also sit out the Brexit referendum.

That worked out well.

This essay was originally published in The Otago Daily Times of Friday, 6 December 2019.

Opinion Polling In 2019: More Art Than Science?

Conflicting Messages: With so few publicly available opinion polls being conducted in this country, we really do need to know how one reputable polling agency can put National on 46 percent, while the other puts it on 37 percent. Nine percentage points is a hell of a gap – well beyond the usual statistical tolerances. The public is entitled to know what happened – and which one of them got it right!

IT WAS ANDREW COOPER wot won it! Or so might the Brexiteers have crowed, paraphrasing the Sun’s famous headline of 1992, when its concerted assault on Neil Kinnock turned what looked like a dead cert for Labour into a surprise win for the Tories’ John Major.

Who is Andrew Cooper?  He was the chief pollster for the “Remain” campaign. The man whose assumptions, about how many Britons would turn out to vote in the European Union Membership Referendum of 23 June 2016, lulled Prime Minister David Cameron into a false sense of security. Allowing the “Leave” campaign’s chief wizard, Dominic Cummings, to steal a victory his enemies had, quite literally, not seen coming.

That’s the thing about polls. It’s not simply a matter of assembling the data: whether by calling landlines, mobile phones, or, acquiring people’s opinions online; it is also about how you go about interpreting the data. What are the assumptions you make about how the hundreds, or thousands, of people you have interviewed will behave when the time comes to cast their votes – or not.

That was Andrew Cooper’s big mistake. According to Tim Shipman, author of “All Out War: The Full Story of How Brexit Sunk Britain’s Political Class”, Cooper assumed that the same percentage of Britons would sit out the 2016 Brexit Referendum as had sat out the General Election of the year before. Take these ‘can’t be arsed’ voters out of the equation, and – Bingo! – “Remain” emerges as the referendum winner every time.

He was, as well all know now, quite wrong in that assumption. The turnout for the Referendum was well up on the 2015 General Election. Cumming had successfully mobilised the forgotten, the marginalised, and the very-bloody-angry. The Britons who had given up voting, because, after Tony Blair, the largely indistinguishable politicians of the Tory and Labour parties always won. This time, however, said Cumming, their votes would count. This time, they could deliver an almighty “Fuck You!” to the entire British Establishment, and there would be nothing – absolutely nothing – they could do about it. “Take Back Control”, bellowed Cumming’s inspired slogan. And, they did.

It is something like this. Something in the interpretation of the raw data. Something about the assumptions they’re making, that explains how Reid Research and Colmar Brunton were able to come up with such wildly different descriptions of New Zealand’s current political landscape. And that ‘something’ is very important. With so few publicly available opinion polls being conducted in this country, we really do need to know how one reputable polling agency can put National on 46 percent, while the other puts it on 37 percent. Nine percentage points is a hell of a gap – well beyond the usual statistical tolerances. The public is entitled to know what happened – and which one of them got it right!

But, is that even possible anymore? Getting it right? After Brexit, Trump, the British snap-election of June 2017, and the Australian Federal Election of May 2019, can “The Polls” be trusted to tell us anything at all reliable about the voting intentions of the public?

Is it possible that opinion polling, like black-and-white television, was something peculiar to a particular historical era? To a time when everybody had a landline telephone; read the same newspapers; watched the same shows, at the same time, on television; and were limited to a choice between two very large and broad-based political parties?

Back then, when people still had a huge amount of faith in the political system, polling struck many people as a welcome, and highly scientific, addition to the process of determining their electoral preferences. When a pollster called, the person picking up the phone felt under a heavy obligation to co-operate fully. To refuse, was tantamount of declaring oneself a bad citizen. Answering the pollster’s questions honestly and comprehensively was the randomly chosen interviewee’s civic duty.

How things have changed. Fewer and fewer voters have landlines. As a consequence, the phonebook is no longer sufficient to the task of preparing a statistically valid survey. And, unfortunately, there is no handy, publicly available, register for cell-phone subscribers. Even if there was, the dutiful citizenry required to make the survey truly representative has largely disappeared.

Because, while the pollsters were learning every little quirk about the voting (and buying) public, the public was also learning a thing or two about polling. What it was for; how it was being used; and by whom. With painful slowness, more and more voters realised they were being used. That what they were participating in wasn’t a matter of civic duty. It was a marketing exercise, impure and simple. They were helping the politicians to manipulate the electorate – i.e. themselves. Clever “communications specialists” were persuading them to buy back their own opinions.

This is the reason why polling agencies have so much difficulty in persuading the people they contact to participate in their surveys. For every one person who agrees to be interviewed, there are dozens who refuse. So many, that in statistical terms, the participating subjects have become unusual in themselves, and their opinions have to be weighted accordingly. Then, there are the respondents who deliberately mislead their interviewers. The “rat-fuckees” who become “rat-fuckers”.

Confronted with these formidable obstacles, opinion polling is becoming less and less a science, and more and more of an art. The trick now lies in assessing how much of the sample to write-off as tricksters; how many as atypical political junkies; how many as persons who enjoy being asked for their opinions, but who will never make it off the sofa on election day. These questions cannot be answered scientifically, they can only be guessed at. Those guesses will be guided by experience; by ex post facto analyses; and, crucially, by intuition.

One pollster will say: “Oh sure, people love Jacinda – but will they turn that love into a bona fide ballot paper stuffed into a ballot-box? Young people may revel in Jacinda’s stardust, but its old people who vote. We need to be conservative in our assumptions about the relative weighting of these two groups.”

Another pollster might say: “Normally, I’d apply a pretty heavy discount to the under-25 preferences. But, you know what? I’m betting that, in 2020, the under-25 vote is going to spike. I’m looking at all those secondary-school kids on the street demanding action on climate change. I’m thinking about the influence they’re going to have on their big brothers and sisters. Heck! The effect they will have on their parents! That’s why I’m assuming a significantly higher turnout.”

The first pollster might predict a solid vote for National. The second, a game-changing surge towards Labour and the Greens. They both can’t be right – but I’m hoping that more than 50 percent of voters would like to see the second prediction proved correct.

This essay was originally posted on The Daily Blog of Tuesday, 11 June 2019.