After the fiasco of US election, do opinion polls matter any more?

The dust is settling on Donald Trump’s unlikely victory, but the question that will be asked for a long time is — how could everyone have got it so wrong? Simply put, the polls have failed.

The past decade has seen unprecedented levels of data adoption into businesses and newsrooms, which is in itself a great advancement.

However, these methods can go wrong and Trump’s march to the White House is the latest example.

The failed predictions this year suggest that the rush to exploit data has outstripped our ability to recognise its limits.

The global polling industry is in a crisis that is threatening its traditional methods, thanks to a group of elections and referendums that were unlike anything we have ever seen.

Who thought that Colombians would not want a peace deal?

Who thought that Britain would be crazy enough to leave the European Union? Donald Trump was a joke, until he was not.

The pertinent question here is: did we all believe that Hillary Clinton would win because of bad polling or did we ignore bad polls because we believed she would win?

Pollsters will be wrestling with that question for a long time.

People’s faith in polling, one of our most trusted tools for understanding voter perceptions and behaviour, has legitimately been shaken.

In Kenya specifically, the trust in objective polls is almost totally gone.

Our local polls have become a political tool that is only mentioned by various camps when they work in their favour.

CHANGING RATINGS

To restore trust in polls, I believe we need to ensure that all stakeholders understand them better.

The biggest question people often have about political polls is: why do they tend to swing in terms of the levels of support various candidates receive?

Simply put, this comes down to how many of a candidate’s supporters respond to pollsters.

A candidate’s ratings go up when more of his/her supporters answer a survey than the opponent’s supporters and vice versa.

Research shows that when things are going badly for a candidate, his/her supporters tend to stop participating in polls and become less vocal about their stance.

For example, given the constant fire-fighting that the Jubilee administration has had to do lately, there is widespread disdain for the government and any voiced support for Jubilee is drowned out by angry citizens, as we have witnessed on various social media platforms.

The effect this has is that Jubilee supporters in some areas are now less likely to respond to polls for fear of being judged or maligned.

Meanwhile, more opposition supporters will become enthusiastic about the election and respond to surveys.

In Trump’s case, the preceding weeks had been full of bad news for him, therefore his supporters were dejected and not participating in polls.

In short, polls were underestimating Trump’s strength among the electorate. They got Clinton right, but got Trump wrong.

If polling is still going to be a core part of voting processes, we will need to figure out how to avoid breakdowns like these in the future.

The arms race for data has meant that our understanding of it has drastically lagged behind the amount of data we are generating.

METHODOLOGY

A few pollsters who understood American politics and the state of the country at the time were able to predict a Trump victory using their data as far out as three months before the election.

They did this by simply realising that you were less likely to report being a Trump supporter in polls and adjusted for this.

Trump’s supporters were more enthusiastic about turnout and less enthusiastic about responding to polls.

There is a high likelihood that you would obtain the same result from Jubilee supporters in Kenya right now if you ran a poll to see how people would vote in 2017.

The fundamental truth here is that undecided voters do not exist.

No one is lying in the polls. Most people do not change their minds last minute.

The pollsters have been pointing fingers at the “sneaky” voters, but the truth is that they often misunderstand them because their methodology is wrong. Simple.

Mr Madung is the data science lead at Odipo Dev Ltd. [email protected]