Increasingly inaccurate: how smartphones and paranoia made polling more difficult

With a Biden and Trump rematch on the horizon, experts seek to increase polling accuracy amid intense scrutiny

With the US presidential election fast approaching, experts say polling firms are facing increased pressure and scrutiny to fine-tune their methods and ensure accuracy. Nick Donaldson / Getty Images

As the world witnesses a record number of elections, and with the US Presidential election fast-approaching, polling companies are under unprecedented pressure to find out why recent polls have been off the mark, according to Professor W Joseph Campbell.

“There's a lot of validity to the question about polling accuracy,” said Prof Campbell, a tenured professor at American University and author Gallup: Polling Failure in U.S. Presidential Elections.

While many point to the 2016 election of Republican Donald Trump, a surprise to many at the time, as one of the first indicators that traditional polling methods were no longer accurate or effective, Prof Campbell said the 2012 election was one of the first inklings of polling problems.

Incumbent US President Barack Obama defeated Republican challenger Mitt Romney by almost 4 percentage points.

“That was somewhat higher than the polls were anticipating,” he said, before pivoting to the 2016 race, where many polls indicated that Donald Trump would be defeated by democratic candidate Hillary Clinton.

American University Professor W Joseph Campbell says although polls and pollsters are under intense scrutiny, the criticisms they face aren't necessarily new.

“She won the popular vote of course, but that blue wall that was supposed to hold for her in Michigan, Wisconsin and Pennsylvania crumbled, quite unexpectedly compared to the polls at the time,” he said.

For the 2020 election, although many polls predicted a victory for democratic candidate Joe Biden, the margin of victory was not as wide as many of the polls projected.

“The pollsters are looking at all these instances and really trying to grapple with it,” Prof Campbell said. “No one really knows for sure why yet,” he said, referring to recent questions surrounding the accuracy of polling.

Polling postmortem

Although it flew under the radar in most circles, a report issued by the American Association for Public Opinion Research in 2021 sought to identify the potential problems in terms of accuracy with regard to the 2020 US presidential election polling, and for that matter, the 2016 presidential election cycle as well.

“The 2020 polls featured polling error of an unusual magnitude: It was the highest in 40 years for the national popular vote and the highest in at least 20 years for state-level estimates of the vote in presidential, senatorial, and gubernatorial contests,” read the report.

That report, in part, determined that there was a potential problem in the form of nonresponse from the US electorate when approached by pollsters before the election.

AAPOR's report also noted that many pollster's may have inadequately accounted for the influx of new voters who had not previously voted in US presidential elections.

“There were many new voters in 2020 and it is unclear whether the proportion of new voters in the polls matched the proportion of actual new voters,” it continued.

The lengthy report, clocking in at more than 100 pages, lists many potential scenarios, but ultimately concludes that nonresponse from voters before the election was likely a top reason for polling inaccuracy, among other factors.

Even that conclusion, however, has some frustrating ambiguity for pollsters seeking a quick fix.

“The primary source of polling error cannot be identified conclusively without knowing how non-respondents and respondents compare,” the report read.

Smartphones proliferate, landlines decline and nobody picks up the phone

The problem of addressing nonresponse from voters who pollsters badly want to speak with, according to Prof Campbell, stretches back several election cycles.

“How do you best reach people,” he said, noting that traditionally, polling had relied on calling landline telephones, which had quickly fallen by the wayside in the last decade.

The proliferation of smartphones, he added, is not necessarily a problem for pollsters, but in that same breath, a potentially bigger problem has come to fruition in that fewer people are actually picking up the phone.

“That limited response really has the potential to skew results,” he said.

Gallup, a prominent US polling and research firm, boasts on its website about its US polls, which include both landline and mobile phone numbers using random digit dialling (RDD) methods, while Quinnipiac University, another prominent polling entity, also boasts RDD along with a mix of mobile phone and landline outreach.

“If there is no answer, we will 'call back' that number. We will call every number where there is no answer at least four times,” read a methodology description from Quinnipiac. “We do call cell phones. This is increasingly important as more than half of the nation only have a cell phone and no landline telephone.”

In some instances, Prof Campbell said that occasionally polling companies and research organisations have begun to incorporate online polling and even text messaging to try to account for societal trends, but cautioned those approaches are not cure-alls to polling woes.

Following various US presidential elections where polls were off the mark, Prof Campbell wrote a book looking at the history of polling methods. Photo: Amazon

“Personally, if I got text message from a pollster, I'm inclined not to answer,” he said. “And I think I lot of people feel the same way.”

Another hypothesis in recent years, according to Prof Campbell, has pointed to the increased polarisation of politics, leading to voters overtly deciding not take part in polls amid increased paranoia and distrust in institutions.

Most recently, even President Biden expressed doubt about recent polls during an interview with ABC's George Stephanopoulus, echoing distrust in pollsters and the potential for the electorate to not take part in polling.

“George – look, you know polling better than anybody. Do you think polling data is as accurate as it used to be?” he said.

Conversely, Prof Campbell said there is also a theory that conservatives, and in particular, supporters of Trump, might be less likely to take part in polls for various reasons, leading to underrepresentation.

“It's something that pollsters have been discussing for a number of years,” he said. “But again, it remains a hypothesis, it's not proven.”

Recency bias

While it might be tempting to think that inaccurate polling is unique to recent election cycles, Prof Campbell said that notion is simply untrue.

Between 1936 and 2020, he said, there are nine separate cases in US elections where polling has been off the mark.

He said that one of the more prominent polling problems occurred in the lead up to the 1948 US presidential election, where democratic incumbent Harry Truman was projected by various pollsters to lose to the republican challenger Thomas Dewey.

Despite the projections, President Truman emerged victorious, defeating Mr Dewey by 4.5 percentage points.

Inaccurate polls led some to incorrectly project that incumbent democratic President Harry Truman would lose to republican challenger Thomas Dewey in the 1948 US presidential election. Photo: Harry S. Truman Library

“Polling was still relatively new,” said Prof Campbell, reflecting on the aftermath that caused pollsters to do research into what went wrong.

“Many of the pollsters stopped polling up to two months before that election,” he explained, adding that some of the pollsters did not expect the public's mood to change much given Mr Dewey's apparent popularity at the time.

That moment would be ensconced in the memories of many, with President Truman holding up a copy of a newspaper which prematurely printed that his challenger would be the next president.

Consequently, for the 1952 presidential election, Prof Campbell said some pollsters, perhaps trying to overcorrect for their previous inaccuracies, failed to project republican Dwight Eisenhower's landslide victory over democrat Adlai Stevenson.

“Make no mistake, polling has had some bad patches in the past,” he said. “We're in another patch right now.”

The future of polling

Despite all the difficulty faced by polling organisations when it comes to finding a method to the madness of reaching voters in an accurate way, Prof Campbell says there are reasons for optimism.

“Election polling is just a mere sliver of a multibillion dollar industry,” he said, noting the accuracy of other polling implementations ranging from consumer preference polling to issue-based polling and sophisticated analytics models. “It's not going to just disappear.”

Some pollsters, he said, have had relative success in recruiting panels of respondents with whom they check in on a regular basis, although that approach can also be flawed an labour intensive.

Regardless, he said the important thing is that pollsters and political scientists are not at all in denial about the increasingly inaccurate polls, pointing to AAPOR's 2021 polling report looking at how to increase accuracy going forward, knowing full well the increased scepticism from the general public.

“How many of these kinds of failures can you sustain consecutively and still remain credible?” he said.