How the polls could be wrong (again) this year

Things just seem to go on – Democrats are finding hope in rosy-looking polls, but they are in for a rude awakening when the votes are tallied on election night.

In 2016, Trump’s victory shocked the world. In 2020, what appeared to be a democratic romp turned into a fingernail. And now, as the 2022 midterms draw closer, polls show Democrats doing surprisingly decently — pointing to a close election rather than the long-awaited GOP wave.

Unless, of course, the polls underestimate Republicans once again.

And lately there’s been debate among election analysts, including Nate Cohn of The New York Times and Nate Silver of FiveThirtyEight, about whether that’s exactly what we should expect this time around.

It’s always been a good idea to approach polls, poll averages, and election predictions with a healthy dose of skepticism. They are all good at getting us close to the result most of the time. But in each cycle, polls are often off by a few points on average, and they can miss a lot more in some races while on target in others.

So sure, polls can be wrong. The debate here revolves around a different question: Have polls so persistently underestimated Republican candidates of late that it’s just common sense to suspect this is happening again?

Or is the recent voting error harder to generalize, meaning we should be more reluctant to suspect bias against the GOP and perhaps Democrats shouldn’t be so concerned?

I think it makes sense for everyone around the world to be deeply skeptical of polls showing a large Democratic lead in states like Wisconsin and Ohio, where polls have consistently grossly overestimated Democrats across multiple election cycles. But the picture is less clear in other states, where voting error has not been as clear or consistent. I wouldn’t blindly “trust” these polls, but I wouldn’t assume they’re likely to be wrong either.

What was wrong with the polls?

The last cycle in which Democrats really felt the polls weren’t preparing them for disappointment was 2012. Polls that year have fluctuated somewhat, but they typically showed President Obama as the favorite for re-election, and forecasting models based on those polls did the same.

However, there was one dissenter – Dean Chambers, founder of the Unskewed Polls website. Chambers, a conservative, argued that most pollsters systematically undercounted Republican voters. So he rebalanced their results to reflect what he expected to be a stronger bias toward Romney—”to equalize them.”

Much derision followed from liberals about this rather crude method, and when the results of the election came out, Chambers got balls in his face – Obama and the Democrats actually did slightly better than the polls showed.

Here’s the fun part: In every election cycle since, Chambers would have had a point.

Andreas Prokop / Vox

First came the midterms of 2014, a year of the GOP wave. The final Senate polls correctly showed Republican takeover, but they underestimated the size of GOP wins in almost every competitive race by an average of nearly 6 points. National House polls showed a similar discrepancy.

In 2016 it was time again. National presidential and parliamentary polls came pretty close to the results, but polls underestimated Trump in most presidential swing states. Polls also underestimated GOP Senate candidates in competitive contests by an average of about 3 points.

Then, at the 2018 midterms, there was another discrepancy between the national House polls (which were reasonably accurate) and the competitive Senate state polls (which underestimated Republicans by an average of 2.5 points).

And in 2020, polls had their worst performance in decades because they narrowed Democrats’ margins at almost every level — in the presidential popular vote, presidential swing states, Senate swing states and the House of Representatives — by an average of nearly 5 percent Points significantly overestimated.

So in the last four cycles, the national polls have been fairly accurate twice and Republicans have been underestimated twice. But relevant for our purposes this year, polls for competing Senate elections have underestimated Republicans in all four election cycles. (And, of course, presidential-state polls twice underestimated Trump, although that’s more relevant for 2024.)

Why were the polls off?

A polling error averaging around 3 points is actually quite normal. All polls are an imprecise science that attempts to model the opinions of a large population based on a sample of a small portion of that population. In sampling (when certain voters are more difficult for the poller to reach) or in weighting (when pollsters try to make sure their sample is representative of the electorate, things might go wrong, they might make wrong assumptions about the odds, where this is likely to turn out to be the case). In addition, undecided voters who decide at the very last minute could disproportionately choose a candidate or side. These things happen!

But when polls consistently diverge in the same partisan direction, and often in the same states or regions, across multiple cycles, it may indicate an underlying problem.

Part of the recent debate among election analysts revolves around whether that actually happened — that is, how we should interpret these last few cycles of polling results. Has there been a consistent overestimation of Democrats — that is, a problem pollsters reached out to Trump-supporting Republicans? Or was it a more mixed set of results that caused people to miss patterns?

If you look at Senate polls on competitive contests from 2014-2020 and the 2016 and 2020 state presidential elections, the pattern of bias seems pretty clear: the poll underestimated Republicans far more often than Democrats in those contests, which are over several cycles extend this point. Often these errors were most pronounced in certain states or regions, e.g. B. in the rust belt or in very red states. So Cohn sees “warning signs” that recent polls may overestimate Democrats in the same states, an “artifact of persistent and unaddressed bias in polling research.”

Silver takes a broader view by including national polls, the governor’s race, and special and special elections in his analysis, and concludes that the picture looks more mixed. He argues that polls will support Democrats in various elections in 2017, 2021 and 2022 (particularly after the Dobbs Decision). He sees 2018 in particular as mixed and shows no “systematic democratic orientation”. And he posits that “Republicans may only benefit from higher turnout if Trump himself is on the ballot,” meaning 2016 and 2020 could be the wrong elections to focus on when thinking about this one year thinks.

A closer look at 2018

However, I have a different interpretation of poll performance in 2018 than Silver. According to his numbers, poll averages underestimated Democrats by about 1 point on average in the House and gubernatorial elections, and Senate polls averaged no partisan bias that year.

But there’s a catch: The Senate card had an unusually large number of contests in deep blue states this year, none of which proved competitive. The Democrats topped the polls in almost all of those contests.

However, if we look at the actual contested races of 2018 — which took place in purple and red states this year — most Democratic candidates did well in their polls, and often by quite a bit.

The final lead was more than 3 points worse for the Democrat than the final FiveThirtyEight poll averages in Florida, West Virginia, Michigan, Ohio, Tennessee, Missouri and Indiana. There was only one competitive state — Nevada — where the Democrat topped the polls by more than 3 points.

To find out which way the Senate would tip, the polls functionally underestimated Republicans again in 2018.

But here’s another caveat: 2022’s competitive Senate map looks nothing like 2018’s. That year, Democrats defended 10 seats in states Trump won two years earlier, including many deep red states (including North Dakota, Indiana and Missouri, where some of the biggest electoral errors lay). The 2014 competition map, another year in which polls significantly underestimated the GOP, was similarly red. But in 2022, the top Democratic spots to defend or take are in all-purple states that Biden narrowly won: Georgia, Nevada, Arizona, Pennsylvania and Wisconsin.

The trick to trying to learn lessons from history is that nothing will ever be the same. Every situation is new and will have similarities and differences to things that happened in the past. A comparison requires the selection of certain past events for study while omitting others. And the more past events you look at, the more conflicting evidence you’ll find.