The 2020 election presented another clear challenge – it took place in the middle of the pandemic. Pollsters have found that some Americans, stuck at home and lonely, are more likely to respond to surveys. Although this was initially seen as a good thing, it might have led to even more prejudice if it meant that the unequal adherence to stay-at-home orders added another source of prejudice to whoever picked up the phone.
Weighing in on a recalled vote is not without concerns.
Voters have been shown to have poor recall of who they voted for or even whether they voted at all, typically being more likely to remember voting for the winner. One study of Canadian voters found up to a quarter of voters were inconsistent when remembering who they voted for.
This distortion of past voting can push polls in different directions depending on who won the most recent election. In 2022, that meant respondents were more likely to say they supported Joe Biden, and pollsters using a recalled vote would end up giving them less weight, meaning Republican support was boosted.
But with a previous winner from a different party, the effect would be reversed. An assessment by The Times found that weighting its 2020 polls with a recall vote from 2016 would make them even more biased toward Mr. Biden. And a report of the American Association for Public Opinion Research, examining how 2020 polls could have been improved, found that polls that weighed in on the recall vote were no better than those that didn’t.
Similarly, in 2022, weighting by recall vote would have made the Times/Siena polls less accurate. As published, without recall vote weighting, the final polls of Senate, governor and House races had a median error of less than two percentage points and zero bias toward Democrats or Republicans. When weighted by recall vote to 2020 election results, average error would have increased by a percentage point, and overall the polls would have been slightly biased toward Republicans.
But that might have been a a consequence of other decisions The Times makes, which includes weighting demographic information available in the voter file that is not always available to other pollsters.
Other pollsters have found the method to provide significant improvements over typical weighting schemes. SSRS used a variety of weighting methods in 2022, including a recall vote for some of its polls, and also experimented with political identification weighting. Its post-election analysis found that using a recalled vote as a weight would have been the most accurate overall approach, increasing average accuracy by more than three percentage points over just weighting on standard demographics.