How accurate are election polls?
The short answer
- Since 1936, final national polls for U.S. president have missed the gap between the top two candidates by an average of 4.4 percentage points, in one direction or the other. (AAPOR, 2017)
- A poll's margin of error covers only the chance of reaching an unusual sample of people; it leaves out other errors, such as guessing wrong about who will vote. (AAPOR, 2017)
- A poll is a snapshot of opinion at the time it was taken, not a forecast; predicting the result is a different job. (AAPOR, 2017)
What the studies found
- In 2016, final national polls missed the popular-vote margin by 2.2 points on average, less than half the 4.4-point average for national presidential polls since 1936. (AAPOR, 2017)
- State polls for president did worse in 2016, missing the margin by 5.1 points on average across 423 polls, compared with 3.2 to 4.6 points in the four elections before. (AAPOR, 2017)
- U.S. poll misses do not always favor the same party: polls underestimated Trump, the Republican, in 2016, but underestimated the Democratic candidates for president in 2000 and 2012. (AAPOR, 2017)
- In 2020, polls from the final two weeks missed the margin by 4.5 points on average nationally and 5.1 points in state presidential polls; the national error was the highest in 40 years. (AAPOR, 2021)
- In 2020 the misses leaned one way: polls made Joe Biden's lead over Donald Trump too big by 3.9 points nationally and by 4.3 points in state presidential polls. (AAPOR, 2021)
- In 2020 Senate and governor races combined, polls were on average 6.0 points too favorable to Democratic candidates on the margin, a bigger miss than in the presidential race. (AAPOR, 2021)
- In 2020, waiting did not help: polls from the last two weeks, the last week and even the last three days were off by about the same amount. (AAPOR, 2021)
- Bigger misses have happened before: national presidential polls in 1980 were off by 6.0 points on the margin. (AAPOR, 2021)
- In 4,221 U.S. state polls for president, Senate and governor from 1998 to 2014, all from the last three weeks, the typical miss on a candidate's two-party vote share was about 3.5 points, about twice what margins of error suggest. (Shirani-Mehr, Rothschild, Goel and Gelman, 2018)
- In that study, the usual 95 percent margin of error included the actual result for only 73 percent of Senate polls, 74 percent of governor polls and 88 percent of presidential polls. (Shirani-Mehr, Rothschild, Goel and Gelman, 2018)
- In that study, Senate and governor polls not uncommonly missed a candidate's two-party vote share by more than 5 points, which would rarely happen from sampling chance alone. (Shirani-Mehr, Rothschild, Goel and Gelman, 2018)
- The same study finds that polls of the same race often share a common error, about 2 points on average, partly because pollsters use similar ways of guessing who will vote. (Shirani-Mehr, Rothschild, Goel and Gelman, 2018)
- In a second set of U.S. state polls from 2004 to 2012, all from the last 100 days, that study finds early polls miss by more than polls near Election Day, though the error levels off in the final weeks. (Shirani-Mehr, Rothschild, Goel and Gelman, 2018)
- In 15 U.S. presidential elections, 1944 to 2000, daily national poll averages 200 days out missed the Democratic two-party vote share by 6.5 points on average, and 2.2 points at the end, which the authors call inflated. (Wlezien and Erikson, 2002)
- The same authors find polls grow more accurate as Election Day nears, with much of the gain coming in the fall campaign after Labor Day. (Wlezien and Erikson, 2002)
- A study of national polls in many countries from 1942 to 2017 finds that, contrary to common belief, the recent performance of polls has not been out of the ordinary. (Jennings and Wlezien, 2018)
- That study finds poll errors change in a regular way over the campaign calendar, and that how well polls do varies with a country's political setting in ways that make sense. (Jennings and Wlezien, 2018)
- A margin of error of plus or minus 3 points means that if the same survey were done 100 times, the result would be expected to land within 3 points of the true value 95 of those times. (Mercer, Pew Research Center, 2016)
- The margin of error for a lead, one candidate's share minus the other's, is about twice the one reported for each candidate, so a 3-point margin of error becomes about 6 points for the lead. (Mercer, Pew Research Center, 2016)
- The margin of error leaves out other errors, such as people who refuse to take polls and the hard job of figuring out who will actually vote. (AAPOR, 2017)
- In 2016 primary polls, which reached 636 people on average, the standard margin of error for one candidate's share would be plus or minus 4 percent, while the average miss on the winner's margin of victory was 9.3 points. (AAPOR, 2017)
- Pollsters often describe a poll as a snapshot in time that measures opinion when it was taken, while forecasting models try to predict a future event. (AAPOR, 2017)
What the evidence can't tell us
- Why the 2020 polls missed cannot be settled with the data available, because no one knows how the people who did not answer polls voted. (AAPOR, 2021)
- Studies measure misses in different ways: the 2016 AAPOR report uses the gap between the top two candidates, while a poll's reported margin of error applies to each candidate's share, not to the gap. (AAPOR, 2017)
Sources
Every study here was read in a free copy, linked below. A number on this page comes only from a study read in full; one read only in its summary is described in words.
- Courtney Kennedy, Mark Blumenthal, Scott Clement, Joshua D. Clinton, Claire Durand, Charles Franklin, Kyley McGeeney, Lee Miringoff, Kristen Olson, Doug Rivers, Lydia Saad, Evans Witt and Chris Wlezien (AAPOR Ad Hoc Committee on 2016 Election Polling) (2017). An Evaluation of 2016 Election Polls in the United States. American Association for Public Opinion Research (AAPOR) task force report, May 2017, free on aapor.org. Free copy
- Josh Clinton (chair) and 18 other members of the AAPOR Task Force on 2020 Pre-Election Polling (2021). Task Force on 2020 Pre-Election Polling: An Evaluation of the 2020 General Election Polls. American Association for Public Opinion Research (AAPOR) task force report, 2021, free on aapor.org. Free copy
- Houshmand Shirani-Mehr, David Rothschild, Sharad Goel and Andrew Gelman (2018). Disentangling Bias and Variance in Election Polls. Journal of the American Statistical Association 113(522); read as the published PDF posted on Sharad Goel's own site (the copy on Andrew Gelman's Columbia page shows no numbers in its text layer). Free copy · Publisher's page
- Will Jennings and Christopher Wlezien (2018). Election polling errors across time and space. Nature Human Behaviour 2, 276-283 (2018); abstract read on the publisher's page. Free copy · Publisher's page (summary only: no figures taken from it)
- Andrew Mercer, Pew Research Center (2016). 5 key things to know about the margin of error in election polls. Pew Research Center, Short Reads, September 8, 2016. Free copy
- Christopher Wlezien and Robert S. Erikson (2002). Pre-Election Patterns: The evolution of voter preferences. Public Perspective (Roper Center for Public Opinion Research), November/December 2002, pp. 30 to 34; the authors' short version of their 2002 Journal of Politics article on the timeline of presidential campaigns, read on the Roper Center's site. Free copy