The news media love political polls. Unfortunately.
/
AI-GENERATED PHOTO BY STAKALO
UPDATE (Nov. 9, 2024): Results of the presidential election fell within the margin of error of most national and swing-state polls. But the deviation in real results went all in Trump’s favor. So the polls’ picture of a neck-and-neck race was statistically accurate but impressionistically wrong, underestimating Trump for the third election in a row.
Nearly everyone who wanted Joe Biden to end his presidential campaign pointed to political public opinion polls. They were one reason he did so. Supporters of Kamala Harris as his replacement cited political public opinion polls. Polls lately have fueled optimism among Harris supporters and panic and strategy changes within the campaign of Donald Trump.
Polls are influencing some really monumental decisions. Sure wish I had more confidence in them. And in how the media report them.
Polls remain the best method for quick and valuable assessment of public views, and they are, for the most part, accurate. But they were wrong in 2016. Although national polls in aggregate reflected Hillary Clinton’s popular vote margin, many state polls failed to reflect Trump’s level of support in some key states and therefore masked his chances of an Electoral College victory. That’s why you, me and most of the world were so stunned that night.
In 2020, polls got the right winner in Biden, but badly underestimated the strength of Trump and other Republican candidates.
“The polling industry is a wreck and should be blown up,” Politico wrote in November 2020. And a New York Post headline from the same month said: “The pollsters were wrong again – why do we listen to them?”
For the 2022 midterm elections, pollsters got bashed for predicting a Republican surge that never happened. On the other hand, FiveThirtyEight.com, a polling website that also rates the many polling organizations that exist, declared the 2022 polling cycle as the most accurate since 1998.
Polls can serve two distinct purposes: Assessing public sentiment at a particular moment and predicting an election outcome. James Stovall, a former UA journalism professor who co-directed Southern Opinion Research in the 1980s and 1990s (when I occasionally worked with him on political polls on behalf of The Birmingham News), said a “well-constructed” public opinion survey can effectively serve both purposes.
But not every survey meets that standard, he added, and the media and the public can easily misinterpret polling data. “Because there are so many public opinion polls being conducted and reported, it is not easy to say they are always ‘accurate’,” he wrote in an email.
He pointed out that all polls have some degree of statistical imprecision. That happens for various reasons. For one, the best polls use a truly random sample of the population (to increase chances that the demographics of the sample match the demographics of the whole population) and “getting a good random sample of the population is more difficult these days.”
The spread of cellphones has made people harder to reach and telemarketing has made people more reluctant to answer calls. The last time one of today’s young people answered a phone call was … never. And when surveyors do get people to answer, they increasingly don’t want to talk. Some current poll directors say that’s a problem especially among Republicans because many Republicans consider polls as part of the news media and they don’t like the news media. “Response rates have always made survey research difficult,” Stovall said.
News organizations report on polls endlessly and sometimes commission polls by a survey company. When presenting any poll, news stories need to include basic information that helps the public assess the worthiness of the poll: Who conducted it; sample size; how and when respondents were surveyed; margin of error; and how questions are phrased, which can (but shouldn’t) steer answers in a particular direction. “Unfortunately, many reporters do not understand these factors and their importance,” Stovall said.
Margin of error is a biggie. It’s an acknowledgement that the poll results based on a sample may not exactly match the results if the whole population were surveyed. If a poll says Harris leads by 2 percentage points and the poll’s margin of error is plus or minus 3 percentage points, then it’s possible that in reality Trump leads by 1 point. Or Harris leads by 5. News stories generally don’t explain margin of error well enough. Or even mention it.
According to a March 2023 article published on FiveThirtyEight.com, candidates leading polls by less than 3 points have won just 55 percent of the time. “In other words, races within 3 points in the polls are little better than toss-ups — something we’ve been shouting from the rooftops for years.” (That would make Trump vs. Harris a toss-up at the moment.)
Unexpectedly low or high election day turnout by particular demographic groups can also make predictive polls come out wrong. So can the simple fact that people change their minds. Sometimes events cause that, sometimes it’s just a change of heart at the last minute.
The constant headlines of poll results are a media staple of every election cycle. You just have to hope such “horse race” coverage doesn’t overwhelm attention to candidate policy and character, and that everyone understands the limits and variables of polling. And I say that even knowing that “electability” was a colossal question for the Democrats this year.
Stovall sees poll news stories as a popular part of campaign coverage. “It seems to be what the audience for political reporting wants to know about. And it certainly seems to be what political reporters and editors like to report.”
To dive deeper into the mechanics of polling, click here.
“None of these polls mean anything at all. They’re meaningless and yet, we keep looking at them. They’re like the Kardashians in a lot of ways. ”