Medill Survey Shows Most News Industry Workers Believe Polls are Overused and Unreliable

People working in U.S. media believe 2020 presidential election coverage was generally fair, but they also think polling was overplayed and unreliable, according to the first-ever Medill Media Industry Survey at Northwestern University.

The survey of nearly 1,400 members of the U.S. news media was conducted by Associate Professor Stephanie Edgerly of Northwestern University’s Medill School of Journalism, Media, Integrated Marketing Communications.

While it may not be surprising that members of the news media would give their own industry high marks for fairness, their concerns about political polling were eye-opening.

Asked about media coverage of polls, 63.1% either agreed or strongly agreed that there was too much coverage, while just 14% disagreed or strongly disagreed. On the question of polls’ accuracy, 56.3% agreed or strongly agreed that polls were an unreliable measure of public opinion, while just 20.1% disagreed or strongly disagreed.

“The narrative around polling has really turned negative and has been taken up by a lot of people who work in the industry,” said Edgerly. “People feel like there are too many polls. But are news organizations going to reduce their polling? Probably not.”

Tim Franklin, Medill senior associate dean and John M. Mutz Chair in local news, assisted Edgerly with the survey’s logistics. He said the news media must confront two key questions regarding polling: “Have polls become sort of the crack cocaine of election years for journalists? And have polls become a substitute for journalists actually going out and talking to real people?”

Medill shared the findings with two people who have a deep understanding of media polling, and both suspected the survey results might show a distaste for “horse-race” polling rather than polling about public policy.

Benjamin Toff, an assistant professor at the University of Minnesota who has written extensively on the news media and polling, said polls that gauge the electorate’s opinion on issues give the public “a voice in the democratic process” in a way that’s more valuable than just “telling you who’s going to win the next election.”

Toff also said polls may be getting the blame for the media’s over-all approach to coverage.

“It’s hard to separate out what’s the polls and what’s broader concerns about the way politics is covered often through a strategy frame or a game frame – everything is through the lens of who’s winning and who’s losing,” Toff said. “I think there’s a general frustration not only among the public but among journalists themselves that it often ends up being the way politics gets covered.”

Frank Mungeam, the Local Media Association’s chief innovation officer, has familiarity with political polling in his former roles as a Knight professor of practice in newsroom innovation at Arizona State University and as vice president for digital audience for the Tegna television chain. Mungeam said journalists’ loose interpretation of horse-race polling has hurt the reputation of public opinion surveys.

“There can be a lack of rigor in how polls get reported that can lead to a lack of faith in polling,” Mungeam said. “And it isn’t that the polling itself is incorrect. It’s that journalists can do a better job of contextualizing who was polled and what was the margin of error.”

The news media have made some improvements recently, Mungeam said.

“I thought I saw in 2020 some learning by the media collectively,” he said. “I saw a lot more contextualizing of horse-race polling information than I did leading up to 2016. … I saw a lot of relevant use of issues polling.”

Biden won the popular vote by about 4.5 percentage points, compared to the 10-point edge for Biden in CNN’s final “Polls of Polls” and the 8.4-point margin for Biden in FiveThirtyEight’s final poll aggregate. State polls also varied markedly from the outcome. For example, Florida went to Trump by a comfortable margin despite polls showing Biden up slightly, and Biden won Wisconsin by a far narrower margin than polling suggested.

Respondents in the Medill survey accepted the view that polls influence both media coverage and voters’ choices. Presented with the statement that polls drive media coverage, 83.5% agreed or strongly agreed while just 6.2% disagreed or strongly disagreed. Given the statement that poll coverage can influence voting behavior, 64.3% agreed or strongly agreed while 11.8% disagreed or strongly disagreed.

The University of Minnesota’s Toff said pollsters tend to resist the idea that their findings influence how people vote, but he doesn’t fully accept that view.

“If you talk to the pollsters who are doing this research they generally speaking … really push back on that and say that’s not what we’re doing, we don’t have an influence in that way, we’re just trying to give as systematically and rigorously as possible a portrait of what the public thinks at any given moment in time,” Toff said. “I will say there’s some evidence that in some cases under some circumstances, polls can have an effect on public opinion in that way.”

In political races where there are a lot of candidates, they can have an impact, he said.

“You see this especially in recent years in primaries where the polls are [the way] candidates can establish their viability and then the press starts to pay more attention to them,” Toff said. “… And it’s not just the press. It’s also voters and citizens who want to donate to campaigns are paying attention to that kind of data. There are a whole host of ways in which it has an indirect effect on the process.”

Members of the media polled by Medill rated the coverage of both the Trump and Biden campaigns as fair.

Asked to respond to the statement that the mainstream news media covered President Donald Trump’s campaign fairly, 71.4% agreed or strongly agreed, while 16.7% disagreed or strongly disagreed. As for Joseph Biden’s campaign, 73.3% agreed or strongly agreed that it was covered fairly while 10.5% disagreed or strongly disagreed.

The news media got even higher marks for fact-checking, though more respondents thought Trump got scrutiny than thought Biden did. Reacting to the statement that the media fact-checked Trump’s campaign, 88.7% agreed or strongly agreed, while 6.6% disagreed or strongly disagreed. As for Biden, 76.4% agreed or strongly agreed that his campaign was fact-checked, while 11% disagreed or strongly disagreed.

On a separate question about how tough reporters were on the presidential campaigns, the poll indicated that media workers thought Biden got off somewhat easier than Trump. Reacting to the statement that the media asked tough questions of the Trump campaign, 77% agreed or strongly agreed, while 15.3% disagreed or strongly disagreed. As for Biden, 54.5% agreed or strongly agreed that reporters grilled his campaign, while 23.8% disagreed or strongly disagreed.

Medill’s Edgerly was not surprised by that result. “Trump had more to answer for [because] he’s an incumbent,” she said.

Edgerly was impressed by the fact-checking numbers: “It shows the dominance of fact-checking in election coverage. Media workers, on the whole, felt good about the fact-checking of both campaigns.”

Franklin, who heads the Medill Local News Initiative, noted “an explosion of philanthropic giving to fact-checking organizations who because of the events of the last 4-5 years have really seen a need for it.”

“What this shows, from my perspective, is that the fact-checking movement and the necessity of fact-checking has really taken hold in the industry as a core public service,” Franklin said.

Respondents also were asked whether they had personally encountered disinformation about the 2020 presidential election, and if so how frequently. Only 2.3% said they had never experienced disinformation, while 13.1% had weekly, 33.6% had many times a week, and 51% had many times a day.

The Medill Media Industry Survey was conducted online from Nov. 30 to Dec. 28, 2020. A list of people working in U.S. media was compiled, then reduced to 15,000 by random sampling. Those 15,000 were invited to participate, and about 9 percent of that sample did so. The survey included managers and journalists from print, digital, TV, radio and magazine outlets. About 43% of the respondents were supervisors.

The Medill Local News Initiative will release more survey results during the week of Jan. 24, 2021.