Here are five questions to ask when analyzing political polls

Not all political polls are created equally so here are five key questions to ask when reading them.
Image: Voters Across The Country Head To The Polls For The Midterm Elections
Voters cast ballots in Kirkwood, Missouri, on Nov. 6, 2018.Scott Olson / Getty Images file

Breaking News Emails

Get breaking news alerts and special reports. The news and stories that matter, delivered weekday mornings.
SUBSCRIBE
By Mark Murray

WASHINGTON — Not all political polls are created equally. And different surveys sometimes show different results.

As the 2020 presidential race kicks into another gear — and with hundreds of different polls expected to be released between now and Election Day — here’s some helpful guidance when examining the horserace numbers.

1. Who conducted the poll?

Was it a reputable news organization? Or a pollster with past experience measuring public opinion in political races? Or none of the above?

The NBC News/Wall Street Journal poll, for example, is conducted jointly by two polling outfits – the Democratic firm Hart Research Associates and the Republican firm Public Opinion Strategies, both of which have a long history polling for political candidates and parties.

2. Do the findings confirm the conventional wisdom, or do they cut against the grain?

In August, a Monmouth University poll found Joe Biden, Elizabeth Warren and Bernie Sanders in a three-way tie for first place in the 2020 Democratic horserace – contrary to other polling showing Biden with a significant lead over his Democratic competitors.

Days later, however, additional polls contradicted the Monmouth finding, all showing Biden with a double-digit advantage.

“[I]t is clear that the Monmouth University Poll published Monday [Aug. 26] is an outlier,” said Patrick Murray, the Monmouth poll’s director. “This is a product of the uncertainty that is inherent in the polling process.”

Let our news meet your inbox. The news and stories that matters, delivered weekday mornings.

Still, the polling averages and conventional wisdom aren’t always right.

In 2016, for instance, the averages had Hillary Clinton narrowly leading Donald Trump in the crucial battleground states of Michigan, Pennsylvania and Wisconsin.

Trump ended up winning all three states.

3. Is it a national poll? Or a state poll?

As the 2016 election also taught us, the national popular vote doesn’t decide the presidency; the Electoral College does.

And so while Clinton won the popular vote by more than 2 percentage points — confirming the national polls that showed her ahead — Trump won the Electoral College by carrying Michigan, Pennsylvania and Wisconsin.

The same is also true for the presidential-nomination race: The national popular vote doesn’t decide who will face Trump in November 2020. Instead, the results (and delegates awarded) from individual state contests do.

4. What are the dates, sample size and margin of error?

Some polls show different results, because they were conducted during different timespans — and thus different events and news cycles.

Polls also have different samples and sample sizes. Was it conducted among 1,000 adults? A thousand registered voters? Five-hundred likely voters (those modeled as most likely to vote in an election)?

And the smaller the sample size, the larger the margin of error.

Also note that a margin of error in a horserace goes in both directions. So a poll’s margin of error of plus-minus 4.0 percentage points would require a candidate’s lead to be greater than 8 points to be considered outside the margin of error.

5. How was the poll conducted?

Maybe the biggest difference among polls is how they’re conducted. Are the questioners live human beings (which NBC News prefers)? Or are they automated voices (a methodological no-no for NBC)?

Respondents also are reached by different methods — landline phone, cellphone or online.

Gold-standard polls, like the NBC/WSJ survey, are live-caller polls with a large number of cellphone interviews.

Carrie Dann contributed.