Saturday, January 12, 2008

What can go wrong?

What can go wrong when pundits and pollsters report on elections? Everything, as the recent NH primary contest shows. There have been plenty of attempts to explain what went wrong, starting with questions about the poll's sampling frame. Just like the Dewey-Truman debacle nearly 60 years ago, phone polling biases may account for the bad data. In this case, pollsters may have failed to take into account the large number of "cell-phone only" respondents who skew younger and less conservative than those with land-line phones (see link below). Also, the early date of the primary meant that college students were still on break--perhaps out of state and less likely to vote.

But the real issue here for those of us who study the mass media is how did the news networks allow themselves to stumble once again in the rush to get it first? Yes, timeliness is a news value...but it should never trump accuracy. As former NBC News anchor Tom Brokaw said, perhaps the news organizations ought to let the voters decide, then report on that!

If you're interested in the polling issues related to cell-phone users, check out this report from Pew Research Center for People and the Press.

12 comments:

Anonymous said...

First, not everything went wrong. Obamas projected turnout was accurate and the margin of Hillary's victory narrow. The Republican numbers too were accurate. So what exactly is your gripe?

Of course the various commentators and journalists from all the networks embarrassed themselves by projecting the wrong winner, but it was the unexpected turnout, which was likely due to several factors as you're aware.

Your claim that timeliness trumped accuracy in this case is disingenous and indicative of an unoriginal perspective of what happened with the polling.

The show of emotion from Hillary (whether genuine or not)and the fact that no polls were conducted that Monday before the election explains, at least in part, the unexpected outcome. All of it amounts to speculation anyhow.

It's human nature to try and predict things. It's ok to say that we sometimes get it wrong. Don't be in such a rush to throw out the baby with the bathwater. Polling is simply a more measurable manner of guaging intuition.

Polling succeeds far more often than it fails and you better than anyone should know that polling methods have refined and evolved greatly since the days of the Dewey-Truman debacle. This wasn't even a fair comparison for you to draw upon.

If you've developed a more efficient and accurate way of polling - present that for discussion.

pueblosam said...

Okay, I'll bite. My gripe is that the news media are reporting polling data that is of questionable accuracy and for what purpose? Why is it so important to know which way the polls are leaning before the actual election or caucus? Why not let people make up their minds and cast their votes, and THEN report the outcome? Reporting polling numbers simply muddies the water and gives people an excuse to stay home. You refer to "unexpected turnout" but it was only unexpected in the sense that pollsters and pundits misjudged the political climate. You said it yourself, "all of it amounts to speculation."

You're right, polling has evolved and it has become much more accurate and useful. But I'm still not convinced that reporting polling data before an election serves any useful purpose...other than to help fill up the 24/7 news cycle.

Anonymous said...

Hook, Line and Sinker…
You ask, “Why is it so important to know which way the polls are leaning before the actual election or caucus?”
I refuse to believe that you and Tom Brokaw are that naïve. I’ll bet you can find footage of Mr. Brokaw regurgitating any and all manner of polls taken before during and after the fact of many past elections. Be careful whose coat tails you ride.
(Answer) That’s like asking children to not shake the unbareably enticing and intriguingly wrapped Christmas presents lying under the tree before the big day. The shaking may just give clue to whether they’re getting the same boring pair of socks they got last year (or last election as it were) or the new XBOX 360 they asked for from Santa.
It’s like watching your favorite football team and knowing all of their stats and because of that careful scrutiny feeling confident on betting some big money on the Super Bowl.
The same holds true for secretively opinionated adults who seek to confirm or affirm their predictions regarding the political climate in this country. It’s really that simple.
You’ve forgotten the fact that it makes for fascinating television, ESPECIALLY when they get it wrong, or when an unexplainable phenomenon occurs. That’s why journalism and political commentary is and always has been a business professor. News coverage must be active, for to be passive would mean that the next network would stumble onto the story first.
What should be plainly obvious is everyone actively concerned with politics ponder a host of scenarios either through outward discourse with colleagues or inward thoughtfulness. And guess what, they watch the news to see if others agree.
You will claim that a bad poll every now and then disenfranchises viewers and voters thus rendering them permanent appendages to their couches. I’ll play devils advocate and say that it motivates those sluggish types who receive a sudden jolt of adrenaline when they see candidates they were initially unsure of getting pummeled by the pundits with whom they disagree.
You claim, “Reporting polling numbers simply muddies the water and gives people an excuse to stay home.”
Well then, prove your claim sir. Obama’s turnout was predicted accurately. The Republican polls were spot on. In other words, they got the numbers for everyone else but Clinton right. Clear as day sir. They measured the political climate just fine as far as they went. In this case it was a premature forecast, that which ended the Sunday before the Tuesday vote.
Your question should be: What happened Monday that influenced the increased turnout on Tuesday?
Maybe you and Mr. Brokaw should go back to New Hampshire and formulate some poll questions to find out what in fact influenced the sudden surge in votes for Hillary. Isn’t Brokaw sitting on the sidelines these days anyway…lmao
"all of it amounts to speculation." Yes Professor, everything said by the media about why ONLY the winner was incorrectly predicted is entirely speculation. The task, then, is to measure that speculation with research maybe using polls, perhaps? Or are you suggesting that we stop using polls for research.

Anonymous said...

I think that there was many factors for the results of the election. Some people tried to bring up race but I believe that played no factor in the polling because if someone wanted to vote for Hillary it would not ever become a race issue in a primary..it only becomes a factor in the general election. Plus I hate to say it but people from Vermont and New Hampshere don't have many blacks around them to be prejudice...lol

The polling was off because of the high number of independents in New Hampshire, the way the media had already given Barack the victory, and unfortunately the powerful clinton machine. If you look at the polling, Obama got the percentage he was supposed to get but some how Hillary was able to snatch up the votes from Biden and Dodd. She was also able to get the votes from older women...Her fake emotion which kept getting played may have pissed off soem women to go and help her plus some of the people who were undecided on there choice. I'm actually not dissapointed that he lost because it was three points and it put the bulls eye back on clinton. If he would have won it would have basicly ended the primaries in S.C. which would have given the GOP smear machine too much time to swift boat Obama. This gives more opportunites for other states to way in on the candidates and gives Barack more time to spread his message and get experience.

The GOP is fractured and it's funny too see all of the candidates steal Barack's message of change then call each other liberals during a debate when that's who they're trying to copy. These guys are still trying to get people to vote for them on fear when at the same time people are worried about keeping their jobs and homes. These guys need to wake up and actually talk to people instead of listening to their contributers. As afr as swift boating Obama it's not going to work this time because his personal life is already out in the open...I had some clown at wok try to tell me that Castro endorses Obama....I told him so what it's not like they communicate on a regular basis and he's taking money. They also tried to say tht a Kerry endorsement could hurt him......I told them I'd rather get a Kerry endorsement then one from Pat Roberts....they were'nt too happy. Unlike Kerry he'll have to fight to defend himself....Kerry mentioned that on Face the Nation today...he said he made a mistake by not defending himself and that he would do anything in his power to defend Barack if they try and do it to him. That's another reason why I think he endorsed Barack...Hillary and Bill didn't try to help him win because she knew if he won she'd have to wait 8 years for her turn.

Anonymous said...

Thanks for your insight.
It makes a lot of sense, and I wonder that if there were an extra day of polling instead of the Monday layoff the media would have better gauged the closenes of the race on the democratic side while perhaps eluding to some of the details you mentioned.

I'm not sure we can really know why Clinton may have picked up the votes from Biden and Dodd, but I'm remembering now that she was always the frontrunner in that state prior to the vote.

Isn't it true Iowa winners get on average a ten point bump heading into New Hampshire? That would explain the numbers up to Sunday. But the Monday void of survey data, and the Obama awe stricken commentators who beat Hillary to death over her choked up response to the audience member's question seems to have affected things. So what's left?

The polling info was correct just after Iowa showing Clinton in the lead but Barack closing fast. The last polls for Sunday proved true, yet Hillary still took the win and with a narrower margin of victory than earlier predicted.

The media made the mistake of assuming that how polls were trending would continue through Tuesday. Seems simple enough, and I'll bet they'll try to poll up to the minute in Nevada.

icedragon472005 said...

I think this is a good example of what happens with false advertising. The headline says that Dewey defeated Truman, which we all know that Truman defeated Dewey. This is also an example of libel, which is written information that is false and misleading. This is a bad blooper in newspapers, which are used as a channel to send out information across the nation.

Posted by Jessica Brown

Anonymous said...

[I think this is a good example of what happens with false advertising.]

What was advertised in this case?

Nothing! no news paper came out with a headline on their front page claiming a false winner.

Instead, talking heads hyped a winner based on a short sided forecast. The pundits embarrassed themselves and demonstrated a bias, not of polling, but of their misconceived opinions.

The people got it wrong here, not the research data itself.

One might be able to make a case that what happened in the Dewey/Truman case, but if your trying to make that case here and now regarding this particular blunder -- you'd be laughed outta court.

Be careful not to confuse apples with oranges.

Anonymous said...

No libel. not even close

marcellus said...

I think that this is good example of bad advertising. giving false information is a big problem. it says dewey defeated truman, that is false information and is a example of libel. they should send false news across the nation like that.

Anonymous said...

The problem is that most of the polling information that is reported is questionable. No one really knows if that information is accurate enough to take a stance. Yet why are they even commenting or elaborating on these polls before the citizens have even casted their votes? Why don't they wait until people have voted before they start issuing news of the poll standing. Polling has evolved very much. Yet I think it is still the wrong thing to do when they write of the polls and their inaccurate information being placed in the media. The polling information really has no relevance until after the final election except to fill the daily news spots.

Aaron Lewis

Anonymous said...

see when things are advertised wrong this can happen. those that don't know the truth are getting a misconception of it all. the heading was backwards and that in it's self causes controversy. leading people to believe that its one way when indeed its another. .
-J.payne

Anonymous said...

I completely agree with the statement that accuracy should not be trumped by timeliness. Just starting out in the journalism filed, I could not believe the carelessness of the major news networks.
My experience with phone polling is that the recipient of the call will pretty much agree with anything to end the phone call. People would rather get off the phone than really think about who they are voting for, especially the younger demographic. Because there is no immediate consequences for who they say they are voting for over the phone, it is not as accurate as the actual poll results. But in the end, accurate results from the ACTUAL polls should be certain before reported.

Megan Moran
MCCNM 101 9:30