IE 11 is not supported. For an optimal experience visit our site on another browser.

Confused? Study casts doubt on studies

A new study looks at scientific studies over the last 13 years and finds that you really shouldn't believe everything the doctors tell you. NBC's Robert Bazell reports.

For years studies suggested vitamin E protected the heart. New research says it doesn't.

"People taking high doses of vitamin E actually had an overall death rate up to 6 percent higher than those not taking the vitamin," Tom Brokaw reported on "NBC Nightly News" Nov. 10, 2004. 

And hormone replacement therapy (HRT) for women went from beneficial to dangerous.

"Women taking the HRT had a three-times greater risk of getting dangerous blood clots," I said on "NBC Nightly News" July 2, 2002.

Those are just two of the reasons that a new study published in the Journal of the American Medical Association looks at medical research itself. It examines 45 studies in three major medical journals that were widely cited by other doctors and scientists over a 13-year period.

It found that 16 percent of the time the results were contradicted. They turned out to be wrong. Only 44 percent of the time were the results confirmed or proven correct.

So why does medical science seem to change so much? We asked some of the top researchers at the Harvard School of Public Health. One day they say something is good for you. The next day, you find out it's bad. Why does that happen?

"Well, there are several reasons for the conflicting information," says Dr. Meir Stampfer. "One is that science evolves. And a lot of the results are tentative. And it's kind of hard to convey to the general public the degree of certainty with which our conclusions hold.”

Scientists can be certain that smoking is dangerous because there have been thousands of studies over many years. But when it comes to the benefits or dangers of a food group or a supplement, they can't know so well because there are usually far fewer studies

"Oftentimes, what people see is the very first study," says Harvard's Dr. Walter Willett. "Because that's what makes the news — is new, in fact. And that first study is often the most unreliable finding. It really becomes a reliable finding when it's been reproduced time and time again by other groups."

The experts say the best advice for the public — and the media — is to not to jump to sudden conclusions while science runs its course.