Time and again, critics of paleontologist Gregory Paul’s controversial research suggesting a correlation between religious belief and social dysfunction come back to his numbers: They don’t add up, these critics say, because he doesn’t understand statistics.
MSNBC.com asked three statistical and assessment experts to review the methodology Paul used in compiling his paper, which is published in the current issue of The Journal of Religion and Society under the title “Cross-National Correlations of Quantifiable Societal Health with Popular Religiosity and Secularism in the Prosperous Democracies.”
By and large, these experts say, the critics are wrong. Paul’s study holds up. But they also say the critics have a point.
Strictly speaking — and that is the only way academics speak — Paul’s study marshals reasonable data that credibly demonstrates, under its terms, that the United States is the most faithful of the 18 Western democracies it studies. It furthermore demonstrates that it scores at or near the bottom of a variety of indicators of social dysfunction, including homicide, juvenile and early adult mortality, teen pregnancy and abortion.
Correlation doesn’t equal causation
All the paper is saying, “in a variety of ways, is that the U.S. is different from other First World democracies in several respects,” said Dr. Joseph B. Kadane, University Professor of statistics at Carnegie Mellon University in Pittsburgh, who has been a consultant to the Justice Department.
That’s a crucial point, the experts said. All Paul is highlighting is a correlation. Nowhere does he explicitly argue that the United States’ higher rate of religiosity is a cause of the higher rate of social problems, the sin of which he is most commonly accused.
“The main caveat — and the author himself is clear on this — is that correlation is not causation,” said Dr. Larry Wasserman, a colleague of Kadane’s at Carnegie Mellon and winner of the 2005 Outstanding Statistical Application Award from the American Statistical Association. “His research suggests a correlation between religiosity and certain social ills, but he leaves open the question of whether religiosity causes these problems.”
On those terms, then, Paul’s paper “is basically sound,” Wasserman said in an e-mail analysis.
Reading between the lines
Kadane agreed, but he observed that, even though Paul draws no direct link, he plainly wouldn’t be upset if you drew it yourself. “Certainly, some of the discussion suggests that this author has one direction of causation in mind,” he said.
That appears to be the complaint of George H. Gallup Jr., the eminent pollster, who fired off a letter to the editor of The Times of London after it highlighted Paul’s paper in September.
Gallup, whose deep Christian faith is widely known, declined to talk further about his criticism, saying he was publishing a full explanation in the coming edition of the religion magazine Touchstone. But his research associate referred MSNBC.com to Tim Cupery, a sociologist of religion at the University of North Carolina, whose separate letter to The Times mirrored Gallup’s thoughts, the associate said.
“The main point of both the scholarly and journalistic articles seems to be a critique of Christianity as detrimental to the social health of the United States, a conclusion that the data do not warrant,” said the letter, which was never published.
Otherwise, Cupery said later in an interview, “why waste good research on this?”
‘Where ... is the actual data?’
Kadane also raised another red flag, saying, “There may be a critique that persuades me” that Paul is wrong — specifically, one that questions his data. “There are differences among nations and among different cities and regions about how these things are reported,” he said.
For example, criminologists and crime reporters have long griped about the unreliability of U.S. crime statistics, which leave the definitions of specific crimes up to the local authorities who report them — authorities who have an incentive to make themselves look good.
Likewise, other data could be unreliable, said Dr. Jennie Robinson Kloos, director of institutional research and planning and a specialist in assessment methodology at the College of St. Catherine in St. Paul, Minn.
“I would start by asking the researcher for more information — details that are not included in this article,” she said in a written analysis, which significantly dissented from the views of Kadane and Wasserman. “I believe that the biggest criticism about his methodology is simply that he doesn’t explain it.”
Paul reported that his data on religious belief came from two trans-national surveys by the International Social Survey Program. (The 1993 survey and the 1998 survey are both available online.) The data on social pathology come from much more diverse auspices, which Paul details only as “a variety of well-documented sources including the UN Development Programme (2000).”
“Where in the paper is the actual data?” Kloos asked. “He has left out the evidence entirely.”
A question of motive
That leads directly to the main complaint of the many religious commentators who have made Greg Paul a punching bag. We don’t know much about him, they say, and what little we do know shows he’s unqualified to do this sort of research.
It’s an argument that Paul — who described himself in an interview only as an “independent researcher in paleontology and evolution” — has no interest in answering. He declined to say where he went to school or what he studied, confirming only that he doesn’t hold a position at any academic institution.
It is a longstanding mystery. None of the highly praised scientific books he has written or edited — spanning more than 17 years for publishing concerns as respected as Simon & Schuster, the Johns Hopkins University Press and National Geographic — provide any information. The books’ biographical notes refer to him simply as a “freelance paleontologist and evolutionary scientist.” The Journal of Religion and Science didn’t identify him at all, beyond the address “Baltimore, Maryland.” (The journal’s editor didn’t reply to several requests for comment.)
There is no doubting, however, that Paul is a widely respected authority in dinosaur paleontology. His book “Dinosaurs of the Air,” published in 2002 by Johns Hopkins, advanced the idea that some of the most familiar dinosaurs had ancestors who could fly. It is a sometimes-controversial contention, but even dinosaur paleontologists who disagree with Paul praise his scholarship and rigor.
Indeed, for many years, Paul has been considered among the foremost illustrators of dinosaur anatomy in the world, inspiring countless scientists and artists alike. His renderings are so authoritative that they were adapted for the dinosaur designs in Steven Spielberg’s movie “Jurassic Park.”
That’s all well and good, say religious scholars, including the Rev. R. Albert Mohler Jr., president of Southern Baptist Theological Seminary, but what does Paul’s expertise in dinosaur paleontology have to do with the sociology of religion? Especially when he is promoted as a featured speaker by the Council for Secular Humanism? Especially when he is well-known as the co-author of a 1996 book, “Beyond Humanity,” which predicts an “Extraordinary Future” grounded in the imminent creation of artificial life?
Paul said such questions were beside the point. No one has persuasively challenged his raw data or what they demonstrate, he contended; if other scientists don’t believe him, they can do the research themselves. That’s how science works.
“No one’s shown what I did was wrong,” said Paul, who said he was planning to write a book on the subject. “They tend to go off on tangents.”