A scientific panel is questioning a method used by the FBI to match bullets from crime scenes, a finding that could give defense lawyers a new route to attack prosecution evidence.
In seeking to tie bullets from a crime scene to others found in a suspect’s possession, the FBI analyzes the lead for traces of seven other metals, a system that the report from the National Research Council said was sound.
However, the study questioned a statistical analysis method known as chaining in which trace elements in a series of bullets in a box. are compared. It noted that the bullets sold together in one package are not necessarily all from the same batch of melted lead.
Questions about the bullet analysis technique were raised in a Los Angeles Times investigation a year ago. The existence of the panel's study, which includes a series of recommendations, was reported by the Times and The Associated Press in November.
In addition to improving the science in comparing bullets, “how those findings are conveyed in court and to a jury remains a critical issue,” said Kenneth O. MacFadden, an independent consultant in research and analytical management based in Chestertown, Md.
Limits should be explained
MacFadden, chairman of the committee that prepared the study, said he considered the most urgent recommendation in the report is to have FBI witnesses in criminal cases more clearly explain the limits of bullet testing procedures.
FBI officials did not immediately respond to a request for comment on the report.
Jack King, a spokesman for the National Association of Criminal Defense Lawyers, said the report “will affect cases on direct appeal, I believe.”
“They cannot match lead samples like fingerprints,” but when an FBI scientist gets on the stand, it gets more credibility from the jury than it deserves, King said.
How chaining works — and doesn't work
Analysis of the metal content of bullets is used when bullet fragments are too small or damaged to compare the marks left on the slug by the barrel of the firearm. The goal is to determine if the bullet from the crime matches other bullets found in the suspect’s possession or weapon.
In chaining, researchers compare the amounts of trace elements in a series of bullets in a box. MacFadden said it’s like saying bullet A is like bullet B and B is like C and C is like D and so on, and then concluding that means A is the same as E because they are part of the same chain.
The committee said this can lead to an artificially large group of bullets that are considered identical, “when this would not be true if other statistical methods were used.” The bureau told the committee that it no longer uses chaining.
The FBI procedure could result in a high-false positive rate but reduces the chances of a false negative, explained committee member Karen Kafadar of the University of Colorado.
The method recommended by the committee would allow the lab to calculate the rate of false positives and negatives, she said. That information would help jurors better evaluate testimony, added MacFadden.
'Quality assurance must be improved'
The overall concept of analyzing the set of trace elements is sound, the committee said, but “the FBI Laboratory’s practices in quality assurance must be improved significantly to ensure the validity of it’s results.”
The rate of lab error is unknown at the FBI, the committee said, because the agency does not have a program of testing its examiners by an external agency and “its internal program does not appear to be designed to determine an error rate.”
In addition, the committee noted that while all bullets from a large batch of lead may be identical, that doesn’t mean that all the bullets in one box being sold came from the same batch, since bullets from various batches may be mixed together.
“The available data do not support any statement that a crime bullet came from, or is likely to have come from, a particular box of ammunition,” the committee concluded.
The findings are the latest in a string of controversies and embarrassments to hit the FBI lab, which pledged to remake itself after a scandal in the 1990s over bad science.
The study was requested by the FBI. The National Research Council is the principal operating arm of the National Academy of Sciences, a private institution chartered by Congress to advise the government on scientific matters.