THURSDAY, Jan. 18, 2007 (HealthDay News) -- Many cancer studies that rely on what scientists call genetic microarrays have critical flaws in their analyses or their conclusions.
This means doctors are taking this flawed research and using it as the basis of treatment for cancer patients -- treatments that may adversely affect patient outcomes.
That's the surprising conclusion of a new study by researchers at the U.S. National Cancer Institute that's published in the Jan. 17 issue of the Journal of the National Cancer Institute.
Microarrays are a tool used to study gene expression, or production. Using microarrays, researchers can study thousands of genes at a time, all on a single glass slide. In cancer research, microarrays are used to study the unique gene pattern of specific tumors, to find new drug targets, and to categorize the characteristics of a patient's tumor to help tailor an individual's treatment.
But, these studies based on microarrays produce vast amounts of data that are easily misinterpreted, the researchers say. Much of the problem owes to a lack of communication between clinicians and the statisticians who analyze the data, according to the new study.
"A lot of the publications trying to tie gene expression to clinical outcomes are flawed," said study co-author Richard M. Simon, chief of the National Cancer Institute's Biometric Research Branch.
It's difficult to analyze a readout where you get 20,000 to 30,000 gene variables, Simon said. "Properly analyzing that data to predict outcomes for patients is difficult," he said.
The genetic technology is very powerful, Simon added. "There are great success stories in being able to use gene-expression technology for being able to figure out which patients respond to what treatment. But there is a need for improvement in the analysis of the data and a close interdisciplinary collaboration with statistical experts in the analysis of these studies," he said.
In the study, Simon and his NCI colleague Dr. Alain Dupuy looked at 90 studies published through the end of 2004 that compared microarray profiling with medical results. The most frequently studied cancers were blood malignancies (24 studies), lung tumors (12 studies), and breast cancer (12 studies).
Simon and Dupuy then specifically looked at the statistical methods and reporting in 42 studies published in 2004. Half of these efforts had at least one basic error. In the 23 studies with an outcome-related gene finding, nine involved inadequate, confusing, or unstated methods to account for false-positive results, Simon and Dupuy found.
In 13 studies, there were unsupported claims of meaningful classifications of results, where the authors did not do adequate analyses to reach their conclusions. In addition, in the 28 studies that predicted outcomes, 12 used biased estimates of the accuracy of their predictions, according to Simon and Dupuy.
To solve the problem, Simon believes there needs to be better collaboration between biologists, doctors and statisticians. "There are still some glitches in that interdisciplinary collaboration," he said. "We need more involvement of statisticians."
One expert believes the study points out a serious problem in cancer research and its application to patient care.
"This paper is surprising and a cause for concern," said Dr. Len Lichtenfeld, deputy chief medical officer at the American Cancer Society.
Genetic data has significant implications for how cancer treatment is designed, Lichtenfeld said. "There are gene tests currently available, for example, that try to help women decide whether or not they need adjuvant chemotherapy for breast cancer. And those studies are being used by doctors today," he said.
Lichtenfeld noted that not many doctors are well versed in how to evaluate the data generated by genetic-based cancer studies. "We are not familiar with all the elements of that analysis. We rely on the authors, the reviewers of the papers and the journals themselves to present information that is accurate," he said.
But, it appears that even the best medical journals are letting incorrect interpretations of the data be published, Lichtenfeld said.
"This research relies on very sophisticated statistical techniques," he said. "Step one is to make sure these techniques are applied appropriately. But it's going to take some more work to make sure the interpretation of the data is correct and that the right message gets transmitted from the study," he said. "It's very important that we have consistency in the evaluation of the data."
More information
The National Center for Biotechnology Information can tell you more about microarrays.
