Study Identifies Common Flaws In Oncology Microarray Studies

Microarrays are a tool used to study gene expression. Researchers can study thousands of genes at a time, all on a single glass slide. In oncology, scientists have used microarrays to study unique gene expression patterns of specific tumor types, to discover new drug targets, and to categorize unique characteristics of a particular tumor to help doctors tailor treatments to an individual patient. However, such studies produce volumes of data that is easily misinterpreted. It has been difficult to replicate such studies, which is considered the best way to validate scientific findings.

To study the statistical methods used in cancer-focused microarray studies, Alain Dupuy, M.D., and Richard M. Simon, D.Sc., of the National Cancer Institute in Bethesda, Md., reviewed 90 studies published through the end of 2004 that related microarray expression profiling to clinical outcome. The most common cancers in those studies were hematologic malignancies (24 studies), lung cancer (12 studies), and breast cancer (12 studies). The studies fell into three general categories: an outcome-related gene finding, such as searching for specific genes that are expressed differently in people who have a good versus bad prognosis; a class discovery, where researchers cluster together tumors with similar gene expression profiles; and supervised prediction, in which the gene expression profiles are used to generate an algorithm or set of rules that will predict clinical outcomes for patients based on their individual gene expression profiles.

The authors closely scrutinized the statistical methods and reporting in 42 studies published in 2004. Half of these studies (21) contained at least one basic flaw. In the 23 studies with an outcome-related gene finding, nine of them had inadequate, unclear, or unstated methods to take into account false-positive findings. In 13 of the 28 studies focused on class discovery, there were spurious claims of meaningful classifications of outcomes, in which the authors did not perform adequate analyses to reach their conclusions. Among the 28 studies reporting supervised prediction, Dupuy and Simon found that 12 of those studies used biased estimates of the accuracy of their predictions.

"...Microarray studies are a fast-growing area for both basic and clinical research with an exponentially growing number of publications," the authors write. "As demonstrated by our results, common mistakes and misunderstandings are pervasive in studies published in good-quality, peer-reviewed journals." To avoid such errors, Dupuy and Simon provide guidelines in the form of a list of "Do's and Don'ts" for researchers. "We believe that following these guidelines should substantially improve the quality of analysis and reporting of microarray investigations," the authors write.

Citation: Dupuy A, Simon RM. Critical review of published microarray studies for cancer outcome and guidelines on statistical analysis and reporting. J Natl Cancer Inst 2006; 99:148-58.

Note: The Journal of the National Cancer Institute is published by Oxford University Press and is not affiliated with the National Cancer Institute. Attribution to the Journal of the National Cancer Institute is requested in all news coverage. Visit the Journal online at http://jnci.oxfordjournals.org/.