Skip to main content

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

  • Opinion
  • Published:

Bias as a threat to the validity of cancer molecular-marker research

Nature Reviews Cancer volume 5, pages 142–149 (2005)Cite this article

Abstract

Claims that molecular markers can accurately diagnose cancer have recently been disputed; some prominent results have not been reproduced and bias has been proposed to explain the original observations. As new '-omics' fields are explored to assess molecular markers for cancer, bias will increasingly be recognized as the most important 'threat to validity' that must be addressed in the design, conduct and interpretation of such research.

This is a preview of subscription content, access via your institution

Access options

Buy this article

  • Purchase on SpringerLink
  • Instant access to full article PDF
Buy now

Prices may be subject to local taxes which are calculated during checkout

Figure 1: Research structure of an experiment.
Figure 2: Checking to see whether randomization was successful.
Figure 3: Guidelines for reporting studies of diagnostic accuracy.

Similar content being viewed by others

References

  1. Ransohoff, D. F. Developing molecular biomarkers for cancer. Science 299, 1679–1680 (2003).

    Article  CAS  PubMed  Google Scholar 

  2. Stears, R. L., Martinsky, T. & Schena, M. Trends in microarray analysis. Nature Med. 9, 140–145 (2003).

    Article  CAS  PubMed  Google Scholar 

  3. Petricoin, E. F. et al. Use of proteomic patterns in serum to identify ovarian cancer. Lancet 359, 572–577 (2002).

    Article  CAS  PubMed  Google Scholar 

  4. Zhu, W. et al. Detection of cancer-specific markers amid massive mass spectral data. Proc. Natl Acad. Sci. USA 100, 14666–14671 (2003).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  5. Li, J., Zhang, Z., Rosenzweig, J., Wang, Y. Y. & Chan, D. W. Proteomics and bioinformatics approaches for identification of serum biomarkers to detect breast cancer. Clin. Chem. 48, 1296–1304 (2002).

    CAS  PubMed  Google Scholar 

  6. Adam, B. L. et al. Serum protein fingerprinting coupled with a pattern-matching algorithm distinguishes prostate cancer from benign prostate hyperplasia and healthy men. Cancer Res. 62, 3609–3614 (2002).

    CAS  PubMed  Google Scholar 

  7. Petricoin, E. F. et al. Serum proteomic patterns for detection of prostate cancer. J. Natl Cancer Inst. 94, 1576–1578 (2002).

    Article  CAS  PubMed  Google Scholar 

  8. FDA Press Office. Protein patterns may identify ovarian cancer [online], <http://www.fda.gov/bbs/topics/NEWS/2002/NEW00797.html> (2002).

  9. Marcus, A. Testing for ovarian cancer is on the way. Wall Street Journal (New York) D1, D2 (1 Oct 2002).

    Google Scholar 

  10. Pollack, A. New cancer test stirs hope and concern. New York Times (New York) D1, D6 (3 Feb 2004).

    Google Scholar 

  11. Food and Drug Administration. Letter to correlogic systems, Inc. [online], <http://www.fda.gov/cdrh/oivd/letters/021804-correlogic.html> (2004).

  12. Food and Drug Administration. Letter to laboratory corporation of America [online], <http://www.fda.gov/cdrh/oivd/letters/030204-labcorp.html> (2004).

  13. Food and Drug Administration. Letter to quest diagnostics [online], <http://www.fda.gov/cdrh/oivd/letters/030204-quest.html> (2004).

  14. Society of Gynecologic Oncologists. Society of gynecologic oncologists statement regarding OvaCheck [online], <http://www.sgo.org/images/pdfs/policy/OvaCheck_statement.pdf> (2004).

  15. Wagner, L. A test before its time? FDA stalls distribution process of proteomic test. J. Natl Cancer Inst. 96, 500–501 (2004).

    Article  PubMed  Google Scholar 

  16. Garber, K. Debate rages over proteomic patterns. J. Natl Cancer Inst. 96, 816–818 (2004).

    Article  PubMed  Google Scholar 

  17. Check, E. Running before we can walk? Nature 429, 496–497 (2004).

    Article  CAS  PubMed  Google Scholar 

  18. Diamandis, E. Analysis of serum proteomic patterns for early cancer diagnosis: drawing attention to potential problems. J. Natl Cancer Inst. 96, 353–356 (2004).

    Article  PubMed  Google Scholar 

  19. Baggerly, K. A., Morris, J. S. & Coombes, K. R. Reproducibility of SELDI-TOF protein patterns in serum: comparing datasets from different experiments. Bioinformatics 20, 777–785 (2004).

    Article  CAS  PubMed  Google Scholar 

  20. Sorace, J. M. & Zhan, M. A data review and re-assessment of ovarian cancer serum proteomic profiling. BMC Bioinformatics 4, 24 (2003).

    Article  PubMed  PubMed Central  Google Scholar 

  21. Baggerly, K. A., Coombes, K. R. & Morris, J. S. Are the NCI/FDA ovarian proteomic data biased? A reply to 'producers and consumers'. Cancer Informatics (in the press).

  22. Baggerly, K. A., Edmonson, S. R., Morris, J. S. & Coombes, K. R. High-resolution serum proteomic patterns for ovarian cancer detection. Endo. Relat. Cancer 11, 583–584 (2004).

    Article  CAS  Google Scholar 

  23. van de Vijver, M. J. et al. A gene-expression signature as a predictor of survival in breast cancer. N. Engl. J. Med. 347, 1999–2009 (2002).

    Article  CAS  PubMed  Google Scholar 

  24. Huang, E. et al. Gene expression predictors of breast cancer outcomes. Lancet 361, 1590–1596 (2003).

    Article  CAS  PubMed  Google Scholar 

  25. Ramaswamy, S. & Perou, C. M. DNA microarrays in breast cancer: the promise of personalised medicine. Lancet 361, 1576–1577 (2003).

    Article  PubMed  Google Scholar 

  26. Ransohoff, D. F. Gene-expression signatures in breast cancer. N. Engl. J. Med. 348, 1715–1717; author reply 1715–1717 (2003).

    Article  PubMed  Google Scholar 

  27. Marshall, E. Getting the noise out of gene arrays. Science 306, 630–631 (2004).

    Article  CAS  PubMed  Google Scholar 

  28. Ambroise, C. & McLachlan, G. J. Selection bias in gene extraction on the basis of microarray gene-expression data. Proc. Natl Acad. Sci. USA 99, 6562–6566 (2002).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  29. Baggerly, K. A. et al. A comprehensive approach to the analysis of matrix-assisted laser desorption/ionization-time of flight proteomics spectra from serum samples. Proteomics 3, 1667–1672 (2003).

    Article  CAS  PubMed  Google Scholar 

  30. Mehta, T., Tanik, M. & Allison, D. B. Towards sound epistemological foundations of statistical methods for high-dimensional biology. Nature Genet. 36, 943–947 (2004).

    Article  CAS  PubMed  Google Scholar 

  31. Ransohoff, D. F. Rules of evidence for cancer molecular-marker discovery and validation. Nature Rev. Cancer 4, 309–314 (2004).

    Article  CAS  Google Scholar 

  32. Ransohoff, D. F. Discovery-based research and fishing. Gastroenterology 125, 290 (2003).

    Article  PubMed  Google Scholar 

  33. Hulley, S. B. et al. Designing Clinical Research: An Epidemiologic Approach (Lippincott Williams & Wilkins, Philadelphia, 2001).

    Google Scholar 

  34. Rothman, K. J. Modern Epidemiology (Little, Brown and Company, Boston/Toronto, 1986).

    Google Scholar 

  35. Hennekens, C. H. & Buring, J. E. Epidemiology in Medicine (Little, Brown and Company, Boston, 1987).

    Google Scholar 

  36. Taubes, G. Epidemiology faces its limits. Science 269, 164–169 (1995).

    Article  CAS  PubMed  Google Scholar 

  37. Ransohoff, D. F. Research opportunity at the interface of molecular biology and clinical epidemiology. Gastroenterology 122, 1199 (2002).

    Article  Google Scholar 

  38. Ransohoff, D. F. Evaluating discovery-based research: when biologic reasoning cannot work. Gastroenterology 127, 1028 (2004).

    Article  PubMed  Google Scholar 

  39. Coombes, R. C. et al. A randomized trial of exemestane after two to three years of tamoxifen therapy in postmenopausal women with primary breast cancer. N. Engl. J. Med. 350, 1081–1092 (2004).

    Article  CAS  PubMed  Google Scholar 

  40. Altman, D. G. et al. The revised CONSORT statement for reporting randomized trials: explanation and elaboration. Ann. Intern. Med. 134, 663–694 (2001).

    Article  CAS  PubMed  Google Scholar 

  41. The case-control study: consensus and controversy. J. Chronic. Dis. 32, 1–144 (1979).

  42. Sackett, D. L. Bias in analytic research. J. Chronic. Dis. 32, 51–63 (1979).

    Article  CAS  PubMed  Google Scholar 

  43. Roos, N. P. et al. Mortality and reoperation after open and transurethral resection of the prostate for benign prostatic hyperplasia. N. Engl. J. Med. 320, 1120–1124 (1989).

    Article  CAS  PubMed  Google Scholar 

  44. Greenfield, S. The state of outcome research: are we on target? N. Engl. J. Med. 320, 1142–1143 (1989).

    Article  CAS  PubMed  Google Scholar 

  45. Concato, J., Horwitz, R. I., Feinstein, A. R., Elmore, J. G. & Schiff, S. F. Problems of comorbidity in mortality after prostatectomy. JAMA 267, 1077–1082 (1992).

    Article  CAS  PubMed  Google Scholar 

  46. Iezzoni, L. I. et al. Comorbidities, complications, and coding bias. Does the number of diagnosis codes matter in predicting in-hospital mortality? JAMA 267, 2197–2203 (1992).

    Article  CAS  PubMed  Google Scholar 

  47. Iezzoni, L. I. The risks of risk adjustment. JAMA 278, 1600–1607 (1997).

    Article  CAS  PubMed  Google Scholar 

  48. Hulley, S. et al. Randomized trial of estrogen plus progestin for secondary prevention of coronary heart disease in postmenopausal women. Heart and Estrogen/progestin Replacement Study (HERS) Research Group. JAMA 280, 605–613 (1998).

    Article  CAS  PubMed  Google Scholar 

  49. Manson, J. E. et al. Estrogen plus progestin and the risk of coronary heart disease. N. Engl. J. Med. 349, 523–534 (2003).

    Article  CAS  PubMed  Google Scholar 

  50. Col, N. F. & Pauker, S. G. The discrepancy between observational studies and randomized trials of menopausal hormone therapy: did expectations shape experience? Ann. Intern. Med. 139, 923–929 (2003).

    Article  PubMed  Google Scholar 

  51. Rennie, D. How to report randomized controlled trials. The CONSORT statement. JAMA 276, 649 (1996).

    Article  CAS  PubMed  Google Scholar 

  52. Bossuyt, P. M. et al. Towards complete and accurate reporting of studies of diagnostic accuracy: The STARD Initiative. Ann. Intern. Med. 138, 40–44 (2003).

    Article  PubMed  Google Scholar 

  53. Bossuyt, P. M. et al. The STARD statement for reporting studies of diagnostic accuracy: explanation and elaboration. Ann. Intern. Med. 138, W1–12 (2003).

    Article  PubMed  Google Scholar 

  54. Brazma, A. et al. Minimum information about a microarray experiment (MIAME)—toward standards for microarray data. Nature Genet. 29, 365–371 (2001).

    Article  CAS  PubMed  Google Scholar 

  55. Ransohoff, D. F. Lessons from controversy: ovarian cancer screening and serum proteomics. J. Natl Cancer Inst. (in the press).

  56. Imperiale, T. F. et al. Fecal DNA versus fecal occult blood for colorectal-cancer screening in an average-risk population. N. Engl. J. Med. 351, 2704–2714 (2004).

    Article  CAS  PubMed  Google Scholar 

  57. Prorok, P. C. et al. Design of the Prostate, Lung, Colorectal and Ovarian (PLCO) cancer screening trial. Control Clin. Trials 21, 273S–309S (2000).

    Article  CAS  PubMed  Google Scholar 

  58. Simon, R., Radmacher, M. D., Dobbin, K. & McShane, L. M. Pitfalls in the use of DNA microarray data for diagnostic and prognostic classification. J. Natl Cancer Inst. 95, 14–18 (2003).

    Article  CAS  PubMed  Google Scholar 

  59. Ntzani, E. E. & Ioannidis, J. P. Predictive ability of DNA microarrays for cancer outcomes and correlates: an empirical assessment. Lancet 362, 1439–1444 (2003).

    Article  CAS  PubMed  Google Scholar 

  60. Friedman, L. M., Furberg, C. D. & De Mets, D. L. Fundamentals of Clinical Trials, 3rd edn (Springer, New York, 1998).

    Book  Google Scholar 

  61. Sullivan Pepe, M. et al. Phases of biomarker development for early detection of cancer. J. Natl Cancer Inst. 93, 1054–1061 (2001).

    Article  Google Scholar 

Download references

Acknowledgements

Thanks to colleagues at the University of North Carolina at Chapel Hill, the National Cancer Institute and elsewhere for reviewing and commenting on earlier versions of the manuscript. Many ideas were developed through participation in activities of the Early Detection Research Network.

Author information

Authors and Affiliations

  1. Departments of Medicine and Epidemiology, University of North Carolina at Chapel Hill, CB# 7080, Bioinformatics Building, 4103, Chapel Hill, 27599–7080, North Carolina, USA

    David F. Ransohoff

  2. Division of Cancer Prevention, National Cancer Institute, National Institutes of Health, Bethesda, 20892–7354, Maryland, USA

    David F. Ransohoff

Authors
  1. David F. Ransohoff
    View author publications

    You can also search for this author in PubMed Google Scholar

Ethics declarations

Competing interests

The author declares no competing financial interests.

Related links

Related links

DATABASES

National Cancer Institute

breast cancer

colorectal cancer

ovarian cancer

prostate cancer

FURTHER INFORMATION

Early Detection Research Network

Rights and permissions

Reprints and permissions

About this article

Cite this article

Ransohoff, D. Bias as a threat to the validity of cancer molecular-marker research. Nat Rev Cancer 5, 142–149 (2005). https://doi.org/10.1038/nrc1550

Download citation

  • Issue Date:

  • DOI: https://doi.org/10.1038/nrc1550

Search

Advanced search

Quick links

Nature Briefing

Sign up for the Nature Briefing newsletter — what matters in science, free to your inbox daily.

Get the most important science stories of the day, free in your inbox. Sign up for Nature Briefing