Skip to main content
Springer Nature Link
Log in

Development and validation of an ultrasound-based nomogram to improve the diagnostic accuracy for malignant thyroid nodules

  • Ultrasound
  • Published:
European Radiology Aims and scope Submit manuscript

Abstract

Objectives

The aim of this study was to develop an ultrasound-based nomogram to improve the diagnostic accuracy of the identification of malignant thyroid nodules.

Methods

A total of 1675 histologically proven thyroid nodules (1169 benign, 506 malignant) were included in this study. The nodules were grouped into the training dataset (n = 700), internal validation dataset (n = 479), or external validation dataset (n = 496). The grayscale ultrasound features included the nodule size, shape, aspect ratio, echogenicity, margins, and calcification pattern. We applied least absolute shrinkage and selection operator (lasso) regression to select the strongest features for the nomogram. Nomogram discrimination (area under the receiver operating characteristic curve, AUC) and calibration were assessed. The nomogram was subjected to bootstrapping validation (1000 bootstrap resamples) to calculate a mean AUC and 95% confidence interval (CI).

Results

The nomogram showed good discrimination in the training dataset, with an AUC of 0.936 (95% CI: 0.918–0.953) and good calibration. Application of the nomogram to the internal validation dataset also resulted in good discrimination (AUC: 0.935; 95% CI, 0.915–0.954) and good calibration. The model tested in an external validation dataset demonstrated a lower AUC of 0.782 (95% CI: 0.776–0.789).

Conclusions

This ultrasound-based nomogram can be used to quantify the probability of malignant thyroid nodules.

Key Points

• Ultrasound examination is helpful in the differential diagnosis of malignant and benign thyroid nodules.

• However, ultrasound accuracy relies heavily on examiner experience.

• A less subjective diagnostic model is desired, and the developed nomogram for thyroid nodules showed good discrimination and good calibration.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price includes VAT (Canada)

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4

Similar content being viewed by others

Abbreviations

AUC:

Area under the curve

CI:

Confidence interval

FNA:

Fine-needle aspiration

Lasso:

Least absolute shrinkage and selection operator

SD:

Standard deviation

References

  1. Yoon JH, Lee HS, Kim EK, Moon HJ, Kwak JY (2016) Malignancy Risk Stratification of Thyroid Nodules: Comparison between the Thyroid Imaging Reporting and Data System and the 2014 American Thyroid Association Management Guidelines. Radiology 278(3):917–924

    Article  PubMed  Google Scholar 

  2. Gharib H, Papini E, Paschke R et al (2010) American Association of Clinical Endocrinologists, Associazione Medici Endocrinologi, and European Thyroid Association Medical Guidelines for Clinical Practice for the Diagnosis and Management of Thyroid Nodules. Endocr Pract 16(3):468–475

    Article  PubMed  Google Scholar 

  3. Xing M (2016) Clinical utility of RAS mutations in thyroid cancer: a blurred picture now emerging clearer. BMC Med 14:12

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  4. Burman KD, Wartofsky L (2015) CLINICAL PRACTICE. Thyroid Nodules. N Engl J Med 373(24):2347–2356

    Article  CAS  PubMed  Google Scholar 

  5. Haugen BR, Alexander EK, Bible KC et al (2016) 2015 American Thyroid Association Management Guidelines for Adult Patients with Thyroid Nodules and Differentiated Thyroid Cancer: The American Thyroid Association Guidelines Task Force on Thyroid Nodules and Differentiated Thyroid Cancer. Thyroid 26(1):1–133

    Article  PubMed  PubMed Central  Google Scholar 

  6. Kim D, Kim DW, Heo YJ et al (2017) Computed Tomography Features of Benign and Malignant Calcified Thyroid Nodules: A Single-Center Study. J Comput Assist Tomogr 41(6):937–940

    Article  PubMed  Google Scholar 

  7. Shi RY, Yao QY, Zhou QY et al (2017) Preliminary study of diffusion kurtosis imaging in thyroid nodules and its histopathologic correlation. Eur Radiol 27(11):4710–4720

    Article  PubMed  Google Scholar 

  8. Pathak KA, Goertzen AL, Nason RW, Klonisch T, Leslie WD (2016) A prospective cohort study to assess the role of FDG-PET in differentiating benign and malignant follicular neoplasms. Ann Med Surg (Lond) 12:27–31

    Article  Google Scholar 

  9. Raue F, Frank-Raue K (2016) Thyroid Cancer: Risk-Stratified Management and Individualized Therapy. Clin Cancer Res 22(10):5012–5021

    Article  CAS  PubMed  Google Scholar 

  10. Kaur K, Sonkhya N, Bapna AS, Mital P (2002) A comparative study of fine needle aspiration cytology, ultrasonography and radionuclide scan in the management of solitary thyroid nodule: A prospective analysis of fifty cases. Indian J Otolaryngol Head Neck Surg 54(2):96–101

    PubMed  PubMed Central  Google Scholar 

  11. Tae HJ, Lim DJ, Baek KH et al (2007) Diagnostic value of ultrasonography to distinguish between benign and malignant lesions in the management of thyroid nodules. Thyroid 17(5):461–466

    Article  PubMed  Google Scholar 

  12. Oberije C, Nalbantov G, Dekker A et al (2014) A prospective study comparing the predictions of doctors versus models for treatment outcome of lung cancer patients: a step toward individualized care and shared decision making. Radiother Oncol 112(1):37–43

    Article  PubMed  PubMed Central  Google Scholar 

  13. Iasonos A, Schrag D, Raj GV, Panageas KS (2008) How to build and interpret a nomogram for cancer prognosis. J Clin Oncol 26:1364–1370

    Article  PubMed  Google Scholar 

  14. Abu-Rustum NR, Zhou Q, Gomez JD et al (2010) A nomogram for predicting overall survival of women with endometrial cancer following primary therapy: toward improving individualized cancer care. Gynecol Oncol 116:399–403

    Article  CAS  PubMed  Google Scholar 

  15. Sternberg CN (2006) Are nomograms better than currently available stage groupings for bladder cancer? J Clin Oncol 24:3819–3820

    Article  PubMed  Google Scholar 

  16. Kumamaru KK, Saboo SS, Aghayev A et al (2016) CT pulmonary angiography-based scoring system to predict the prognosis of acute pulmonary embolism. J Cardiovasc Comput Tomogr 10(6):473–479

    Article  PubMed  Google Scholar 

  17. Vasquez MM, Hu C, Roe DJ, Chen Z, Halonen M, Guerra S (2016) Least absolute shrinkage and selection operator type methods for the identification of serum biomarkers of overweight and obesity: simulation and application. BMC Med Res Methodol 16(1):154

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  18. Werga P, Wallin G, Skoog L, Hamberger B (2000) Expanding role of fine-needle aspiration cytology in thyroid diagnosis and management. World J Surg 24:907–912

    Article  CAS  PubMed  Google Scholar 

  19. Theoharis CG, Schofield KM, Hammers L, Udelsman R, Chhieng DC (2009) The Bethesda thyroid fine-needle aspiration classification system: year 1 at an academic institution. Thyroid 19(11):1215–1223

    Article  PubMed  Google Scholar 

  20. Mathur A, Weng J, Moses W et al (2010) A prospective study evaluating the accuracy of using combined clinical factors and candidate diagnostic markers to refine the accuracy of thyroid fine needle aspiration biopsy. Surgery 148(6):1170–1176

    Article  PubMed  Google Scholar 

  21. Zhang F, Zhao X, Han R, Du M, Li P, Ji X (2017) Comparison of Acoustic Radiation Force Impulse Imaging and Strain Elastography in Differentiating Malignant From Benign Thyroid Nodules. J Ultrasound Med 36(12):2533–2543

    Article  PubMed  Google Scholar 

  22. Chandramohan A, Therese M, Abhraham D, Paul TV, Mazhuvanchary PJ (2017) Can ARFI elastography be used to differentiate parathyroid from thyroid lesions? J Endocrinol Invest 41(1):111–119

    Article  PubMed  Google Scholar 

  23. Zhan J, Diao XH, Chen L, Jin JM, Chen Y (2017) Role of Contrast-Enhanced Ultrasound in Diagnosis of Thyroid Nodules in Acoustic Radiation Force Impulse “Gray Zone”. Ultrasound Med Biol 43(6):1179–1186

    Article  PubMed  Google Scholar 

  24. Colakoglu B, Yildirim D, Alis D et al (2016) Elastography in Distinguishing Benign from Malignant Thyroid Nodules. J Clin Imaging Sci 6:51

    Article  PubMed  PubMed Central  Google Scholar 

  25. Zahir ST, Vakili M, Ghaneei A, Sharahjin NS, Heidari F (2016) Ultrasound Assistance In Differentiating Malignant Thyroid Nodules From Benign Ones. J Ayub Med Coll Abbottabad 28(4):644–649

    PubMed  Google Scholar 

  26. Iqbal M, Mehmood Z, Rasul S, Inamullah H, Shah SS, Bokhari I (2010) Carcinoma thyroid in multi and uninodular goiter. J Coll Physicians Surg Pak 20(5):310–312

  27. Chammas MC, de Araujo Filho VJ, Moysés RA et al (2008) Predictive value for malignancy in the finding of microcalcifications on ultrasonography of thyroid nodules. Head Neck 30(9):1206–1210

  28. Rahimi M, Farshchian N, Rezaee E, Shahebrahimi K, Madani H (2013) To differentiate benign from malignant thyroid nodule comparison of sonography with FNAC findings. Pak J Med Sci 29(1):77–80

    PubMed  PubMed Central  Google Scholar 

  29. Kim EK, Park CS, Chung WY et al (2002) New sonographic criteria for recommending fine-needle aspiration biopsy of nonpalpable solid nodules of the thyroid. AJR Am J Roentgenol 178:687–691

    Article  PubMed  Google Scholar 

  30. Nikiforov YE, Seethala RR, Tallini G et al (2016) Nomenclature Revision for Encapsulated Follicular Variant of Papillary Thyroid Carcinoma: A Paradigm Shift to Reduce Overtreatment of Indolent Tumors. JAMA Oncol 2(8):1023–1029

    Article  PubMed  PubMed Central  Google Scholar 

Download references

Funding

This study has received funding from the science and technology project of Foshan (2017AB003623 and 2017AB003683).

Author information

Author notes

  1. Baoliang Guo and Fusheng Ouyang contributed equally to this work.

Authors and Affiliations

  1. Department of Radiology, Shunde Hospital of Southern Medical University (The First People’s Hospital of Shunde), No.1, Penglai Road, Daliang District, Shunde, Foshan, Guangdong, People’s Republic of China

    Bao-liang Guo, Fu-sheng Ouyang, Zi-wei Liu, Shao-jia Lin, Wei Meng, Hai-xiong Chen, Shao-ming Yang & Qiu-gen Hu

  2. Department of Ultrasound, Shunde Hospital of Southern Medical University (The First People’s Hospital of Shunde), Foshan, Guangdong, People’s Republic of China

    Li-zhu Ouyang

  3. Department of Laboratory, Lecong Hospital of Shunde, Foshan, Guangdong, People’s Republic of China

    Xi-yi Huang

Authors
  1. Bao-liang Guo
    View author publications

    You can also search for this author inPubMed Google Scholar

  2. Fu-sheng Ouyang
    View author publications

    You can also search for this author inPubMed Google Scholar

  3. Li-zhu Ouyang
    View author publications

    You can also search for this author inPubMed Google Scholar

  4. Zi-wei Liu
    View author publications

    You can also search for this author inPubMed Google Scholar

  5. Shao-jia Lin
    View author publications

    You can also search for this author inPubMed Google Scholar

  6. Wei Meng
    View author publications

    You can also search for this author inPubMed Google Scholar

  7. Xi-yi Huang
    View author publications

    You can also search for this author inPubMed Google Scholar

  8. Hai-xiong Chen
    View author publications

    You can also search for this author inPubMed Google Scholar

  9. Shao-ming Yang
    View author publications

    You can also search for this author inPubMed Google Scholar

  10. Qiu-gen Hu
    View author publications

    You can also search for this author inPubMed Google Scholar

Corresponding author

Correspondence to Qiu-gen Hu.

Ethics declarations

Guarantor

The scientific guarantor of this publication is Qiugen Hu

Conflict of interest

The authors of this manuscript declare no relationships with any companies, whose products or services may be related to the subject matter of the article.

Statistics and biometry

No complex statistical methods were necessary for this paper.

Informed consent

Written informed consent was waived by the Institutional Review Board.

Ethical approval

Institutional Review Board approval was obtained.

Methodology

• retrospective

• diagnostic study

• performed at two institutions

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Guo, Bl., Ouyang, Fs., Ouyang, Lz. et al. Development and validation of an ultrasound-based nomogram to improve the diagnostic accuracy for malignant thyroid nodules. Eur Radiol 29, 1518–1526 (2019). https://doi.org/10.1007/s00330-018-5715-5

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00330-018-5715-5

Keywords