Skip to main content

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

  • Brief Communication
  • Published:

Convolutional neural networks for automated annotation of cellular cryo-electron tomograms

Nature Methods volume 14, pages 983–985 (2017)Cite this article

Subjects

Abstract

Cellular electron cryotomography offers researchers the ability to observe macromolecules frozen in action in situ, but a primary challenge with this technique is identifying molecular components within the crowded cellular environment. We introduce a method that uses neural networks to dramatically reduce the time and human effort required for subcellular annotation and feature extraction. Subsequent subtomogram classification and averaging yield in situ structures of molecular components of interest. The method is available in the EMAN2.2 software package.

This is a preview of subscription content, access via your institution

Access options

Buy this article

  • Purchase on SpringerLink
  • Instant access to full article PDF
Buy now

Prices may be subject to local taxes which are calculated during checkout

Figure 1: Workflow of automated tomogram annotation using a PC12 cell as an example.
Figure 2: Results from automated tomogram annotation.

Similar content being viewed by others

Accession codes

Primary accessions

Electron Microscopy Data Bank

Referenced accessions

Electron Microscopy Data Bank

References

  1. Lučič, V., Rigort, A. & Baumeister, W. J. Cell Biol. 202, 407–419 (2013).

    Article  Google Scholar 

  2. Galaz-Montoya, J.G. et al. J. Struct. Biol. 194, 383–394 (2016).

    Article  CAS  Google Scholar 

  3. Chen, Y., Pfeffer, S., Hrabe, T., Schuller, J.M. & Förster, F. J. Struct. Biol. 182, 235–245 (2013).

    Article  Google Scholar 

  4. Asano, S. et al. Science 347, 439–442 (2015).

    Article  CAS  Google Scholar 

  5. Pfeffer, S., Woellhaf, M.W., Herrmann, J.M. & Förster, F. Nat. Commun. 6, 6019 (2015).

    Article  CAS  Google Scholar 

  6. Ding, H.J., Oikonomou, C.M. & Jensen, G.J. J. Struct. Biol. 192, 279–286 (2015).

    Article  Google Scholar 

  7. Rigort, A. et al. J. Struct. Biol. 177, 135–144 (2012).

    Article  CAS  Google Scholar 

  8. Page, C., Hanein, D. & Volkmann, N. Ultramicroscopy 155, 20–26 (2015).

    Article  CAS  Google Scholar 

  9. Frangakis, A.S. et al. Proc. Natl. Acad. Sci. USA 99, 14153–14158 (2002).

    Article  CAS  Google Scholar 

  10. LeCun, Y., Bengio, Y. & Hinton, G. Nature 521, 436–444 (2015).

    Article  CAS  Google Scholar 

  11. Garduño, E., Wong-Barnum, M., Volkmann, N. & Ellisman, M.H. J. Struct. Biol. 162, 368–379 (2008).

    Article  Google Scholar 

  12. Hecksel, C.W. et al. Microsc. Microanal. 22, 487–496 (2016).

    Article  CAS  Google Scholar 

  13. Wang, R. et al. Proc. Natl. Acad. Sci. USA 112, 14266–14271 (2015).

    Article  CAS  Google Scholar 

  14. Dai, W. et al. Nature 502, 707–710 (2013).

    Article  CAS  Google Scholar 

  15. Hashem, Y. et al. Nature 494, 385–389 (2013).

    Article  CAS  Google Scholar 

  16. Asenjo, A.B. et al. Cell Rep. 3, 759–768 (2013).

    Article  CAS  Google Scholar 

  17. Koning, R.I. et al. J. Struct. Biol. 161, 459–468 (2008).

    Article  CAS  Google Scholar 

  18. Garvalov, B.K. et al. J. Cell Biol. 174, 759–765 (2006).

    Article  CAS  Google Scholar 

  19. Scheuring, S. & Sturgis, J.N. Science 309, 484–487 (2005).

    Article  CAS  Google Scholar 

  20. Tang, G. et al. J. Struct. Biol. 157, 38–46 (2007).

    Article  CAS  Google Scholar 

  21. Nair, V. & Hinton, G.E. In Proc. 27th Int. Conf. Mach. Learn. (eds. Fürnkranz, J. & Joachims, T.) 807–814 (ICML, 2010).

  22. Vincent, P., Larochelle, H., Bengio, Y. & Manzagol, P.-A. In Proc. 25th Int. Conf. Mach. Learn (eds. McCallum, A. & Roweis, S.) 1096–1103 (ICML, 2008).

  23. Hinton, G.E., Srivastava, N., Krizhevsky, A., Sutskever, I. & Salakhutdinov, R.R. Preprint at http://arxiv.org/abs/1207.0580 (2012).

  24. Dieleman, S., Willett, K.W. & Dambre, J. Mon. Not. R. Astron. Soc. 450, 1441–1459 (2015).

    Article  Google Scholar 

  25. Zhou, J. & Troyanskaya, O.G. Nat. Methods 12, 931–934 (2015).

    Article  CAS  Google Scholar 

  26. Noh, H., Hong, S. & Han, B. Preprint at http://arxiv.org/abs/1505.04366 (2015).

  27. Krizhevsky, A., Sutskever, I. & Hinton, G.E. In Advances in Neural Information Processing Systems 25 (eds. Pereira, F., Burges, C.J.C., Bottou, L. & Weinberger, K.Q.) 1097–1105 (Curran Associates, 2012).

  28. Dai, W. et al. Nat. Protoc. 9, 2630–2642 (2014).

    Article  CAS  Google Scholar 

  29. Apostol, B.L. et al. Proc. Natl. Acad. Sci. USA 100, 5950–5955 (2003).

    Article  CAS  Google Scholar 

  30. Wirtz, E., Leal, S., Ochatt, C. & Cross, G.A. Mol. Biochem. Parasitol. 99, 89–101 (1999).

    Article  CAS  Google Scholar 

  31. Kaminsky, R., Beaudoin, E. & Cunningham, I. Acta Trop. 45, 33–43 (1988).

    CAS  PubMed  Google Scholar 

  32. Chen, M. et al. Protocol Exchange https://doi.org/10.1038/nprot.2017.095 (2017).

    Article  CAS  Google Scholar 

Download references

Acknowledgements

We gratefully acknowledge support of NIH grants (R01GM080139, P01NS092525, P41GM103832), Ovarian Cancer Research Fund and Singapore Ministry of Education. Molecular graphics and analyses performed with UCSF ChimeraX, developed by the Resource for Biocomputing, Visualization, and Informatics at the University of California, San Francisco.

Author information

Author notes

  1. Wei Dai

    Present address: Department of Cell Biology and Neuroscience, Center for Integrative Proteomics Research, Rutgers University, Piscataway, New Jersey, USA

Authors and Affiliations

  1. Graduate Program in Structural and Computational Biology and Molecular Biophysics, Baylor College of Medicine, Houston, Texas, USA

    Muyuan Chen

  2. Verna Marrs and McLean Department of Biochemistry and Molecular Biology, Baylor College of Medicine, Houston, Texas, USA

    Muyuan Chen, Wei Dai, Stella Y Sun, Darius Jonasch, Michael F Schmid, Wah Chiu & Steven J Ludtke

  3. Department of Biological Science, Centre for BioImaging Sciences, National University of Singapore, Singapore

    Cynthia Y He

Authors
  1. Muyuan Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Wei Dai
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Stella Y Sun
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Darius Jonasch
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Cynthia Y He
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Michael F Schmid
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Wah Chiu
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Steven J Ludtke
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

M.C. designed the protocol. W.D., S.Y.S. and C.Y.H. provided the test data sets. M.C. and D.J. tested and refined the protocol. M.C., W.D., S.Y.S., M.F.S., W.C. and S.J.L. wrote the paper and provided suggestions during development.

Corresponding author

Correspondence to Steven J Ludtke.

Ethics declarations

Competing interests

The authors declare no competing financial interests.

Integrated supplementary information

Supplementary Figure 1 Missing wedge artifact in CryoET.

a. Slice view of a tomogram in the X-Y, Y-Z and X-Z plane. b. Fourier Transform of projection of the same tomogram in X-Y, Y-Z and X-Z plane.

Supplementary Figure 2 Convolutional neural network structure.

Supplementary Figure 3 Reusability of trained neural network.

a-d. Annotation of ribosomes in a tomogram of cyanobacteria taken with a Zernike phase plate using different CNNs. a. Ribosomes annotated by a CNN trained on 5 positive and 50 negative samples from the same tomogram (89% correct). b. Using a CNN trained on 5 positive and 50 negative samples from another ZPP tomogram (86% correct). c. Using a CNN trained on 5 positive and 50 negative samples from a non ZPP tomogram (78% correct). d. Using a CNN trained on 10 positive and 100 negative samples from the tomogram in a (91.3% correct). e-h. Annotation of microtubules in two tomograms (I and II) of PC12 cells using two CNNs. CNN A is trained on 5 positive and 50 negative samples from tomogram I. CNN B is trained on 5 positive and 50 negative samples from tomogram II. e. Microtubules in tomogram I annotated by CNN A. f. Microtubules in tomogram II annotated by CNN A. g. Microtubules in tomogram I annotated by CNN B. h. Microtubules in tomogram II annotated by CNN B.

Supplementary Figure 4 Comparison of raw tomogram slices and corresponding automatic annotation in Fig 2.

a. A slice of the Cyanobacteria tomogram in Fig2. b. 3D view of the tomogram annotation of a. c. A slice of the Trypanosome tomogram in Fig2. d. 3D view of the tomogram annotation of c. e. A slice of the platelet tomogram in Fig2. f. 3D view of the tomogram annotation of e.

Supplementary Figure 5 Impact of training inaccuracy on CNN annotations.

a. One of the training samples of microtubules of PC12 cell used in Fig.1b. b. Corresponding manual annotation with a minor error marked by the red arrow. c. Annotation of the patch using a CNN trained on the sample with incorrect annotation and all the other positive and negative samples with correct manual annotation used in Fig.1b. Note the error is fixed in the annotation. d. A slice view of the tomogram. e. Annotation of the slice using the CNN in c.

Supplementary Figure 6 Removal of high contrast artifact.

a. Volume rendering of a region of the tomogram shown in Fig 1. b. Annotation of double membrane in the tomogram using the same CNN used in Fig 1. Note the carbon edge is falsely recognized as double membrane. c. Automatic annotation of carbon edge using a CNN. d. Removal of carbon edge from double membrane annotation in b.

Supplementary information

Supplementary Text and Figures

Supplementary Figures 1–6 and Supplementary Table 1.

Life Sciences Reporting Summary

Supplementary Protocol

Supplementary Protocol.

Workflow of automated tomogram annotation

Demonstration of tomogram annotation workflow with our software package, using a PC12 cell tomogram as an example. The process includes selection of training set, annotation of the tomogram, and subtomogram averaging results.

Annotation of the PC12 cell tomogram

Movie showing the PC12 cell tomogram used in the figure and its annotation, including volume rendering and slice view of the input tomogram, and a 3D view of annotated features.

Annotation of the human platelet cell tomogram

Movie showing the human platelet cell tomogram used in the figure and its annotation, including volume rendering and slice view of the input tomogram, and a 3D view of annotated features.

Annotation of the African Trypanosomes cell tomogram

Movie showing the African Trypanosomes cell tomogram used in the figure and its annotation, including volume rendering and slice view of the input tomogram, and a 3D view of annotated features.

Annotation of the Cyanobacteria cell tomogram

Movie showing the Cyanobacteria cell tomogram used in the figure and its annotation, including volume rendering and slice view of the input tomogram, and a 3D view of annotated features.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Chen, M., Dai, W., Sun, S. et al. Convolutional neural networks for automated annotation of cellular cryo-electron tomograms. Nat Methods 14, 983–985 (2017). https://doi.org/10.1038/nmeth.4405

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1038/nmeth.4405

This article is cited by

Search

Advanced search

Quick links

Nature Briefing AI and Robotics

Sign up for the Nature Briefing: AI and Robotics newsletter — what matters in AI and robotics research, free to your inbox weekly.

Get the most important science stories of the day, free in your inbox. Sign up for Nature Briefing: AI and Robotics