Convolutional neural network transformer (CNNT) for fluorescence microscopy image denoising with improved generalization and fast adaptation
- PMID: 39107416
- PMCID: PMC11303381
- DOI: 10.1038/s41598-024-68918-2
Convolutional neural network transformer (CNNT) for fluorescence microscopy image denoising with improved generalization and fast adaptation
Abstract
Deep neural networks can improve the quality of fluorescence microscopy images. Previous methods, based on Convolutional Neural Networks (CNNs), require time-consuming training of individual models for each experiment, impairing their applicability and generalization. In this study, we propose a novel imaging-transformer based model, Convolutional Neural Network Transformer (CNNT), that outperforms CNN based networks for image denoising. We train a general CNNT based backbone model from pairwise high-low Signal-to-Noise Ratio (SNR) image volumes, gathered from a single type of fluorescence microscope, an instant Structured Illumination Microscope. Fast adaptation to new microscopes is achieved by fine-tuning the backbone on only 5-10 image volume pairs per new experiment. Results show that the CNNT backbone and fine-tuning scheme significantly reduces training time and improves image quality, outperforming models trained using only CNNs such as 3D-RCAN and Noise2Fast. We show three examples of efficacy of this approach in wide-field, two-photon, and confocal fluorescence microscopy.
© 2024. This is a U.S. Government work and not under copyright protection in the US; foreign copyright protection may apply.
Conflict of interest statement
The authors declare no competing interests.
Figures
Update of
-
Convolutional Neural Network Transformer (CNNT) for Fluorescence Microscopy image Denoising with Improved Generalization and Fast Adaptation.ArXiv [Preprint]. 2024 Apr 6:arXiv:2404.04726v1. ArXiv. 2024. Update in: Sci Rep. 2024 Aug 6;14(1):18184. doi: 10.1038/s41598-024-68918-2. PMID: 38903737 Free PMC article. Updated. Preprint.
Similar articles
-
Convolutional Neural Network Transformer (CNNT) for Fluorescence Microscopy image Denoising with Improved Generalization and Fast Adaptation.ArXiv [Preprint]. 2024 Apr 6:arXiv:2404.04726v1. ArXiv. 2024. Update in: Sci Rep. 2024 Aug 6;14(1):18184. doi: 10.1038/s41598-024-68918-2. PMID: 38903737 Free PMC article. Updated. Preprint.
-
Evaluation of Swin Transformer and knowledge transfer for denoising of super-resolution structured illumination microscopy data.Gigascience. 2024 Jan 2;13:giad109. doi: 10.1093/gigascience/giad109. Gigascience. 2024. PMID: 38217407 Free PMC article.
-
Learning low-dose CT degradation from unpaired data with flow-based model.Med Phys. 2022 Dec;49(12):7516-7530. doi: 10.1002/mp.15886. Epub 2022 Aug 8. Med Phys. 2022. PMID: 35880375
-
Rapid 2D 23Na MRI of the calf using a denoising convolutional neural network.Magn Reson Imaging. 2024 Jul;110:184-194. doi: 10.1016/j.mri.2024.04.027. Epub 2024 Apr 19. Magn Reson Imaging. 2024. PMID: 38642779
-
Recent developments in denoising medical images using deep learning: An overview of models, techniques, and challenges.Micron. 2024 May;180:103615. doi: 10.1016/j.micron.2024.103615. Epub 2024 Mar 2. Micron. 2024. PMID: 38471391 Review.
References
-
- Minaee, S. et al. Image segmentation using deep learning: A survey. IEEE Trans. Pattern Anal. Mach. Intell.44, 3523–3542 (2022). - PubMed
LinkOut - more resources
Full Text Sources