Stability-based validation of clustering solutions
- PMID: 15130251
- DOI: 10.1162/089976604773717621
Stability-based validation of clustering solutions
Abstract
Data clustering describes a set of frequently employed techniques in exploratory data analysis to extract "natural" group structure in data. Such groupings need to be validated to separate the signal in the data from spurious structure. In this context, finding an appropriate number of clusters is a particularly important model selection question. We introduce a measure of cluster stability to assess the validity of a cluster model. This stability measure quantifies the reproducibility of clustering solutions on a second sample, and it can be interpreted as a classification risk with regard to class labels produced by a clustering algorithm. The preferred number of clusters is determined by minimizing this classification risk as a function of the number of clusters. Convincing results are achieved on simulated as well as gene expression data sets. Comparisons to other methods demonstrate the competitive performance of our method and its suitability as a general validation tool for clustering solutions in real-world problems.
Similar articles
-
Weighted rank aggregation of cluster validation measures: a Monte Carlo cross-entropy approach.Bioinformatics. 2007 Jul 1;23(13):1607-15. doi: 10.1093/bioinformatics/btm158. Epub 2007 May 5. Bioinformatics. 2007. PMID: 17483500
-
Clustering of change patterns using Fourier coefficients.Bioinformatics. 2008 Jan 15;24(2):184-91. doi: 10.1093/bioinformatics/btm568. Epub 2007 Nov 19. Bioinformatics. 2008. PMID: 18025003
-
Graph-based consensus clustering for class discovery from gene expression data.Bioinformatics. 2007 Nov 1;23(21):2888-96. doi: 10.1093/bioinformatics/btm463. Epub 2007 Sep 14. Bioinformatics. 2007. PMID: 17872912
-
Comparing algorithms for clustering of expression data: how to assess gene clusters.Methods Mol Biol. 2009;541:479-509. doi: 10.1007/978-1-59745-243-4_21. Methods Mol Biol. 2009. PMID: 19381534 Review.
-
Gene expression profiling--Clusters of possibilities.Methods. 2010 Apr;50(4):323-35. doi: 10.1016/j.ymeth.2010.01.009. Epub 2010 Jan 15. Methods. 2010. PMID: 20079843 Review.
Cited by
-
Finding reproducible cluster partitions for the k-means algorithm.BMC Bioinformatics. 2013;14 Suppl 1(Suppl 1):S8. doi: 10.1186/1471-2105-14-S1-S8. Epub 2013 Jan 14. BMC Bioinformatics. 2013. PMID: 23369085 Free PMC article.
-
Dynamical models reveal anatomically reliable attractor landscapes embedded in resting state brain networks.bioRxiv [Preprint]. 2024 Jan 16:2024.01.15.575745. doi: 10.1101/2024.01.15.575745. bioRxiv. 2024. PMID: 38293124 Free PMC article. Preprint.
-
A highly efficient multi-core algorithm for clustering extremely large datasets.BMC Bioinformatics. 2010 Apr 6;11:169. doi: 10.1186/1471-2105-11-169. BMC Bioinformatics. 2010. PMID: 20370922 Free PMC article.
-
Re-investigation of functional gastrointestinal disorders utilizing a machine learning approach.BMC Med Inform Decis Mak. 2023 Aug 26;23(1):167. doi: 10.1186/s12911-023-02270-9. BMC Med Inform Decis Mak. 2023. PMID: 37633899 Free PMC article.
-
Shrinkage Clustering: a fast and size-constrained clustering algorithm for biomedical applications.BMC Bioinformatics. 2018 Jan 23;19(1):19. doi: 10.1186/s12859-018-2022-8. BMC Bioinformatics. 2018. PMID: 29361928 Free PMC article.
Publication types
MeSH terms
LinkOut - more resources
Full Text Sources
Other Literature Sources