Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
. 2014 Feb;82 Suppl 2(0 2):127-37.
doi: 10.1002/prot.24391. Epub 2013 Nov 22.

Assessment of protein disorder region predictions in CASP10

Bohdan Monastyrskyy  1 , Andriy Kryshtafovych, John Moult, Anna Tramontano, Krzysztof Fidelis
Affiliations

Assessment of protein disorder region predictions in CASP10

Bohdan Monastyrskyy et al. Proteins. 2014 Feb.

Abstract

The article presents the assessment of disorder region predictions submitted to CASP10. The evaluation is based on the three measures tested in previous CASPs: (i) balanced accuracy, (ii) the Matthews correlation coefficient for the binary predictions, and (iii) the area under the curve in the receiver operating characteristic (ROC) analysis of predictions using probability annotation. We also performed new analyses such as comparison of the submitted predictions with those obtained with a Naïve disorder prediction method and with predictions from the disorder prediction databases D2P2 and MobiDB. On average, the methods participating in CASP10 demonstrated slightly better performance than those in CASP9.

Keywords: CASP; assessment of disorder prediction; intrinsically disordered proteins; prediction of disordered regions; unstructured proteins.

PubMed Disclaimer

Figures

Figure 1
Figure 1
Distribution of disordered regions of different lengths in CASP10 targets. Internal regions are shown with darker shades.
Figure 2
Figure 2
Fraction of disordered residues at the termini of CASP10 targets. L is the target length (in number of residues).
Figure 3
Figure 3
Performance of groups as binary disorder/order classifiers according to the MCC, Acc, and precision scores. The error bars indicate boundaries of the 95% confidence intervals for each measure. Groups are ordered according to decreasing MCC score.
Figure 4
Figure 4
(A) ROC and (B) PR curves for the probability-based disorder region predictions for all CASP10 groups. The three groups with the atypical ROC curves in (A) used only a very limited number of disorder probability values: Groups G167 and G168 used only two different values (one for ordered and another for disordered residues); Group G115 used only five different values (two for ordered and three for disordered residues, and two out of the five values were only assigned to a very small number of residues). Groups in the legend are sorted according to decreasing AUC_ROC score.
Figure 5
Figure 5
Comparison of group performance on the full-length and termini-trimmed targets according to the (A) MCC and (B) AUC scores. Scores on the trimmed targets are marked as “internal.”
Figure 6
Figure 6
Comparison of group performance for four different thresholds of the minimum length of disordered regions. The two panels show data for two different evaluation measures (MCC and AUC). Each group is shown with a different color; groups in the legend are sorted according to the AUC score (across and then down); the artificial average group (“AVRG,” black thicker line) is added to the graph for reference.
Figure 7
Figure 7
Comparison of performance for the best 12 groups in the last four CASPs according to the MCC (A) and AUC (B) scores. Groups in each panel and are sorted according to decreasing scores in each CASP. CASP8* scores are calculated without a very long and completely unstructured target T0500, the correct prediction of which would over-inflate the evaluation scores.
See this image and copyright information in PMC

Similar articles

See all similar articles

Cited by

See all "Cited by" articles

References

    1. Romero P, Obradovic Z, Kissinger C, Villafranca JE, Dunker AK. Identifying disorder regions in proteins from amino acid sequence. Proc IEEE Int Conf Neural Netw. 1997;1:90–95.
    1. Wright PE, Dyson HJ. Intrinsically unstructured proteins: reassessing the protein structure-function paradigm. J Mol Biol. 1999;293:321–331. - PubMed
    1. Dunker AK, Obradovic Z, Romero P, Garner EC, Brown CJ. Intrinsic protein disorder in complete genomes. Genome Inform Ser Workshop Genome Inform. 2000;11:161–171. - PubMed
    1. Tompa P. Intrinsically unstructured proteins. Trends Biochem Sci. 2002;27:527–533. - PubMed
    1. Uversky VN. A decade and a half of protein intrinsic disorder: biology still waits for physics. Protein Sci. 2013;22:693–724. - PMC - PubMed

Publication types

MeSH terms

Substances

LinkOut - more resources