Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
. 2005 May;14(5):1315-27.
doi: 10.1110/ps.041253405.

All are not equal: a benchmark of different homology modeling programs

Björn Wallner  1 , Arne Elofsson
Affiliations

All are not equal: a benchmark of different homology modeling programs

Björn Wallner et al. Protein Sci. 2005 May.

Abstract

Modeling a protein structure based on a homologous structure is a standard method in structural biology today. In this process an alignment of a target protein sequence onto the structure of a template(s) is used as input to a program that constructs a 3D model. It has been shown that the most important factor in this process is the correctness of the alignment and the choice of the best template structure(s), while it is generally believed that there are no major differences between the best modeling programs. Therefore, a large number of studies to benchmark the alignment qualities and the selection process have been performed. However, to our knowledge no large-scale benchmark has been performed to evaluate the programs used to transform the alignment to a 3D model. In this study, a benchmark of six different homology modeling programs- Modeller, SegMod/ENCAD, SWISS-MODEL, 3D-JIGSAW, nest, and Builder-is presented. The performance of these programs is evaluated using physiochemical correctness and structural similarity to the correct structure. From our analysis it can be concluded that no single modeling program outperform the others in all tests. However, it is quite clear that three modeling programs, Modeller, nest, and SegMod/ ENCAD, perform better than the others. Interestingly, the fastest and oldest modeling program, SegMod/ ENCAD, performs very well, although it was written more than 10 years ago and has not undergone any development since. It can also be observed that none of the homology modeling programs builds side chains as well as a specialized program (SCWRL), and therefore there should be room for improvement.

PubMed Disclaimer

Figures

Figure 1.
Figure 1.
Example of models produced with an alignment containing an error. (A) SWISS-MODEL model, (B) Backbone model, (C) Modeller model, (D) Native structure. The N-terminal helix in the backbone model is clearly wrong, since the distance between two adjacent residues is 40 Å. For the Modeller model this has no great impact (it is just one of many restraints); however, for the SWISS-MODEL model the error is enough to break the sheet in order to include the helix in the model. Figures were made using MOLSCRIPT (Kraulis 1991).
Figure 2.
Figure 2.
Histogram over the number of models that contain missing residues, i.e., where the program for some reason does not model all residues in the target sequence. SCWRL3 does not attempt to model loops; therefore, this number represents the alignments containing gaps.
Figure 3.
Figure 3.
Different measures used to assess the quality of the protein models. (A) RMSD values transformed using 1/(1 + RMSD) to avoid problem with high values. (B) MaxSub. (C) Backbone quality. (D) Side-chain quality as measured by fraction of correct side-chain torsion angles (χ1 and χ2). Error bars are constructed using standard error.
Figure 4.
Figure 4.
(A) Fraction of residues with “bad” bond angles, bond length side-chain planarity according to WHAT_CHECK for each method and also for the native structure. For a residue to be part of the any category it has to be classified as “bad” for any of the categories above. (B) The sequence identity dependence for the residues from the any “bad” category above. (C) Models with MaxSub score >0.6. (D) Acceptable model are models that have a MaxSub score of at least 0.6 and not more than 10% of its residues missing or with bad chemistry. 3D-JIGSAW and Builder have a significantly lower number of acceptable models and were removed for clarity.
Figure 5.
Figure 5.
Improvement over template as measured by (A) the difference between the fraction of models that gets significantly (ΔMX > 0.02) improved, fimp, and the fraction that gets significantly deteriorated, fdet, or by (B) the average fraction of models that gets improved, 〈fimp〉, and deteriorated, 〈fdet〉. 3D-JIGSAW and Builder were removed for clarity.
Figure 6.
Figure 6.
Two examples of a model that is improved upon modeling: in red, the template structure model is shown; in green, the final model; in blue, the native structure. (A) Modeller7v7 model of domain d1heia2 with d8ohm 2 as a template. The alignment contains no gaps, and the sequence identity between the target and template sequence is 91%. The RMSD between the template and the native structure is 2.06 Å, and for the Modeller7v7 model only 1.40 Å. The MaxSub score is also improved from 0.75 to 0.87. (B) 3D-JIGSAW model of domain d1unka with d1emva as a template. The alignment contains a single residue gap, and the sequence identity between target and template sequence is 58%. The RMSD between the template and the native structure is 2.05 Å, and for the 3D-JIGSAW model 1.30 Å.The MaxSub score is improved from 0.80 to 0.87. Figures were made using MOLSCRIPT (Kraulis 1991).
See this image and copyright information in PMC

Similar articles

See all similar articles

Cited by

See all "Cited by" articles

References

    1. Abagyan, R.A. and Batalov, S. 1997. Do aligned sequences share the same fold? J. Mol. Biol. 273 355–368. - PubMed
    1. Bates, P.A., Kelley, L.A., MacCallum, R.M., and Sternberg, M.J. 2001. Enhancement of protein modeling by human intervention in applying the automatic programs 3D-JIGSAW and 3D-PSSM. Proteins 45 (Suppl. 5): 39–46. - PubMed
    1. Blundell, T.L., Sibanda, B.L., Sternberg, M.J., and Thornton, J.M. 1987. Knowledge-based prediction of protein structures and the design of novel molecules. Nature 326 347–352. - PubMed
    1. Brenner, S.E., Chothia, C., and Hubbard, T. 1998. Assessing sequence comparison methods with reliable structurally identified evolutionary relationships. Proc. Natl. Acad. Sci. 95 6073–6078. - PMC - PubMed
    1. Brooks, B.R., Bruccoleri, R.E., Olafson, B.D., States, D.J., Swaminathan, S., and Karplus, M. 1983. CHARMM: A program for macromolecular energy minimization and dynamics calculations. J. Comput. Chem. 4 187–217.

Publication types

LinkOut - more resources