Methods of model accuracy estimation can help selecting the best models from decoy sets: Assessment of model accuracy estimations in CASP11

Andriy Kryshtafovych, Alessandro Barbato, Bohdan Monastyrskyy, Krzysztof Fidelis, Torsten Schwede, Anna Tramontano

Research output: Contribution to journalArticlepeer-review

58 Scopus citations


The article presents assessment of the model accuracy estimation methods participating in CASP11. The results of the assessment are expected to be useful to both-developers of the methods and users who way too often are presented with structural models without annotations of accuracy. The main emphasis is placed on the ability of techniques to identify the best models from among several available. Bivariate descriptive statistics and ROC analysis are used to additionally assess the overall correctness of the predicted model accuracy scores, the correlation between the predicted and observed accuracy of models, the effectiveness in distinguishing between good and bad models, the ability to discriminate between reliable and unreliable regions in models, and the accuracy of the coordinate error self-estimates. A rigid-body measure (GDT_TS) and three local-structure-based scores (LDDT, CADaa, and SphereGrinder) are used as reference measures for evaluating methods' performance. Consensus methods, taking advantage of the availability of several models for the same target protein, perform well on the majority of tasks. Methods that predict accuracy on the basis of a single model perform comparably to consensus methods in picking the best models and in the estimation of how accurate is the local structure. More groups than in previous experiments submitted reasonable error estimates of their own models, most likely in response to a recommendation from CASP and the increasing demand from users.
Original languageEnglish (US)
Pages (from-to)349-369
Number of pages21
StatePublished - Sep 7 2015
Externally publishedYes

Bibliographical note

KAUST Repository Item: Exported on 2022-06-03
Acknowledged KAUST grant number(s): KUK-I1-012-43
Acknowledgements: Grant sponsor: US National Institute of General Medical Sciences (NIGMS/NIH); Grant number: R01GM100482; Grant sponsor: KAUST Award; Grant number: KUK-I1-012-43.
This publication acknowledges KAUST support, but has no KAUST affiliated authors.

ASJC Scopus subject areas

  • Medicine(all)


Dive into the research topics of 'Methods of model accuracy estimation can help selecting the best models from decoy sets: Assessment of model accuracy estimations in CASP11'. Together they form a unique fingerprint.

Cite this