Cryo-EM Ligand Modeling Challenge Results & Recommendations Published

June 25, 2024: A paper co-authored by all organizers and participants in the EMDataResource Ligand Challenge has now been published in Nature Methods: Outcomes of the EMDataResource Cryo-EM Ligand Modeling Challenge. The paper describes the results of our 2021 Challenge and recommends best practices for assessing cryo-EM structures of liganded macromolecules reported at near-atomic resolution.

Abstract

The 2021 EMDataResource Ligand Model Challenge aimed to assess the reliability and reproducibility of modeling ligands bound to protein and protein/nucleic-acid complexes in cryogenic electron microscopy (cryo-EM) maps determined at near-atomic resolution (1.9-2.5 Å). Three published maps were selected as targets: E. coli beta-galactosidase with inhibitor, SARS-CoV-2 virus RNA-dependent RNA polymerase with covalently bound nucleotide analog, and SARS-CoV-2 virus ion channel ORF3a with bound lipid. Sixty-one models were submitted from 17 independent research groups, each with supporting workflow details. The quality of submitted ligand models and surrounding atoms were analyzed by visual inspection and quantification of local map quality, model-to-map fit, geometry, energetics, and contact scores. A composite rather than a single score was needed to assess macromolecule+ligand model quality. These observations lead us to recommend best practices for assessing cryo-EM structures of liganded macromolecules reported at near-atomic resolution.

Recommendations Summary

1. For ligand-macromolecular complexes, the macromolecular model should be subject to standard geometric checks as done for X-ray crystallographic based models. These include standard covalent geometry checks and MolProbity evaluation, including CaBLAM, clashscore. Sugar pucker and DNATCO conformational analysis should be checked for nucleic acid components. The macromolecular model-map fit should be evaluated by EM Ringer, Q score, and FSC. Serious local outliers usually indicate an incorrect local conformation.
2. Ligands in macromolecular complexes should conform to standard covalent geometry measures (bond lengths, angles, planarity, chirality) as recommended by the wwPDB validation report. Additional checks that should be applied to ligands include fit to density using methods applicable to cryo-EM such as Q-score, occupancy (density strength, both absolute and relative to surroundings), and identification of missing atoms, including any surrounding ions. Ligand energetics should also be examined (e.g., strain energy).
3. The detailed interaction of the ligand with its binding site is of great importance and should be assessed by several independent metrics. Pharmacophore modeling is an optimized and time-tested energetic measure for how well the site would bind the specific ligand. LIVQ scores (Ligand+Immediate enVironment Q scores), introduced here, measure the density fit of the surrounding residues as well as the ligand itself. Probescore both quantifies and identifies specific all-atom contacts of H-bond, clash, and van der Waals interactions. All three types of measures should be taken into account. If the ligand model shows only weak interaction with its environment, the model is not right.
4. Future cryo-EM Model Challenges should be organized similarly to the well-established CASP and CAPRI challenge events of the X-ray crystallography and prediction communities, with incorporation of automated checks and immediate author feedback on all model submissions. A future Challenge might focus on validation of RNA models, including identification of RNA-associated ions, owing to the rapidly rising numbers of RNA-containing cryo-EM structures.