Some protein structures are good, others less so. Here's an all time classic that explains how we're supposed to tell the two apart:
Klywegt G. & Brunger, A. "Checking your imagination: applications of the free R value". Structure, 4(8) 897- 904 (1996)
Gerard and Axel "review applications of the free R value, and discuss practical issues and caveats related to the use and interpretation of this statistic." Then they go on to "... present a survey of free R values of published X-ray crystal structures of macromolecules." Dry stuff, I admit it. Mind you though, checking data quality and putting your own data in context with everything else that's out there is a key element of what we do. And it's a reminder of what we need to do to make an impression that lasts for generations to come. In light of the recent fabrications our field has seen I have come to embrace such self-policing efforts that can hardly be valued enough.
Here's an interesting vignette about the quality of protein crystal structures (Acta Cryst (2007)D63, 941-950). Brown and Ramaswamy conclude their retrospective analysis in the PDB hat protein structures published in high impact journals are worse than those in low-impact journals. Here it is in their own words:
" The most striking result is the association between structure quality and the journal in which the structure was first published. The worst offenders are the apparently high-impact general science journals."
Wow. How can that be? My own explanation is that sometimes high-impact structures, published in high-impact journals, are determined and authored by researchers that have little experience in X-ray crystallography as they have focused their work on very difficult biology / biochemistry projects for a long time. The X-ray diffraction & structure determination is then done with simple-to-use software tools, enabling biochemists that have not even yeard of Bragg's law to 'do a protein structure'.
Take my room mate for example, with whom I shared 10 stressful days at Cold Spring Harbor, trying to learn the basics of Macromolecular Crystallography back in 2000. On the last day at BNL we walk by a beamline that has the cover of Cell posted on their hutch - that was his structure.
Beware of protein structures in NATURE and SCIENCE? The lower the impact factor the better the structures.
I'm not saying that there's anything wrong with newcomers publishing protein structures - don't get me wrong here! The point is that high-impact structures are often the result ofbreakthroughs in biochemistry and crystal growing rather than advances in structure determination technologies. And therefore researchers trained in biochemistry get to determine structures to get their first author paper. Researchers with little experience in X-ray crystallography. The solution to this? Get a buddy. Make friends with a crystallographer who has seen the pitfalls of interpreting noisy maps, overrefining and trusting output file statistics. In my view, having a seasoned crystallographer co-authoring a structure paper adds to credibility. More so than the name of the publishing journal.
<this is where the original post ended. Thanks to Roger, here's a twist to this interesting story:
Turns out that when several factors (such as size, resolution, asymmetric unit volume & deposition date) are matched in controls and are corrected for, the quality of protein structures published in high impact journals is not much worse than those published other journals as described by
Randy Read and Gerard Kleywegt who followed up on their earlier research and show in Case controlled structure validation Acta Crystallogr D Biol Crystallogr. 2009 February 1; 65(Pt 2): 140-147 this table:
Only slightly poorer validation statistics of protein structures published in high vs. those in low impact journals. Take into account size, resolution, asymmetric unit volume & deposition date (done by matching controls) and the differences disappear.(Original Table)
Higher impact journals seem to publish protein structures with lower resolution, but the effect is small. What do you think? Does that bother anyone? (original image)
They show that there are serveral factors that were not analyzed in the earlier study -e.g. the size of the structure, date of publication etc. - and once included, the effect 'high-impact journals having lower quality protein structures' became very small.
Thanks for pointing out this paper, Roger! >