Yesterday, Columbia University professor of statistics Andrew Gelman published a warning that The Lancet was risking its reputation by refusing to rectify errors in the main paper on the PACE trial that appeared in the journal in 2011.
In his article on a popular statistics blog, Professor Gelman described the PACE authors’ refusal to share data from the study as “unforgivable”. In a reference to the much-criticised “normal range” analyses that appeared in the Lancet paper, he said, “No paper with an analysis in which you can get worse and be counted as improved should ever be published.”
Professor Gelman said that despite this “absurd” analysis, PACE has been presented as “definitive” in  the scientific literature and by public health agencies. He said that he suspects that the trial has stood so long because “The Lancet’s brand name gave this paper a pass.”
He noted that although Lancet editor Richard Horton said that the paper had undergone “endless rounds of peer review”, it is flagged online as having been “fast-tracked”, a process that takes four weeks according to current Lancet policy.
Professor Gelman said that, regardless of whether the paper had been reviewed repeatedly or very little, typically “reviewers have neither access to the raw data nor the time for careful reanalysis” and that “mistakes are inevitable.” But, he said, “What I can’t excuse is the journal editor’s dogged defense of a flawed paper.”
He went on to argue that if a poor paper was allowed to hide behind a journal’s reputation then it was right that the journal’s reputation should be degraded. He said, “The Lancet editor is using his journal’s reputation to defend the controversial study. But, as the study becomes more and more disparaged, the sharing of reputation goes the other way.”
Professor Gelman described imagining a scientist in future saying, “The Lancet, eh? Isn’t that the journal that published the discredited Iraq survey, the Andrew Wakefield paper, and that weird PACE study?”
He concluded his article by saying, “Long run, reputation should catch up to reality. But before the long run comes, there are a few people out there with chronic fatigue syndrome who don’t feel like waiting.”
Professor Gelman recently published two other articles critical of PACE (here and here). He is one of a growing band of academics who have become interested in the shortcomings of the PACE trial since a series of critical blog posts by Dr. David Tuller, Professor James Coyne and others in recent months.
The PACE trial was a £5 million randomised trial whose authors claim that it showed that cognitive therapy and graded exercise are beneficial for chronic fatigue syndrome patients.