Get caught up! Start with Cochrane Analysis: What’s Here, What’s Missing, Conclusions
What has gone on before
Cochrane Reviews are well-known systematic reviews of studies in healthcare, and are internationally recognized — enough to be considered the last word in healthcare as far as some medical publications are concerned. In 2019, Larun et al. wrote a Cochrane analysis of exercise as a therapy for chronic fatigue syndrome entitled Exercise therapy for chronic fatigue syndrome. In response, I wrote an analysis of where the review went wrong. If you’d like to catch up, you can read the original article by clicking here: Cochrane Analysis: What’s Here, What’s Missing, Conclusions.
There were a number of concerning things about the review and its conclusions. These included:
- Studies only used the Oxford (1991) and Fukuda (1994) sets of criteria. Neither of these require PEM, now considered the hallmark symptom of the disease. Despite this omission, the reviewers claim the studies had low selection bias;
- A claim of low bias for the PACE trial publications;
- The sole inclusion of studies with subjective measures, and the omission of the few exercise studies that employed objective measures;
- The omission of post-treatment harms, pain, and quality of life in all but one study;
- Rating most exercise studies as having low or very low evidence, yet concluding that exercise “likely” was helpful;
- Most studies followed patients for a short period of time; this isn’t immaterial, since early interventions can be mistaken for cures in early improvers and since the placebo effect is strongest when an intervention is first administered.
Yet the report concluded that “Exercise therapy probably has a positive effect on fatigue in adults with CFS compared to usual care or passive therapies”, although they did at least add that they are ‘uncertain’ if the effect persists in the long-term — and later added that they could not be certain of the generalizability of the results for patients that met other sets of diagnostic criteria.
Setting the Stage
The question of graded exercise in ME could well be one of the more contentious medical issues of our time. The battle lines are drawn, with patients and biomedical researchers in one camp and psychiatrists in the other. What’s at stake is disabled people’s well-being: our own large survey of people with ME showed that over two-thirds of people with ME reported deterioration from graded exercise, and with the same therapeutic interventions being considered for Long COVID, the stakes are higher than ever. With lives (and funding) at stake, neither side is likely to back down without a fight.
Hilda Bastian is best known to the ME/CFS community for her science blog Absolutely Maybe. Her interest in the controversies surrounding CBT and GET for ME began with James Coyne, who criticized their methodology and encouraged her to look into them, along with Tom Kindlon’s comments on PubMed Commons, where she was Editor-in-Chief. Bastian’s background in healthcare consumer advocacy is also a strong motivator for her often lighthearted criticisms of PACE, punctuated with her trademark one-panel cartoons. Bastian wrote her first article on PACE called Consumer-Contested Evidence: Why the ME/CFS Exercise Dispute Matters so Much, and followed it up with It’s a Start: The Amended Version of the Cochrane Review on Exercise and CFS.
What you may not know about Bastian is that she is a founding member of the Cochrane Collaboration, the group that compiles the Cochrane Review, and has worked for the US government, leading the PubMed Health project.
Bastian forms The Cochrane re-analysis team
Cochrane tapped Bastian to lead an independent advisory group to look into the exercise review. She was dismayed to see that consumer concerns had not been meaningfully addressed and set out to challenge the status quo, suggesting changes in the typical structure and remit of the group. Once on board and considering Cochrane’s plans for the re-analysis, Bastian proposed an authorial mix of methodologists with no preconceptions about ME, so as to judge each study purely on its merits; health care providers who treat people with ME; and ‘consumers’: people with ME and their carers.
Unique aspects of the current process include:
- The group will create an ongoing analysis of community concerns about the current review, and suggestions for improving it. One of the working products of the process will be a paper from the IAG that will provide a framework for consultation in the coming months.
- The protocol is the roadmap of methods that the review committee will follow to find, select, analyze, and interpret the evidence. The group will create a new protocol, which will then be open for community feedback. There will be opportunity for stakeholders to raise their concerns in virtual meetings, as well as in writing, before the protocol is finalized and published.
- An important issue of community concern with this review was its original editorial group within Cochrane, and the review has been re-located.
Beyond securing Cochrane’s agreement to this new structure, Bastian’s most Herculean task was to find people from many sides of the debate (including ME agnostics), and bring them into a working group where the outcome could change the way that medical providers treat ME for good.
Without a mix of people with different backgrounds and views, the review could never be taken seriously by those institutions that need change. There’s no echo chamber here. In order to see movement, the results will need to be plausible in the eyes of a variety of audiences, and that requires a plurality reaching eventual consensus. That will likely mean compromises that displease people on both sides.
Bastian’s coordination and mediation experience will be put to the test!
You can find a list of the authors and committee members below, including our #MEAction representative, Director of Scientific and Medical Outreach, Jaime Seltzer.
authors
- Consumers:
Mary Dimmock (USA, carer, with ME/CFS activist background)
Kay Hallsworth (UK, person with ME/CFS, non-ME/CFS-activist background); - Clinicians:
Todd Davenport (USA, physiotherapy)
Julia Newton (UK, medical); - Systematic reviewers with relevant methodological expertise:
David Nunan (UK, experience with systematic reviews of exercise as an intervention)
Argie-Angeliki Veroniki (Greece, statistician); - Systematic reviewers from Cochrane response:
Hanna Bergman (Israel)
Nicholas Henschke (Germany, lead author).
Independent advisory board
- Nominees of three organizations representing people with ME/CFS:
Penelope McMillan (with Penelope Del Fante, and proxy, Geoffrey Hallmann) (Australia) – ME/CFS Australia;
Jaime Seltzer (USA) – #MEAction;
Samuli Tani (Finland) – European ME Alliance;
- Two people from a clinical background with active interest in ME/CFS:
Lily Chu (USA) (medical practitioner);
Peter Gladwell (UK) (physiotherapist);
- Individuals who had been involved in discussions with Cochrane preceding the IAG:
Hilda Bastian (Australia, Lead);
Mike Clarke (Ireland/UK, systematic review methodologist); and
George Faulkner (UK)
