The British Medical Journal publication “BMJ Evidence-Based Medicine” unveiled an analysis signed by two investigative journalists and a gynecologist based in Switzerland. The paper brings to light flaws that void the positive conclusions reached by Cochrane. This leaves unanswered the question of HPV vaccine’s efficacy.
The BMJ Evidence-Based Medicine article, signed by investigative reporters Catherine Riva and Serena Tinari with MD Jean-Pierre Spinosa, highlights ethical and methodological flaws underlying the Cochrane review of the HPV vaccine. Here are some of the most serious:
1. The studies’ quality was not properly assessed;
2. The post-hoc subgroup analyses were presented as RCT results;
3. The reporting bias was not acknowledged;
4. Selective reporting was not taken into consideration;
5. The trial designs were biased;
6. The unpublished data were not included; and
7. Some authors had conflicts of interest.
Riva, Tinari and Spinosa’s analysis entails unpublished information obtained by the journalists via a Freedom of Information Act (FOIA) request from the Food and Drug Administration, as well as exclusive tables. These industry and regulatory data document that HPV vaccine’s studies were affected by “outcome switching” and “post-hoc group analysis,” two highly controversial practices known to impact studies’ results.
The Cochrane review authors did not mention nor consider these serious limitations. This is troubling, as it reveals that Cochrane based its “high certainty” conclusion (that HPV vaccines are effective) on flawed science. According to evidence-based medicine, such methodological problems void Cochrane’s positive conclusions on HPV vaccine’s efficacy.
Riva, Tinari and Spinosa ultimately decided to submit their analysis to BMJ Evidence-Based Medicine, having tried for years—unsuccessfully—to make Cochrane aware of the flaws in its HPV vaccine review protocol as well as in the review as it was finally published in 2018. Through Cochrane’s established platforms, five letters and two analyses about these issues were indeed addressed since 2012 to the Cochrane HPV vaccine review group, all of them co-signed by Canadian researchers A. Lippman, P. Biron, G. Rail, A. Taillefer, L. Spring, N. Arya and F. Turcotte.
BMJ Evidence-Based Medicine published Riva, Tinari and Spinosa’s findings, complete with all data, tables and references. The authors chose the open source platform Zenodo.org to also make available their correspondence with Cochrane as it provides evidence that this organization had many opportunities to make use of the findings. Such shortcomings are puzzling as Cochrane’s meta-analyses are considered the gold standard to assess public health interventions’ benefits and risks. Its reviews are supposed to apply evidence-based medicine methods on the best available evidence, and adhere to strict ethical guidelines. The analysis published by BMJ Evidence-Based Medicine establishes that regarding the HPV vaccine, Cochrane failed to respect its basic principles.
Riva C, Tinari S, Spinosa JP. Lessons learnt on transparency, scientific process and publication ethics. The short story of a long journey to get into the public domain unpublished data, methodological flaws and bias of the Cochrane HPV vaccines review. BMJ Evidence-Based Medicine Published Online First: 06 December 2018. doi:10.1136/bmjebm-2018-111119
About the Journal
BMJ Evidence-Based Medicine publishes original evidence-based research, insights and opinions on what matters for health care. The journal focuses on the tools, methods, and concepts that are basic and central to practicing evidence-based medicine.
About the Authors
Catherine Riva (CR) and Serena Tinari (ST) are freelance investigative journalists. They co-founded Re-Check, a non-profit organization to research and map health affairs. Jean-Pierre Spinosa (JPS), a gynaecologist and surgeon, is a lecturer at the University of Lausanne (Faculty of Medicine). CR and JPS coauthored a book on HPV vaccination (‘La piqûre de trop?’, Xenia, 2010). CR published four investigations on Swiss media on the HPV vaccine. ST, co-chair of the Swiss network of investigative journalists and ICIJ member, authored three Swiss National TV documentaries on the HPV vaccine.