The Lancet paper by Shang et al.
In August 2005, the Lancet published a study1 which was reported as having compared 110 similar trials on homeopathy and conventional medicine, and reached the conclusion that homeopathy is no better than placebo.
An accompanying editorial by The Lancet editor entitled ‘The end of homoeopathy’2 lead to widespread media attention.
However, the conclusions of this study were in fact based on only 8 of the 110 trials, none of which involved usual homeopathic care. Furthermore, if you switch just one of the 8 trials they picked from the 110 for a different one, the results are reversed, showing that homeopathy works beyond placebo.3
This demonstrates that the findings of this paper are completely unreliable.
The sequence of events surrounding this issue of The Lancet, the political controversy which followed and key flaws of the study itself, are summarised in ‘Homeopathy and the Lancet‘.3
A paper with continued global impact
The Shang et al. paper has become notorious because, despite such fundamental flaws, it continues to be mis-quoted to this day as definitive ‘proof’ that homeopathy doesn’t work e.g.:
“Over 150 trials have failed to show it works”
Dr Ellie Cannon – resident GP for the Mail on Sunday. ‘Health Comment’, Mail on Sunday, 14 September 2014
“We’ve had over 100 trials of homeopathy now. For any medical treatment, after 100 trials have failed to give good overall evidence of benefit, this is the point at which a sensible person, with no vested interests in that one particular treatment, would loudly and clearly state: “No more money should be spent researching this blind alley”.
Ben Goldacre – British physician, academic and science writer.‘Pharma Chameleon’, New Europe, 17 April 2011 – Issue 93.
Results of the Shang paper
Shang et al.’s findings as reported in the abstract of the paper are:
110 homoeopathy trials and 110 matched conventional-medicine trials were analysed. The median study size was 65 participants (range ten to 1573). 21 homoeopathy trials (19%) and nine (8%) conventional-medicine trials were of higher quality. In both groups, smaller trials and those of lower quality showed more beneficial treatment effects than larger and higher-quality trials. When the analysis was restricted to large trials of higher quality, the odds ratio was 0·88 (95% CI 0·65–1·19) for homoeopathy (eight trials) and 0·58 (0·39–0·85) for conventional medicine (six trials).
Note: These ‘odds ratios’ mean that, for these particular 8 trials of homeopathy, if you analyse them together, the results suggest that the effects seen are due to placebo rather than a real clinical effect.
How reliable are these results?
The role of the Shang paper in the homeopathy debate cannot be overstated, so the issue of quality and reliability of the paper is paramount.
Unfortunately, the poor scientific quality of the paper is obvious, having been described by experts in the field of homeopathy research3,4,5,6,7 as well as independent researcher Prof Hahn, who highlights the main problem:
To conclude that homeopathy lacks clinical effect, more than 90% of the available clinical trials had to be disregarded.8
Relevance of Shang’s 2005 paper in 2015
Reliability of the analysis is not the only problem with the Shang paper. As we take a fresh look at the evidence in 2015, we also need to consider how well this study reflects the entirety of today’s evidence base.
A rigorous study by Mathie et al. published in 2014, found that homeopathic medicines, when prescribed during individualised treatment, are 1.5- to 2.0-times more likely to have a beneficial effect than placebo.13
This study includes 151 placebo-controlled randomised trials – 41 more than Shang’s team identified in 2005, but which would have met their inclusion criteria if they had been available at the time.
This demonstrates the extent to which the 10 year-old Shang et al. paper, which now covers only 73% of the eligible trials, has been superseded.
Read HRI’s brief summary of Mathie et al.’s study or listen to Robert Mathie presenting his findings at the HRI Rome 2015 conference.
Shang et al. key facts