The Australian Report - evidence of serious procedural and scientific misconduct in producing this report
The Australian report
In March 2015, the Australian National Health and Medical Research Council (NHMRC) published an Information Paper on homeopathy, commonly referred to as ‘The Australian Report’.1
This document concludes that “…there are no health conditions for which there is reliable evidence that homeopathy is effective”.
This report triggered headlines around the world suggesting NHMRC had found that homeopathy doesn’t work for any condition.3
Australian Report key facts
An extensive detailed investigation by Gerry Dendrinos of the Australian Homeopathic Association (AHA) into NHMRC’s conduct, combined with an in-depth scientific analysis of the report by HRI, revealed evidence of serious procedural and scientific misconduct in producing this report:
Complaint submitted to Commonwealth Ombudsman
In August 2016 HRI’s in-depth scientific analysis was used as part of a submission of complaint to the Commonwealth Ombudsman brought by Complementary Medicines Australia, Australian Homoeopathic Association and Australian Traditional Medicine Society.
Download the Executive Summary of the Ombudsman submission here.
Rachel Roberts, Chief Executive of HRI says “NHMRC’s review is just bad science. Decision-makers and the scientific community rely on these kinds of reports and need to trust their accuracy. This is not about anyone’s personal opinion as to whether homeopathy works or not. It is about the importance of evidence being reported objectively, whatever it says, and the NHMRC did not do that.”
As the complaint is ongoing, our full analysis – some 60 pages – cannot be shared as yet, but HRI’s data provided details demonstrating the following scientific failures by NHMRC which necessitate retraction of the Australian Report:
The complainants are now waiting to hear back from the Ombudsman regarding their submission. As NHMRC’s inaccurate Homeopathy Review has had a significant impact on the field of homeopathy research worldwide, HRI will share any news regarding the complaint as the case progresses.
Roberts says: “The public has a right to know that there are high quality studies showing homeopathy works for some medical conditions, such as hay fever, sinusitis and diarrhoea in children – information that was lost only due to NHMRC’s mishandling of the evidence. If the evidence on conventional medicine was treated this way there would be an outcry – and rightly so. NHMRC’s job was to accurately summarise the body of evidence for homeopathy for the public, a task in which they categorically failed.”
The missing first report
NHMRC’s investigation into Homeopathy ran from 2010 to 2015. NHMRC initially worked with an external contractor – from April 2012-August 2012 – to produce a review of the evidence on Homeopathy to inform the Australian public.
The report produced was called ‘A Systematic Review of the Evidence on the Effectiveness of Homeopathy’.
This review, paid for by Australian tax payers, was never made public and NHMRC continue to refuse to release it, despite repeated Freedom of Information requests.
After terminating the contract with the first review team, a second external contractor – OptumInsight – was hired to do the Homeopathy review again from December 2012-March 2015.
HRI is not alone in criticising the accuracy of NHMRC’s findings
FOI requests have brought to light that two independent experts also raised concerns over the conclusions of the 2015 report during peer review, prior to final publication. The Australasian Cochrane Centre commented that for some conditions, “…. ‘no reliable evidence’ does not seem an accurate reflection of the body of evidence”; a second expert felt “uncertain of the definitive nature of the Report’s conclusions”. NHMRC chose not to act on this feedback and did not amend their conclusions.
The real story behind the headlines
The Information Paper is designed for the general public and aims to provide a summary of the findings of a review of systematic reviews, carried out by NHMRC to assess the evidence base for effectiveness of homeopathy in humans.2
Confusing ‘lack of evidence of effect’ with ‘evidence of a lack of effect’
The Report’s conclusion that there is ‘no reliable evidence’ that homeopathy works, has been widely misunderstood, with people believing it meant that the NHMRC found that homeopathy does not work for the conditions tested, which would be a completely different result.
This misunderstanding triggered widespread media coverage, propagating the inaccurate story that the NHMRC found homeopathy to be no better than placebo for all the conditions.
In fact, NHMRC concluded that homeopathy worked no better than placebo for only 13 of the 61 health conditions they investigated.
HRI’s in-depth scientific analysis of the Australian Report revealed multiple examples of bias and misreporting which explain how NHMRC arrived at such a definitive and negative position, at odds with the conclusions of the majority of other more academically rigorous systematic reviews and meta-analyses on homeopathy.
Most crucially, NHMRC’s findings hinge primarily on their definition of reliable evidence: for a trial to be deemed ‘reliable’ it had to have at least 150 participants and a quality score of 5/5 on the Jadad scale (or equivalent on other scales). Trials that failed to meet either of these criteria were dismissed as being of ‘insufficient quality and/or size to warrant further consideration of their findings’.
Setting such a high quality threshold is very unusual, but the N=150 minimum sample size criterion is arbitrary, without scientific justification, and unprecedented in evidence reviews.
Out of 176 individual studies the NHMRC included in the homeopathy review, only 5 trials met their definition of ‘reliable‘, none of which, according to their analysis, demonstrated effectiveness of homeopathy. This explains why NHMRC concluded there is ‘no reliable evidence’ that homeopathy is effective.
For context it is worth remembering that if it was indeed the case that ‘no reliable evidence’ existed for homeopathy, it would put homeopathy in the same evidence category (“unknown effectiveness”) as 46% of conventional treatments used in the NHS, but in fact this conclusion is inaccurate.
Contrary to NHMRC’s findings, there are ‘good quality, well-designed studies with enough participants for a meaningful result‘ (to use NHMRC’s description of a reliable study) which show that certain homeopathic treatments are effective for certain conditions such as hay fever, sinusitis, upper respiratory tract infections, diarrhoea in children and lower back pain. The fact that the results of such studies were unjustifiably dismissed means that NHMRC have misled the public by misreporting the evidence for effectiveness of homeopathy.
Find out more on our Australian Report FAQs page
Nathalie Parmentier is Klassiek Homeopaat en Orthomoleculaire Therapeut in Laren en omgeving en Amsterdam