Dana Ullman's Homeopathy Studies Thread

I have had a quick look at the full article (Homeopathy in HIV infection: a trial report of double-blind placebo controlled study, Br Homeopath J. 1999 Apr;88(2):49-57.) I'm unable to post links yet, but the abstract is in Linda's post above mine.
This is not my field of expertise, but there are some obvious comments which spring to mind.
Firstly this is a small study. Two groups of 50 were studied, with the results from one being claimed to be significant. From this group, 12 patients dropped out (7 placebo, 5 treatment) leaving 18 patients receiving placebo, and 20 on active treatment.
However, the main issue is data interpretation. The trial comprised of randomising the patients and giving them either individualised homeopathic treatment or placebo. The outcome measurement was the CD4 count. Now, in this group, at the end of the trial the CD4 in placebo was 452 (SD 158 - units are not given) and in the treatment group 534 (SD 278) with a p value of 0.52 !!

The study is negative - there is no difference between placebo and treatment groups!

You may wonder then how the authors have managed to claim it as success for homeopathy. This assertion is based upon an increase in the CD4 count in the treatment group over the course of the trial, from 433 to 534 (again, no units provided.) The authors claim a p value of 0.008 using the Wilcoxon matched pairs Signed Rank test. Im afarid my stats knowledge is rudimentary so maybe others can help - but that does not strike me as the appropriate test for this sort of data - isn't Wilcoxon non parametric?

Overall, Im afraid this study strikes me as similar to others in this literature - the authors have such a bias to believing in homeopathy that they twist the results to produce the outcome they want. How you can publish a study with no difference between placebo and treatment groups and claim it is effective is just beyond belief.
 
Overall, Im afraid this study strikes me as similar to others in this literature - the authors have such a bias to believing in homeopathy that they twist the results to produce the outcome they want. How you can publish a study with no difference between placebo and treatment groups and claim it is effective is just beyond belief.

With the variation in the CD4 counts you have given, it is amazing that they can show any sig diff between anything. I would be interested in how they can show a p=.04 for differences between groups pre and post treatment.

As to the motivation in calling this a success, you only have to look at the journal title. They could have gained respect by publishing in a more main stream journal as a small trial yet again showing that homeopathy failed to demonstrate any effect.
 
I have had a quick look at the full article (Homeopathy in HIV infection: a trial report of double-blind placebo controlled study, Br Homeopath J. 1999 Apr;88(2):49-57.) I'm unable to post links yet, but the abstract is in Linda's post above mine.
This is not my field of expertise, but there are some obvious comments which spring to mind.
Firstly this is a small study. Two groups of 50 were studied, with the results from one being claimed to be significant. From this group, 12 patients dropped out (7 placebo, 5 treatment) leaving 18 patients receiving placebo, and 20 on active treatment.
However, the main issue is data interpretation. The trial comprised of randomising the patients and giving them either individualised homeopathic treatment or placebo. The outcome measurement was the CD4 count. Now, in this group, at the end of the trial the CD4 in placebo was 452 (SD 158 - units are not given) and in the treatment group 534 (SD 278) with a p value of 0.52 !!

The study is negative - there is no difference between placebo and treatment groups!

You may wonder then how the authors have managed to claim it as success for homeopathy. This assertion is based upon an increase in the CD4 count in the treatment group over the course of the trial, from 433 to 534 (again, no units provided.) The authors claim a p value of 0.008 using the Wilcoxon matched pairs Signed Rank test. Im afarid my stats knowledge is rudimentary so maybe others can help - but that does not strike me as the appropriate test for this sort of data - isn't Wilcoxon non parametric?

Overall, Im afraid this study strikes me as similar to others in this literature - the authors have such a bias to believing in homeopathy that they twist the results to produce the outcome they want. How you can publish a study with no difference between placebo and treatment groups and claim it is effective is just beyond belief.
Hmmm...
What were the CD4 counts for the groups at the start of the trial? When do the study authors decide to measure the final CD4 end point - at a fixed time point?
CD4 counts are very variable in patients with HIV, and therefore doctors tend to use trends rather than isolated levels to base treatment decisions on, simply because of this. CD4 percentages are often a better marker - less prone to "noise" in the system. Did the authors look at these?

The final arbiter of efficacy would be a delay in progression to clinical AIDS or a reduction in death rate. I presume this was not performed, although with such a small study this is perhaps not surprising.

What is surprising is that this study appears all over the Homeopathc literature like a rash, expounding the benefits of homeopathy in HIV. One would imagine that in the last 15 years the authors would have been able to verify their findings with a larger, better powered study which had more end points. Perhaps they have tried, but have failed? Knowing the HIV trial literature as I do, I can guarantee that a study of this nature, were it talking about the effect of a new HIV drug, would not be considered for publication in the biomedical literature that deals with HIV, other than as a preliminary proof of concept study which would require confirmation/validation.

I'd appreciate a link to the study when you can post these, or you could PM me the link - thanks.
 
Last edited:
You may wonder then how the authors have managed to claim it as success for homeopathy. This assertion is based upon an increase in the CD4 count in the treatment group over the course of the trial, from 433 to 534 (again, no units provided.) The authors claim a p value of 0.008 using the Wilcoxon matched pairs Signed Rank test. Im afarid my stats knowledge is rudimentary so maybe others can help - but that does not strike me as the appropriate test for this sort of data - isn't Wilcoxon non parametric?

Yes, it's a non-parametric test. The only justification for using it would be if the distribution of the CD4 counts is grossly different from a normal distribution and if the other methods of transforming the data (such as squaring, taking the log or square root, using the reciprocal) failed. But then the results of a goodness-of-fit test (to test for a normal distribution) on the original and the transformed data should be part of the results section. Since it is otherwise wrong to use the Wilcoxon test, you have to include proof that it is necessary to do so in this case. Else it just looks like you are data mining in order to find a way to support a conclusion that is unsupported by the results.

Linda
 
Overall, Im afraid this study strikes me as similar to others in this literature - the authors have such a bias to believing in homeopathy that they twist the results to produce the outcome they want. How you can publish a study with no difference between placebo and treatment groups and claim it is effective is just beyond belief.


I found this after reading a recent post on DC's blog. It's the next paper from the same issue as the paper DC was discussing. I've only seen the abstract (anyone got access to full text?), but, er, interesting conclusions in this one as well:

The effects of a 45x potency of arsenicum album on wheat seedling growth -- a reproduction trial
Binder M, Baumgartner S, Thurneysen A
Forsch Komplementarmed Klass Naturheilkd. 2005 Oct;12(5):284-91

This attempted to repeat the results of a trial published in 1997, in which (apparently - I have so far been unable to track down any other reference to the 1997 trial) it had been found that 45x arsenic produced increased growth in seedlings that had previously been poisoned* with arsenic.

CONCLUSION: The result of this replication trial is a reversal of the original study, since Arsenicum album 45x inhibited wheat shoot growth instead of enhancing it. Nevertheless, high homeopathic potencies may induce statistically significant effects in biological systems. However, the magnitude and direction of these effects seem to depend on yet unknown parameters.


"Unknown parameters"? Do they mean things other than the remedy administered?





*Won't anyone think of the seedlings?
 
Deetee,

In reply to your questions:
CD4 counts at the start of the trial: The 100 patients were in 2 groups; 1 with asymptomatic HIV infection, the other with persistent generalised lymphadenopathy. The authors claimed positve results only for the second group. The data are presented as means (SD). No units are given.
Placebo group CD4: 432 (146)
Treatment group: CD4 490 (229) p=0.6.

The trial was ceased after 6 months, at which time the results were:
Placebo CD4 452 (158)
Treatment CD4 534 (278)

No mention is made of trends or CD4 percentages and no other clinical endpoints were looked at.
Linda, in the stats section all the authors say is non parametric tests were used to obtain the significance level due to the non normal distribution of haematological and immunological test values.
 
The data are presented as means (SD). No units are given.
Placebo group CD4: 432 (146)
Treatment group: CD4 490 (229) p=0.6.

The trial was ceased after 6 months, at which time the results were:
Placebo CD4 452 (158)
Treatment CD4 534 (278)

Just eyeballing the figures they have SDs of about 50% in pre and post treatment groups and about a 10% diff between the means and they found a significant difference? I wonder how many stats tests they used until one gave them the result they needed.
 
I haven't got $30, I'm afraid.

What we appear to have is a trial that showed a rise of 8.9% in CD4 count in patients getting homeopathy compared to 4.6% rise in CD4 count among the controls. Sorry, these figures mean absolutely nothing clinically, even if they may be of borderline statistical significance.

I am used to HIV drug trials demonstrating rises in CD4 count from around 40/mm3 baseline to over 200 or so - that is in the region of >400% .
A drug reputedly causing a single figure percentage rise would be laughed out of town.
 
Last edited:
Deetee,

In reply to your questions:
CD4 counts at the start of the trial: The 100 patients were in 2 groups; 1 with asymptomatic HIV infection, the other with persistent generalised lymphadenopathy. The authors claimed positve results only for the second group. The data are presented as means (SD). No units are given.
Placebo group CD4: 432 (146)
Treatment group: CD4 490 (229) p=0.6.

The trial was ceased after 6 months, at which time the results were:
Placebo CD4 452 (158)
Treatment CD4 534 (278)

No mention is made of trends or CD4 percentages and no other clinical endpoints were looked at.

There's not enough information to do a paired t-test, but the 95% confidence intervals for before treatment are 390-590, and after treatment are 412-656. With that much overlap, I can't see how, even with the somewhat more sensitive paired test, it could possibly return a statistically significant difference.

So we're left with the Wilcoxon test returning a significant result on one of several comparisons. And you gotta wonder why one moment they're giving us information about means and standard deviations and the next moment they're saying that information can't be used, if it's not because they just liked the result better when they didn't.

Linda, in the stats section all the authors say is non parametric tests were used to obtain the significance level due to the non normal distribution of haematological and immunological test values.

Non-normality alone is not a sufficient excuse and parametric tests are robust even to moderate departures from normality. And summarizing the results with means and standard deviations tends to put the lie to that excuse.

But, even if we ignore all that in order to avoid any appearance of bias, as Deetee has already pointed out, a difference of that magnitude is too trivial to be useful.

Linda
 
Last edited:
There's not enough information to do a paired t-test, but the 95% confidence intervals for before treatment are 390-590, and after treatment are 412-656. With that much overlap, I can't see how, even with the somewhat more sensitive paired test, it could possibly return a statistically significant difference.

So we're left with the Wilcoxon test returning a significant result on one of several comparisons.

What's even more odd is to get a 'significant' result from a Wilcoxon test when the groups' data clearly overlap so much. Never mind the appropriateness of test choice, if a paired t-test can't yield a significant result then a non-parametric test would not...unless the data are wildly skewed so the SD of each group is created by a small number of huge values but all the rest of the data line themselves up neatly to have nearly all higher rank values in one group and lower rank values in the other.

Hmmm...
 
What's even more odd is to get a 'significant' result from a Wilcoxon test when the groups' data clearly overlap so much. Never mind the appropriateness of test choice, if a paired t-test can't yield a significant result then a non-parametric test would not...unless the data are wildly skewed so the SD of each group is created by a small number of huge values but all the rest of the data line themselves up neatly to have nearly all higher rank values in one group and lower rank values in the other.

Hmmm...

Yeah. I was trying to figure out how the data could be distributed such that a non-parametric test was actually more powerful than a parametric test (instead of the usual way around). I thought a platykurtic distribution may do the trick (if only to reduce the power of the parametric test). That would fit with my guess of what the distribution of CD4 counts would look like, given that CD4 counts and stage of HIV infection aren't completely independent so stratifying people by stages may have the effect of imposing cut-off points on the CD4 counts.

It's an interesting intellectual exercise, if nothing else.

Linda
 
Dana has just cited the Reilly allergy studies, the Diarrhea trials, The oscillococcinum trials, and the COPD study yet again over on the quackometer blog.

I'm getting a terrible feeling of deja woo.
 
From the other thread:

Mr. Duck first threatened to close off discussion just after I posted a link to the famous book that Dr. James Manby Gully wrote (in 5 editions). My link was to the 5th edition, where he stated quite emphatically that homeopathic medicines were effective in various chronic ailments, either alone or with water-cure.

Your statement that the duck-meister closed off discussion because there was "nothing new to say" was his and your clever spin, but not the truth. It IS that simple. And then, when I begun to discuss this excellent study in CHEST, Mr. Duck got more worried.

I'm sure he was quacking in his boots.

You folks spin so much and so often that you get yourselves dizzy.

As for the CHEST trial, why do you reference statistical significance when it works for you and ignore it when it doesn't? Please show me where there was a statistically significant difference in the treatment and control groups?

Well, if you set the standards so that you have a ninety-five percent chance of picking up any differences, it turns out that the differences between the treatment and control groups were statistically significant. ;)

But heck, let's agree that there were some (non-significant differences). Let's look at the results. Let's assume that everyone who had home oxygen were at Grade 3 prior to treatment (that's 5 treatment people and 9 control people)...and yet, after treatment, there was one ONE treatment person in Grade 3 and now 13 people of the control group in Grade 3.

You are mixed up. The 'Grade' referred to in the table of outcomes is a grading of the tracheal secretions. It has nothing to do with the stage of COPD.

Remember...COPD is a progressive disease, and yet, the treated group improve substantially (not just significantly).

How are you measuring improvement?

Please explain this...

And yet, I do continually go back to this study because THIS disease is a serious one; it is not self-limiting; it was conducted at the University of Vienna Hospital; it was published in a major journal. That's more than 3 strikes...you're out...

Puhleeze. You know full well that you go back to this study because the authors concluded that the treatment was effective. If it said otherwise, you'd be dropping it like a hot potato.

That said, the challenge to homeopaths is that we have to replicate this study, and I have heard that it is presently being replicated.

As for studies that have been replicated in homeopathy, there are those studies to which I have referred in the past that you love to ignore or minimize their real data...but all of your critiques of them are left wanting.

Of course you found them wanting. They showed that you were wrong.

Linda
 
This is a reply from here
But heck, let's agree that there were some (non-significant differences). Let's look at the results. Let's assume that everyone who had home oxygen were at Grade 3 prior to treatment (that's 5 treatment people and 9 control people)...and yet, after treatment, there was one ONE treatment person in Grade 3 and now 13 people of the control group in Grade 3.

Remember...COPD is a progressive disease, and yet, the treated group improve substantially (not just significantly).

Please explain this...
You don't seem to realise the implications of your comments.
If the two groups are different this study then becomes just one in a long series of before and after experiments, it has not increased anyone's understanding.

And yet, I do continually go back to this study because THIS disease is a serious one; it is not self-limiting; it was conducted at the University of Vienna Hospital; it was published in a major journal. That's more than 3 strikes...you're out...
Your baseball references are repetively tiresome, try something a little more interesting.

That said, the challenge to homeopaths is that we have to replicate this study, and I have heard that it is presently being replicated.

As for studies that have been replicated in homeopathy, there are those studies to which I have referred in the past that you love to ignore or minimize their real data...but all of your critiques of them are left wanting.
I've had to say this to other homeopaths and no doubt I'll be saying it again in the future.
You have a system of treatment that has no basis in science. In fact for it to occur at least one and probably more fully accepted and proven areas of science will have to be disproved. The attempts to do so have met with manifest ignominy:- Benveniste, R. Roy, flaking of silica from glass, memory of water.
Because of this absence of mechanism, you or some other homeopath has to come up with nearly incontrovertible proof that some effect happens. It has to be startling and beyond reproach. Nothing you have presented comes anywhere near this standard. Some of you have already realised that this is never going to happen and have tried to discredit the double blind random placebo controlled study with wrong headed applications of quantum physics. You personally have reverted to silly insults and others have cried about a conspiracy against homeopathy.
None of this cuts, show some real, positive proof along the lines of the request of BSM.
And please leave out the insulting premise that we are glued to existing theory. Most scientists I know would be fascinated to see existing theory overturned but the proof has to be very good, if it's not, your treament modality will treated not skeptically but with profound suspicion.
BSM's clock has been running for sometime now, personally I think it puts homeopathy in a good light, it really should have 200 years added to it.
 
Ive just been looking through the study, and something has struck me. In table 1, patient characteristics, the main difference, as has been noted is that 9 of the placebo group were receiving home oxygen, whereas only 5 of the treatment group were (25 in both group). No statistical analysis is provided for this comparison - there is no p value. I accept that p values are not designed to test for differences between the pre treatment groups but it would be interesting to see the likelihood of this particular grouping. Can any of our more statistically- savvy contributors comment?

My other thoughts can be adddressed by Mr Ullman:
Firstly, I often see that negative controlled trials are dismissed by homoepaths because the treatments were not individually tailored, and homeopathy is "holistic", and not constrained by our petty concepts of pathology. Yet this study was clearly not individualised,(after all the patients couldn't talk) so why is it valid?

Secondly I note in other threads that you decry criticisms of this study by saying it was published in a major journal: "Surely you don't know better than the editors of Chest??". Yet you seem to take a lot of time to rail against the Shang study. Surely you don't know better than the editors of The Lancet?
 
My other thoughts can be adddressed by Mr Ullman:
Firstly, I often see that negative controlled trials are dismissed by homoepaths because the treatments were not individually tailored, and homeopathy is "holistic", and not constrained by our petty concepts of pathology. Yet this study was clearly not individualised,(after all the patients couldn't talk) so why is it valid?

Secondly I note in other threads that you decry criticisms of this study by saying it was published in a major journal: "Surely you don't know better than the editors of Chest??". Yet you seem to take a lot of time to rail against the Shang study. Surely you don't know better than the editors of The Lancet?

Hey Criticalist,
Using a single homeopathic medicine without individualization for a specific pathology is only effective in rare and exceptional situations, such as Oscillococcinum within the first 48 hours of the flu, Kali bichromicum (potassium dichromate) in the treatment of thick tracheal discharges, and Arnica in certain post-surgical cases.

Shang's use of Oscillococcinum in a flu PREVENTION trial made by no sense to any homeopath, nor did the use of homeopathic doses of thryoid for weight-loss. Even one of the trials by Fisher using Arnica for long-distance running was ill-conceived (and Fisher has acknowledged this), especially since no previous small trial was conducted to evaluate if a larger trial was warranted.

The bottomline is that Shang's inclusion of only the large studies primarily included trials that had no external validity to homeopathic practice (or the homeopathic methodology).

It is interesting that Shang purposefully ignored analyzing the 21 high quality homeopathic trials and the 9 high quality allopathic ones, though this analysis will soon be published.
 
Remember...COPD is a progressive disease, and yet, the treated group improve substantially (not just significantly).

Yes, COPD is a progressive disease, but the study said nothing whatsoever about the progression of COPD over the study period or any other interval. Progression of COPD was not tested in the study, it would not have come to light incidentally during the study, it is not something on which this study allows you to comment.

But this is what you do all the time, take one tiny piece of information then build a great mountain of speculation on it. Even if that information was valid, the speculation is not.
 
Last edited:

ISF - Join now!

Every member here is approved by hand. No bots, no spam, just people who care about evidence and honest debate.

Membership is free!

Create your free account

Back
Top Bottom