Homeopathy: Rustum Roy drops the bomb

[pgwenthold]

Originally Posted by Cuddles

Which is the whole point. There is not a large peak in the absorption of ethanol. There is one in the data Roy presents. Therefore he was not using pure ethanol.

To ask the same question that I asked JJM: on what basis do you call the peak in Roy's spectrum to be "large"? As far as we can tell, it is on the scale of 1% of the max, which would make it a minor peak.

How big does a peak have to be to be significant?

You make this type of objection, and Roy's response is just that it is too small to have been considered by the folks at Aldrich. He could be right. Now you have to go and ask the people at Aldrich if they ever blew their spectra up far enough to see if there might be a small peak at 324 nm, and their answer might be no. All this and something might come out of it. Maybe.

Meanwhile, there are massive problems with the data and study that you could drive a truck through.

[JJM]

Originally Posted by wollery

Watch out, they'll be comparing the size of their burettes in a minute!

I should have followed my better judgment and kept quiet.

BTW, I am retired, so I don't have a burette any more. You know the saying- if you don't use it you lose it.

[wilsontown]

Heh, my blog entry about the graphs got mentioned on Bad Science and Phillip Ball's blog.

Fame at last.

[Pipirr]

I missed this at the time, but Rustum Roy posted a comment on Philip Ball's article in Nature, "Here lies one who's name was writ in water".

Link to the comments section here.

Here's a choice quote:

Quote:

{snip}While I have never been involved in any way, intellectually, financially or personally—unlike Charles Darwin—with homeopathy, our water structure group contributed one of the papers (1) based on our rather extensive work (2) on the radically different Materials Science view of the “structure” of water, and its peripheral relation to homeopathy.

{snip}...Ball cites mostly the single paper critical paper from this issue, with not one word on either our paper, with the first two major papers referenced therein, or to Martin Chaplin’s introductory paper. Chaplin is clearly the most prominent figure in his field.

Herein Ball joins far too may scientists in “a much more dangerous, and ludicrous, practice condemned by our greatest scholars from Aristotle to Whitehead, of requiring an “explanation” or “theory” before accepting and dealing with solid, repeatable facts and observations which remain the only basis of real science.

Good lord.

Did you spot the little dig there at Charles Darwin? There's also a comment about a "second-tier hired magician in disguise." No prizes for guessing to whom that refers.

Anyway, he's complaining that Philip Ball made no substantive comments on the experimental papers, and argues that scientists demand a theory for homeopathy over the experimental evidence for it.

The evidence, of course, is what this thread is all about.

[Pipirr]

Originally Posted by wilsontown

Heh, my blog entry about the graphs got mentioned on Bad Science and Phillip Ball's blog.

Fame at last.

Here's the link to Philip Ball's blog:

http://philipball.blogspot.com/2007/...hy-theres.html

[wollery]

Originally Posted by pgwenthold

To ask the same question that I asked JJM: on what basis do you call the peak in Roy's spectrum to be "large"? As far as we can tell, it is on the scale of 1% of the max, which would make it a minor peak.

How big does a peak have to be to be significant?

You make this type of objection, and Roy's response is just that it is too small to have been considered by the folks at Aldrich. He could be right. Now you have to go and ask the people at Aldrich if they ever blew their spectra up far enough to see if there might be a small peak at 324 nm, and their answer might be no. All this and something might come out of it. Maybe.

Meanwhile, there are massive problems with the data and study that you could drive a truck through.

It was significant enough for him to point it out as an important difference between the "pure" ethanol spectrum and the homeopathic solutions! Or did you think the dashed line was there for amusement purposes?

[Rolfe]

Originally Posted by pgwenthold

But no "description" of a UV spectrum is going to mention a minor bump at 320 nm, even if it is there. Note that this supposed feature is claimed to be minimally detectable, which means that the extinction coefficient is probably less than 1/1000th of that at lambda max, if that. You can't find this in a general description of the spectrum (just as a description of the vibrational spectrum won't mention every single bump, either)

You'd really need to see the spectrum to examine it properly.

Originally Posted by JJM

Where did you get this idea? Look at the "standard" spectrum in Roy's paper, it is not a minor bump at 324 nm. And the absorbance below that is way above the reported spectrum. You need considerable experience with spectra before you can argue with us about them. If you know what you are doing, you can get a lot of information from the published descriptions of spectra; otherwise it would be pointless to publish them.

Also, comparing your own spectra to his is a meaningless exercise since you don't know what impurities are causing his results. We know he is not using "spectral grade" ethanol. All you could say is yours look like his, or not.

Originally Posted by pgwenthold

Here is the Aldrich description of the spectrum of spectro grade ethanol

UV absorption λ: 205 nm Amax: 1.00
λ: 210 nm Amax: 0.65
λ: 220 nm Amax: 0.35
λ: 255 nm Amax: 0.04
λ: 300-400 nm Amax: 0.01

The stuff from 300 - 400 nm is 1% of the intensity at the 205 max. That IS a minor bump in the spectum.

Boys, just calm down a minute. I don't entirely see where the disagreement is. Pgwenthold, as far as I can make it out, the data you quote absolutely back up what JJM is saying.

Remember, this isn't a spectrum of a solute we're looking at here - it's the spectrum of the solvent. Thus, concentration isn't a variable. Thus, the actual absorbance readings themselves are absolute values and can be compared.

The absorbance at 205, 210 and even 220 isn't the issue. These aren't peaks Aldritch are defining, they're just points on a curve, and they're giving the maximum absorbance you can expect from their pure product at those wavelengths. Below 250 everything is just shooting up towards opacity, as all these solvents do - they transmit pretty minimally at low-UV wavelengths. What Aldritch are showing is what level of background absorbance you're going to get from this solvent at these wavelengths, and how this increases steeply as you get into the low-UV wavelengths. JJM pointed this out here, which is the post that caused the penny to drop with me.

Maybe this page makes it clearer. Look at the table (about half way down) in the section headed "solvent cut-off wavelengths", which quotes wavelengths above which the solvents are considered to be suitable for UV spectroscopy, because they don't absorb too much.

Quote:

In this table, approximate wavelengths (nm) are specified below which the solvent absorbance may be unacceptable. For quantitative work, the cut-off may be set at a wavelength (L0) where the absorbance for 10 mm pathlength of the solvent exceeds 0.05 absorbance unit (relative to water), i.e. A1 cm > 0.05. For qualitative work, it may still be feasible to work at significantly lower wavelengths and most analysts accept a cut-off based on the wavelength (L1) for A1 cm > 1.0. However, if the UV absorption curve rises steeply, the accessible wavelength range may not be greatly extended.

In this context they quote 240nm as the lowest useful wavelength for ethanol for quantitative work, but say it may be practical down to 205nm for qualitative work. Note how very steep that rise is - from pretty minimal to a high absorbance reading (usually, 1.2 to 1.5 absorbance units is considered to be as high as you can practically go without diluting the solution, as the logarithmic scale makes anything above that very imprecise - basically, you're tending towards opacity) in only 35nm. It's this rise that Aldritch's data are describing more exactly.

In effect, the part of the spectrum we're actually interested in is the 250 to 400nm range. Above that (in the visible region) nothing seems to be absorbing significantly, and below that anything in ethanol and certainly all of Roy's samples are in the "rising steeply" phase where no useful information is going to be obtained.

What we do know for sure from the Aldritch data (and from the NPL page) is that ethanol does not absorb significantly in this range. The NPL table shows <0.05 units for everything above 240nm, and the Aldritch data records that we're down to 0.04 by 255nm, for spectroscopic grade ethanol. Frankly, considering that we're operating on a scale of 0 to about 1.2 units for a standard spectrophotometer, 0.05 is the backside of bugger-all.

Now compare that to the spectrum Roy presents for "plain ethanol". Yes, the shooting up towards opacity begins at a much higher wavelength (in fact just below 400nm), but also, in that context, the lambda-max blip (if you want to call it that) at about 325nm is also significant. (I wonder if it is closer to 330nm in the printed paper? - that would make it even more likely to be acetone, no?) Anyway, there is an absorbance there of about 0.65 units - just a tad more than 0.05, no? (Actually, 0.01 by 325nm, from the Aldritch data.)

I think we can be pretty sure that Roy did not get his "plain ethanol" from Hahnemann laboratories. I think it's pretty obvious he's picked up something in his own lab, presumably labelled "ethanol", but this something has had a very significant level of impurities in it, probably including acetone.

Now if we turn to his figure 3, we see in the top graph a huge difference between this "plain ethanol" and the "succussed ethanol". That's also showing a bit of a shoulder at 325, and another at about 280, but in the region we're looking at, the absorbance is overall very much less.

Anyone want to believe that shaking and diluting ethanol with itself removes UV absorbance from the compound? No, I didn't think so. And where do we think he got his "succussed ethanol"? Given that his astoundingly selective M&M section makes great play about the samples being "hand-succussed by trained experts", I think we know where - especially as three "potencies" are also shown for that substance. It's clear that Hahnemann Laboratories supplied the succussed ethanol samples, prepared by their "trained experts", in the same way as the Nux Vom and the Nat Mur, but without the addition of the mother tinctures.

Actually, this is good design. When we were discussing the slide show, which didn't have that figure, I said that was one of the things he should have done. However it all depends on proper provenance of the solvent for all measurements, and this is where it completely falls down. He should have insisted that all preparations came from the same stock bottle, and also used that same stock bottle for his unsuccussed spectrum. But he didn't.

So, I believe he got all the succussed and diluted samples from Hahnemann Laboratories. And he got the plain ethanol from somewhere else, and that was filthy dirty.

So far so good.

Now what about these succussed and diluted samples?

In the previous thread about this, after JJM linked to the Aldritch data, Murthy posted rather more of it.

Originally Posted by Gavinimurthy

There are nine types of ethanols mentioned on this site.

http://www.sigmaaldrich.com/catalog/search/TablePage/14577624

See the data for 493546 Ethanol.

UV absorption λ: 240 nm Amax: 0.40

λ: 250 nm Amax: 0.30

λ: 260 nm Amax: 0.30

λ: 270 nm Amax: 0.10

λ: 340 nm Amax: 0.10

*************

This is the data for specroscopic grade

UV absorption λ: 210 nm Amax: 0.40

λ: 220 nm Amax: 0.25

λ: 230 nm Amax: 0.15

λ: 240 nm Amax: 0.05

λ: 270-400 nm Amax: 0.01

The difference of absorbance between the two grades is self explanatory.

Quite fortuitously, he seems to have spotted something. JJM picked up on it.

Originally Posted by JJM

The "493546" Ethanol is potable, anhydrous ethanol. It is not as clean as the spectroscopic grade. No matter, neither corresponds to the spectrum of pure ethanol shown on p. 28 of the lecture you are promoting. Most particularly, and this is very important (so- pay attention), the slide on p. 28 shows an absorption max. at 325 nm which does not belong to pure ethanol.

The "nux vomica" spectra could simply be potable alcohol.

Indeedy, he has a point. If we were to plot the Aldritch data for the "493546" ethanol, it wouldn't be exactly the same as Roy's spectra for the succussed and diluted stuff. In fact these traces (the Hahnemann-supplied material) actualy absorb rather less than the quoted figures around 250-260nm. However, we're in the same general ball park.

Pipirr sent me a graph he'd done including the Aldritch spectroscopic and 493546-grade data for comparison with the Roy data from the slide show. I don't know why he didn't just post it here, but Pipirr - could you? Or would you mind if I did? It makes the point beautifully.

Also, the succussed stuff does vary itself. The Nat Mur traces are generally low-absorbance, and almost identical between the three potencies, while the Nux Vom absorbs rather more than the succussed ethanol, especially around 250nm, and there is more variability between the potencies.

So what I'm thinking is, first, as we concluded, the unsuccussed ethanol has a completely different provenance. However, we also have no guarantee that everything coming from Hahnemann laboratories all originated in the same stock bottle. So second, I think we're looking at slightly different sources of ethanol for each of the three hand-succussed preparations - no mother tincture, Nat Mur and Nux Vom respectively.

I've checked the web page Roy quotes in his M&M section, and there's no mention of the source or grade of the "alcohol". But I know what many homoeopaths say they use and recommend. Could we possibly be looking at slight differences in the UV spectrum of different bottles of vodka?

This is of course speculation, and I wouldn't dream of putting it in the letter we're discussing, but I'm suspicious the answer is close to this.

I still haven't digested everything in that paper, but I need to go to bed now! However, I'd like to know just how certain Wilsontown is that Roy has actually duplicated traces, rather than that we're just looking at almost-identical traces of the same basic stuff. If it's possible they're just similar then perhaps we shouldn't pursue that line too forcefully.

Nighty-night.

Rolfe.

[Pipirr]

Originally Posted by Rolfe

{snip}If we were to plot the Aldritch data for the "493546" ethanol, it wouldn't be exactly the same as Roy's spectra for the succussed and diluted stuff. In fact these traces (the Hahnemann-supplied material) actualy absorb rather less than the quoted figures around 250-260nm. However, we're in the same general ball park.

Pipirr sent me a graph he'd done including the Aldritch spectroscopic and 493546-grade data for comparison with the Roy data from the slide show. I don't know why he didn't just post it here, but Pipirr - could you? Or would you mind if I did? It makes the point beautifully.

This one?



The two short curves (the circles) were plotted from the Sigma Aldrich data (with a few interpolated points added, to be able to draw the line). I had to guess at the exact values for Roy's traces (the triangles), but they look more or less like what's in the paper.

[wilsontown]

Quote:

I still haven't digested everything in that paper, but I need to go to bed now! However, I'd like to know just how certain Wilsontown is that Roy has actually duplicated traces, rather than that we're just looking at almost-identical traces of the same basic stuff. If it's possible they're just similar then perhaps we shouldn't pursue that line too forcefully.

Yes, I suppose it is possible that we're looking at 'almost-identical traces of the same basic stuff'. I did wonder about that, but I dismissed it because there is quite a bit of variation in the graphs that are presented in the figures. If you compare, say, the graphs in figure 2, you can see that they aren't all just identical. Possibly you could explain it if each of the different 'potencies' was made with a different source of alcohol. I think the main problem is that it's just not clear what figure 2 is supposed to show. The authors say that figure 2 shows 'the envelope of differences within a series of 10 preparations of each remedy of Nat mur and Nux vom.' On the face of it, this seems to say that they show the variation in spectra across ten samples. If so, then the fact the same spectra appear in figure 1 which 'demonstrates the differences between the remedies' is dodgy. If figure 2 is actually showing something else, it might be less dodgy. It might be worth emphasising the lack of clarity in what figure 2 represents, rather than the apparent duplication of the graphs.

On the other hand, I think the problem with figure 2a and the top graph of figure 1 (that they appear identical) is definitely real. This doesn't happen with any of the other graphs in figure 2 when compared to figure 1. I guess there are two possible explanations for it: 1. The graphs got mixed up. 2. At least one of the 30c NV samples was identical to one of the 30c NM samples.

Neither explanation would really support the conclusions of the paper...

[JJM]

Thanks Rolfe, Pipirr,

Concerning the exact lambda-max. for the peak, a while back someone posted a nice close-up (with a scale) showing it at 324nm. Considering that it is riding atop a strong, rising absorbance at wavelengths shorter than 400, it could be skewed from 330.

[wilsontown]

What I find really extraordinary about this is the lack of care taken in the preparation of the paper. I'm writing a paper now (when not pratting around with graphs in Corel Draw) and I agonise over every sentence and every figure to make sure what I've written is crystal clear. Then my boss will agonise over it some more, it will be submitted and reviewed, and then (hopefully) copy-edited and published. In other words, apart from me, another four or five people will read the paper to make sure that it makes sense and that the conclusions are reasonably justified.

How has this paper managed to be published, and still be more or less unintelligible?

[Mojo]

Originally Posted by Pipirr

I missed this at the time, but Rustum Roy posted a comment on Philip Ball's article in Nature, "Here lies one who's name was writ in water".

...snip...

Did you spot the little dig there at Charles Darwin?

I suspect that he may have got his information about Darwin from Dana Ullman (who has also posted a comment there), whose claims about Darwin and other 19th century figures were discussed in this thread.

As for Darwin's opinion of homoeopathy, There's a letter in the Darwin Correspondence Project, which Darwin wrote some time after he was treated by a hydropath called James Gully, who, in addition to his "water cure" and diet and exercise regimes, also administered some homoeopathic remedies to Darwin. This gives us some idea about what Darwin thought of homoeopathy:

Quote:

You speak about Homœopathy; which is a subject which makes me more wrath, even than does Clair-voyance: clairvoyance so transcends belief, that one's ordinary faculties are put out of question, but in Homœopathy common sense & common observation come into play, & both these must go to the Dogs, if the infinetesimal doses have any effect whatever. How true is a remark I saw the other day by Quetelet, in respect to evidence of curative processes, viz that no one knows in disease what is the simple result of nothing being done, as a standard with which to compare Homœopathy & all other such things. It is a sad flaw, I cannot but think in my beloved Dr Gully, that he believes in everything— when his daughter was very ill, he had a clair-voyant girl to report on internal changes, a mesmerist to put her to sleep—an homœopathist, viz Dr. Chapman; & himself as Hydropathist! & the girl recovered.

So much, incidentally, for the repeated claims from homoeopaths that detractors of homoeopathy have never had any personal experience of homoeopathic treatments...

[Rolfe]

Originally Posted by wilsontown

Yes, I suppose it is possible that we're looking at 'almost-identical traces of the same basic stuff'. I did wonder about that, but I dismissed it because there is quite a bit of variation in the graphs that are presented in the figures. If you compare, say, the graphs in figure 2, you can see that they aren't all just identical. Possibly you could explain it if each of the different 'potencies' was made with a different source of alcohol. I think the main problem is that it's just not clear what figure 2 is supposed to show. The authors say that figure 2 shows 'the envelope of differences within a series of 10 preparations of each remedy of Nat mur and Nux vom.' On the face of it, this seems to say that they show the variation in spectra across ten samples. If so, then the fact the same spectra appear in figure 1 which 'demonstrates the differences between the remedies' is dodgy. If figure 2 is actually showing something else, it might be less dodgy. It might be worth emphasising the lack of clarity in what figure 2 represents, rather than the apparent duplication of the graphs.

On the other hand, I think the problem with figure 2a and the top graph of figure 1 (that they appear identical) is definitely real. This doesn't happen with any of the other graphs in figure 2 when compared to figure 1. I guess there are two possible explanations for it: 1. The graphs got mixed up. 2. At least one of the 30c NV samples was identical to one of the 30c NM samples.

Neither explanation would really support the conclusions of the paper...

Wilsontown, see the post I'm about to make. I've been staring at these traces since last night, composing a post about it in between doing some actual work, and I'm agreeing with you big time here....

Rolfe.

[JJM]

Originally Posted by wilsontown

{snip} How has this paper managed to be published, and still be more or less unintelligible?

Welcome to the world of quack literature.

As far as I can tell- his only conclusion is that the spectra of the various homeo-preps are "different." Your observation that differently-labelled preps in different figures are the same seems to invalidate that point.

[Rolfe]

Wilsontown, JJM (and anyone else who is following the story so far….), regarding the duplication of traces between figures 1 and 2, here’s my take on it. (I have additionally designated the three sets of axes in figure 1 as 1a, 1b and 1c, and the six sets of axes in figure 2 as 2a-1, 2a-2, 2a-3, 2b-1, 2b-2 and 2b-3 to allow clearer discussion.)

Wilsontown has convinced me that we’re genuinely looking at duplicated traces rather than different but similar traces. In fact although there are 30 lines on various graphs, and a lot of unsubstantiated rhetoric about repeatability (including a claim to have run 10 replications for each of the six different remedies and potencies), there are only 15 different UV traces in the entire paper!

• One “plain ethanol” (presented three times, 3a, 3b and 3c)
• One “succussed ethanol” 6C (presented once, 3a)
• One “succussed ethanol” 12C (presented once, 3a)
• One “succussed ethanol” 30C (presented once, 3a)
• Two Nat Mur 6C (one presented once, 2a-3; the other three times, 1c, 2a-3 and 3b)
• Two Nat Mur 12C (one presented once, 2a-2; the other three times, 1b, 2a-2 and 3b)
• Two one Nat Mur 30C (presented three times, 1a, 2a-1 and 3b) The second Nat Mur 30C trace, which should have been included in 2a-1, has been omitted in favour of duplicating one of the Nux Vom 30C traces from 2b-1
• Two Nux Vom 6C (one presented once, 2b-3; the other three times, 1c, 2b-3 and 3c)
• Two Nux Vom 12C (one presented once, 2b-2; the other three times, 1b, 2b-2 and 3c)
• Two Nux Vom 30C (one presented once, 2b-1, the other four times, 1a, 2b-1 and 3c, and once masquerading as Nat Mur 30C in 2a-1)

It goes like this.

Wilsontown, you originally said that the top graph of figure 2a (2a-1) is identical to the top graph of figure 1 (1a), which would imply that the same trace has been presented as being Nat mur 30C in figure 2, but Nux Vom 30C in figure 1. However, you also pointed out that the upper traces of each 30C graph in figure 2 (open circles in 2a-1 and closed circles in 2b-1) are identical. The shapes are certainly extremely similar, and I suspect a genuine foul-up here, especially as there’s no sign at all of that shape in figure 3b, where all three potencies of Nat Mur look very very similar, and essentially the same as the lower trace for 30C Nat Mur in figure 2a-1, indeed all the other Nat Mur traces in figure 2a. I think that anomalous upper trace is probably genuinely from a Nux Vom sample (see my “different bottles of vodka” hypothesis), and has been presented as Nat Mur in figure 2a-1 in error. However, just discussing the provenance of the traces in figure 1, this means that rather than the 30C graph in figure 1 being a straight duplicate of the entire 30C Nat Mur graph from figure 2a-1, it is better to argue that the Nat Mur 30C trace presented in figure 1 is the lower of the two Nat Mur 30C traces from figure 2a-1, while the Nux Vom 30C trace in figure 1 is the upper of the two Nux Vom 30C traces from figure 2b-1.

This then agrees with what you suggested for the 12c and 6C graphs. On each occasion the “representative” trace for the preparation presented in figure 1 is one of the two traces for that preparation presented as the “envelope of differences” in figure 2. The implication of this “envelope of differences” thing seems to be that these are the most extreme traces of the ten samples run of each preparation. To choose one of the two extreme traces as the “representative” trace for the preparation on each occasion is very odd. When we then realise that for 30C and 6C the traces have been chosen so as to exaggerate the differences between the remedies, it’s more than odd, it’s downright naughty.

This is most obvious in the 30C (top) graphs, which actually look very similar, and yet it appears that the upper trace for the Nux Vom has been chosen for figure 1 compared to the lower trace for the Nat Mur, to make it appear that there is a difference which simply isn’t evidence in the data in figure 2. (Of course the fact that the upper Nat Mur trace in figure 2 actually looks like an accidental duplication of the upper Nux Vom trace from the same figure is a bit of a complication here!)

In the 6C graphs, the two Nat Mur traces are virtually identical, but the upper of the two Nux Vom traces has been chosen to exaggerate the apparent difference between the remedies. In the 12C graphs again the Nat Mur traces are virtually identical, but in this case the lower of the two Nux Vom traces has been chosen, which in fact minimises the difference, although as the Nat Mur traces are all low-absorbance, a difference is still evident. (Cynical old me thinks we may be looking at another foul-up here, with the authors intending to pick the upper trace for 12C Nux Vom (2b-2), but in fact picking the wrong one to incorporate into figure 1! However, I note that in each case – Nat Mur as well as Nux Vom – the chosen trace is the filled-circle one, I don’t know if this is significant or not.)

The other comparison which is worth a look is figure 3 with figure 2. Needless to say, in spite of all the generalised rhetoric about repeatability, only one single “plain ethanol” trace is presented, identical in 3a, 3b and 3c. Looking at the Nux Vom potencies in 3C, these are the same traces from 2b that where chosen for figure 1 – i.e. the upper 30C trace, the lower 12C trace and the upper 6C trace, again the filled circles in each case. I think the same thing has been done for 3b, with the filled-circle traces from 2a being reproduced just as they were in figure 1. The diluted-and-succussed ethanol traces in 3a are of course new, with nothing to compare them with. In all three cases, although the different “remedies” do look different, I’d have great difficulty declaring that there was any difference at all between the three “potencies” of each remedy.

It’s figure 3, which wasn’t in the slide show at all, which really makes me think that Hahnemann laboratories simply used alcohol from three different bottles to prepare the Nux Vom, Nat Mur and no-mother-tincture preparations, and that all we’re seeing is slight differences in levels of impurities between these three bottles. I see no convincing evidence that there’s any difference between the different potencies of the same remedy.

I think I’ve more or less got my head round this now, and I think we’ve got the evidence of very sloppy work which simply doesn’t support the authors’ conclusions. The absence of any statistical treatment or presentation of the ten duplicate preparations allegedly run for each potency is particularly shocking, in my opinion. (They might even have found statistical differences between the two remedies, due to the different alcohol sources, but they haven’t even tried.)

It might be the weekend before I get a draft letter prepared, but I’m certain we ought to do it. If the scrutineers have missed this then shame on them, but one could argue that this isn’t the editor’s fault, if he was relying on the scrutineering process. If we, who have noticed the problem, then fail to bring it to the attention of the editor, then it is we who are remiss. Whether the editor then chooses to do anything about it – well, that will be the interesting bit.

Rolfe.

[wilsontown]

Rolfe:

Yup, I think your conclusions are reasonable. Had a quick look at figure 3, and I agree that the spectra seem to come from figure 2.

I think JJMs point is sound: that there's only really one conclusion in the paper, that the graphs look different. There's no attempt to consider critically why that might be the case. Just a vague attempt to imply it has something to do with water structure, which might vaguely be connected to homeopathy in some way.

[Rolfe]

Originally Posted by Pipirr

This one?

http://www.internationalskeptics.com...ba42514cd1.jpg

The two short curves (the circles) were plotted from the Sigma Aldrich data (with a few interpolated points added, to be able to draw the line). I had to guess at the exact values for Roy's traces (the triangles), but they look more or less like what's in the paper.

Thanks, Pipirr, that's great. This shows that none of the traces Roy presents is spectroscopic grade ethanol. However, while the "plain ethanol" (which I suspect didn't come from Hahnenann Laboratories and may just have been a bottle lying around Roy's lab) seems to have a huge level of impurities in it, all the diluted-and-succussed samples, which did come from Hahnemann Laboratories, might at not too much of a stretch resemble Sigma's 493546 ethanol.

All Hahnemann Laboratories say about their solvent is that it is "alcohol". In the paper, however, Rao et al. refer to it as "95% ethanol". JJM said that 493546 ethanol was "potable alcohol", and the Sigma catalogue refers to it as 200 proof, that is absolute alcohol. So maybe that isn't the one to go for. I'm going to check a bit more of the data on the Sigma catalogue just to see.

Rolfe.

[wilsontown]

This is excellent digging, folks.

Something else that I think may be worth pointing out here, that I've pointed out elsewhere (Ben Goldacre's journal club on this paper)....

A quote from this paper:

“Under the “normal” succussing procedures, it can be argued that very considerable pressures (of the order of 10 kbar) could be generated as a result of the shaking. Dachille and Roy [22] showed that mere grinding in a mortar and pestle gives rise to high pressures up to 20 kbar, and the figures for force per unit area are strongly dependent on the size of the water particles and the velocity of the shaking. By analogy with similar liquids, such as ethanol, there will be many different structures of water formed both by the pressures generated in succussing in some combination with the epitaxy on any additives.”

10 kbar is ~10,000 atmospheres. 20 kbar is ~20,000 atmospheres. 20 kbar is equivalent to the lithostatic pressure at a depth of 60 km. This claim seems extremely unlikely. I note that the authors write “it can be argued that very considerable pressures (of the order of 10 kbar) could be generated”, but don’t actually make any attempt to argue this point.

Plus the reference to Dachille and Roy [22] does not appear in the reference list. The actual reference [22] is to Bates et al., and the paper contains no mention of mortar and pestle at all, being concerned with high-pressure forms of Germanium.

Again, they're making an assertion, without evidence, in this case about high pressures caused by succussion. They're saying that this might have some role in creating different water structures, for which the paper provides no evidence. It's entirely speculation.

[Rolfe]

Originally Posted by wilsontown

Rolfe:

Yup, I think your conclusions are reasonable. Had a quick look at figure 3, and I agree that the spectra seem to come from figure 2.

I think JJMs point is sound: that there's only really one conclusion in the paper, that the graphs look different. There's no attempt to consider critically why that might be the case. Just a vague attempt to imply it has something to do with water structure, which might vaguely be connected to homeopathy in some way.

Yes, that is the main conclusion. Just that they "look different". Not, you notice, any statistical proof that the lines are actually different! (As I said, I think there may be real differences between the three basic preparations from Hahnemann Labs - Nat Mur, Nux Vom and no-mother-tincture - but I think it's probably because the alcohol came from different bottles rather than anything more esoteric, and I don't see any sign of a statistically significant difference between the different potencies of the same remedy.) And of course no attempt to speculate why they might be different, and no mention of having controlled for the most obvious potential confounder, variations in the purity of the ethanol.

Also, what the blue blazes do UV spectra of 95% ethanol have to do with the structure of water? Is it perhaps significant that they got their apparent differences from ethanol, which is more likley to attract impurities than water?

I do so agree with your point about the sheer sloppiness of the entire thing. Not just the misappropriated trace, but quite a lot about the actual paper, like no clear description of M&M, no statistical treatment of results, no clear conclusions, and maybe worst of all, a major heading "Preliminary studies of homoeopathic medicines using Raman and infrared spectroscopy" - which then goes on to present ultraviolet and Raman spectroscopy, while declaring that IR spectroscopy didn't work so they aren't going to present any results.

Any competent scrutineer should have thrown it out. But then this is peer reviewed remember, and in the case of this journal, the "peers" are homoeopaths.

I'm just so astonished that apparently senior research scientists with years of experience of preparing papers properly should perpetrate such an abomination. Can it really just be carelessness, or is it deliberate obfuscation?

Rolfe.

[Pipirr]

Originally Posted by Rolfe

I've checked the web page Roy quotes in his M&M section, and there's no mention of the source or grade of the "alcohol". But I know what many homoeopaths say they use and recommend. Could we possibly be looking at slight differences in the UV spectrum of different bottles of vodka?

If it helps at all, here's a typical UV-Vis spec of four different vodkas.



I can't imagine that either Rao or Hahnemann Labs would have used vodka.




Except when they were writing the paper....

[Rolfe]

Originally Posted by Pipirr

If it helps at all, here's a typical UV-Vis spec of four different vodkas.

http://www.internationalskeptics.com...c359275365.jpg

I can't imagine that either Rao or Hahnemann Labs would have used vodka.




Except when they were writing the paper....

OK, fair enough! It was just a thought, based on the fact that I know one homoeopath who does use vodka to do his dilutions. Of course vodkas tend to have additives, and quite a lot of them. Though I wonder about some sort of raw hooch?

Anyway, I checked all the ethanols in the Sigma catalogue, and only the spectroscopic grade (493511 for the 95% and 459828 for the 100%, also noted as HPLC grade), and the “USP/NF” (493538 for the 95% and 493546 for the 100%) have UV absorption data recorded.


Both strengths of spectroscopic grade have the same UV data, as we discussed earlier –

λ: 210 nm Amax: 0.40

λ: 220 nm Amax: 0.25

λ: 230 nm Amax: 0.15

λ: 240 nm Amax: 0.05

λ: 270-400 nm Amax: 0.01

Both strengths of USP/NF also have the same UV data, as discussed earlier –

λ: 240 nm Amax: 0.40

λ: 250 nm Amax: 0.30

λ: 260 nm Amax: 0.30

λ: 270 nm Amax: 0.10

λ: 340 nm Amax: 0.10

This latter looks like a relatively rough guide compared to the spectroscopic grade data, and makes me even more inclined to believe that it is something of this grade we’re seeing in the Hahnemann Laboratories samples.

As far as I can make out, USP/NF stands for “United States Pharmacopoeia - National Formulary”. Sounds exactly like what they might procure for their processes, and it seems to me quite possible that this might have a bit of between-batch variation in it. If they simply took the remedies off the shelf, and they were prepared at different times, then batch variation in the pharmaceutical-grade alcohol could easily explain what we’re seeing in these traces I think.

Er - just as JJM said about two days ago....

Rolfe.

[Rolfe]

Here is the first draft of the proposed letter. Constructive criticism please.

Quote:

Dear Sir,

We wish to draw to your attention serious anomalies and incongruities in the UV absorption data presented in the paper by Rao et al., published in your August 2007 issue [1].

In a study of this nature, which in effect is examining multiple samples of ethanol, the over-riding concern must be absolute uniformity in the source of the solvent. For the data to be valid, it is essential that every drop of ethanol used must be sourced from the same stock bottle. However, the authors fail to make any mention of this point, and it is clear from the results presented that the source of ethanol was most certainly not uniform.

The most striking anomaly is the UV spectrum presented for “plain ethanol”, a single trace repeated three times in figure 3. The provenance of this sample is not recorded in the paper. This trace reveals extremely high absorbance (greater than 0.8 absorbance units) at 250nm, falling off steeply towards 400nm but still above 0.4 units by 350nm, and demonstrating an absorbance peak of 0.65 units with a l max of about 330nm. It is simply impossible to represent this trace as being ethanol of any recognised degree of purity. Spectroscopic grade ethanol has an absorbance of less than 0.05 units between 250 and 400nm [2], and even USP/NF pharmaceutical grade ethanol has an absorbance of less than 0.3 units at 250nm, falling off to less than 0.1 units by 270nm [3]. If the substance measured by the authors as “plain ethanol” was indeed ethanol at all, it is clear that it contained extremely high levels of impurities, possibly including acetone.

In contrast, the spectra of the samples which were diluted and succussed (Nat mur, Nux vomica and the “succussed ethanol” with no mother tincture), and which were presumably all supplied by Hahnemann Laboratories as detailed on page 178, demonstrate substantially lower levels of impurities. While still not being spectroscopic grade ethanol, these samples could well represent ordinary pharmaceutical grade ethanol. It is these samples the authors claim are “different”. However the evidence presented for this is weak to nonexistent.

Figure 1 presents one trace each for Nat mur and Nux vomica, each at 6C, 12C and 30C potencies. The traces are said to be “representative”, however with no information on repeatability or how the “representative” traces were selected, it is impossible to say whether there is in fact any real difference between any of the six spectra.

Figure 2 purports to address this point, but then fails to present the necessary data. The legend declares that 10 samples of each of the six remedy preparations were analysed. The normal way to present such data would be as mean absorbance ± standard deviation for each wavelength point, or at least for a representative selection of wavelength points. Statistical analysis could then be used to demonstrate whether or not there was a real difference between any of the remedies or potencies. However, the authors have chosen instead to present something they term “envelope of differences”. This is not a recognised method of handling such data, and it is not explained. We surmise that “extreme” high and low traces for each preparation were chosen to provide an impression of the range of results obtained. However, the fact that on two of the graphs the traces appear to cross may argue against this interpretation.

A further difficulty with figure 2 is that the upper (open circles) trace in the top graph of fig 2a (30C Nat mur) appears to be a duplicate of the upper (closed circles) trace in the top graph of fig 2b (30C Nux vom). Comparison with other traces of the two remedies indicates that this trace is really one of Nux vom, which has been duplicated into the Nat mur graph in error.

Paucity of data, ambiguity of presentation and lack of statistical analysis prevent any conclusions being drawn from the information in figure 2.

Comparison of figure 2 with figure 1 reveals that all six traces presented in figure 1 are taken from figure 2, in each case the closed-circles traces. If indeed the traces in figure 2 represent the extreme range of results obtained, this is startling, as the traces in figure 1 are stated to be “representative”. In addition, while it does appear that the Nux vom samples tended to demonstrate higher absorbances than the Nat mur samples (excluding the obvious mistake noted above), in two out of the three potencies the higher Nux vom trace from fig 2 has been chosen for inclusion in fig 1, thus exaggerating the apparent difference.

Figure 3 (b and c) again repeats the same six traces as figure 1, this time grouped by remedy. Presented in this way, it is clear that there is no difference between the three potencies of Nat mur, and that while variation between the Nux vom potencies is a little more pronounced, again all three traces appear to come from the same population. The same is true of the three potencies of “succussed ethanol” presented in fig 3a.

On simple visual inspection it does appear that there may be genuine differences between the three remedies (although no statistics are presented to allow this to be tested), with the Nat mur showing the lowest absorbtion and the Nux vom the highest, with the succussed ethanol lying somewhere between. Nevertheless, these differences are entirely consistent with small differences in purity of the ethanol stock used for preparation of the three remedies – small, that is, relative to the very high level of impurity evident in the “plain ethanol” sample presented alongside. This degree of variation in UV absorbance is entirely to be expected between different batches of pharmaceutical grade ethanol, which is not prepared with spectroscopic analysis in mind. The authors make no mention of having stipulated to Hahnemann Laboratories that all material sent to them should be prepared from the same stock bottle, and the data presented indicate that the different remedies, possibly prepared at different times, simply came from different bottles of ethanol.

We hope you agree that these are very serious points, and it is regrettable they were not identified by your own scrutineering process. It is clear that the data presented are wholly inadequate to support the authors’ assertion that UV spectroscopy can differentiate between the two remedies, and between different potencies of the remedies. If the authors wish to test their assertion so that it can be substantiated it will be necessary to repeat the work from the beginning, ensuring that all samples used in the study are sourced from the same bottle of stock solvent, that all duplicate preparations for precision assessment are separately prepared de novo from the mother tinctures, and that sufficient data are generated to allow robust and valid statistical analysis of the results.

Yours faithfully,

Rolfe
JJM
Wilsontown
Pipirr

References:

1. Rao, M. L., Roy, R., Bell, I. R & Hoover, R. (2007) The defining role of structure (including epitaxy) in the plausibility of homeopathy. Homeopathy 96, 175-182.
2. Sigma Aldrich catalogue, ACS spectrophotometric grade ethanol 95.0%, at www.sigmaaldrich.com/catalog/search/ProductDetail/SIAL/493511
3. Sigma Aldrich catalogue, USP/NF grade ethanol 190 proof, at www.sigmaaldrich.com/catalog/search/ProductDetail/ALDRICH/493538

Rolfe.

[Cuddles]

I'd say it looks good. The only incredibly minor point is that the start of the fourth paragraph should read "In contrast, the spectra...".

[wilsontown]

Quote:

Here is the first draft of the proposed letter. Constructive criticism please.

I must admit I can't find a thing wrong with it. Good work!

[Rolfe]

Originally Posted by Cuddles

I'd say it looks good. The only incredibly minor point is that the start of the fourth paragraph should read "In contrast, the spectra...".

Rolfe.

[JJM]

Rolfe,

The letter is great, don't change anything. And thanks for writing it.

[Rolfe]

Well, thanks chaps. Calling Pipirr, now....

One extra little thought. I took literally the comment on Bad Science that a Google search on "envelope of differences" returned only references to people talking about Rustum Roy, and nobody knew what it meant. However, I thought I'd better check.

"Envelope of differences" gets very few returns, but three of them are not the Roy paper, and in these the term seems to mean something.

"Envelope of difference" gets a few more like that. Audio stuff, meteorology, crystallography and one relating to FTIR spectra.

The term does seem to have an actual meaning, and to be a legitimate way of handling some data. The trouble is, none of the links attempts to define it, it's just used in passing.

Whatever it is, it's still a totally illegitimate way of handling the data Roy says he has (the 10 repetitions for each remedy preparation). Nevertheless, I don't want to appear completely ignorant of something that has a legitimate definition.

Anybody care to see if they can do better than me searching the Google returns for a clue about what the term actually means?

Rolfe.

[Pipirr]

Originally Posted by Rolfe

Well, thanks chaps. Calling Pipirr, now....

Looks great in passing - I'll read it thoroughly later today.

Nice work!

[Professor Yaffle]

Can't contribute much to this discussion (though I have enjoyed reading it) - but I found a paper which defined envelope of differences for the purposes of their usage and I wondered if you could work out a definition from that?

http://www.riskmania.com/pdsdata/AMu...tRates-ch4.pdf

Quote:

Envelope of differences: the difference
between the position’s median Mark-to-
Future value and its potential future value at
the 97.5th or 2.5th percentile at each time step.

[wollery]

I think this thread should be renamed;

Homeopathy: Rustum Roy drops the ball.

[Rolfe]

Originally Posted by Professor Yaffle

Can't contribute much to this discussion (though I have enjoyed reading it) - but I found a paper which defined envelope of differences for the purposes of their usage and I wondered if you could work out a definition from that?

http://www.riskmania.com/pdsdata/AMulti-factorStatisticalModelforInterestRates-ch4.pdf

Bloody hell, Prof! That one redefines "incomprehensible"!

I still think all Roy means by the term is that he's picked the most extreme traces. Otherwise, why would the same traces appear in other graphs where there has been no attempt at duplication?

Maybe all I need to do is to make it less obvious that I don't know what the term actually means, and just comment that it's not defined, and not an appropriate way to present the data in question.

Such as -

Quote:

However, the authors have chosen instead to present only two traces for each preparation, as "envelopes of differences", which is not an appropriate method of handling such data as most of the information is lost and statistical comparisons are prevented. The way in which these traces are derived is not explained.

And so on. This is a bit clumsy and maybe I'll think of a more elegant form of words in the morning. Mean time, all contributions welcome.

Envelopes of differences? Sheeeeshhhhh....

Rolfe.

[Pipirr]

I went through it with the paper in hand and have almost nothing to add. You were very thorough!

The only thing I would consider changing is:

Change this:
In a study of this nature, which in effect is examining multiple samples of ethanol, the over-riding concern must be absolute uniformity in the source of the solvent.


To this:
In a study of this nature, in which multiple ethanol-based samples are examined, the over-riding concern must be the absolute uniformity in the source of the solvent.

Or something like it. The original wording might be taken as a dig at the principles of homeopathy (it's all just ethanol!), by the sensitively minded...

[steenkh]

I do not have the paper, but did not some of you mention that Roy discusses water properties at length, but he only analyses ethanol. Perhaps this criticism could be slipped in somewhere without over-playing our hand (and make the editor toss the letter in the bin)

[Rolfe]

Originally Posted by Pipirr

I went through it with the paper in hand and have almost nothing to add. You were very thorough!

The only thing I would consider changing is:

Change this:
In a study of this nature, which in effect is examining multiple samples of ethanol, the over-riding concern must be absolute uniformity in the source of the solvent.


To this:
In a study of this nature, in which multiple ethanol-based samples are examined, the over-riding concern must be the absolute uniformity in the source of the solvent.

Or something like it. The original wording might be taken as a dig at the principles of homeopathy (it's all just ethanol!), by the sensitively minded...

I did think about that, but actually Roy never disputes that he's just analysing multiple samples of ethanol. That's his entire point. He thinks he's found changes in the structure of ethanol. Even though there could theoretically be traces of mother tincture in the 6C preparations, he's not claiming that physical presence of solute is having any influence. I can change it if you like, but I think it's probably not a contentious point in this context.

As far as other problems with the paper are concerned, how long have we got? There are a myriad other howlers and boo-boos and nonsenses in this. Wilsontown mentioned a reference that wasn't in the reference list (and what was there in its place didn't support the assertion being made). The entire structure of the thing is appalling - have the authors never heard of Introduction/M&M/Results/Discussion? At one point they claim they're going to present FTIR data, then don't - UV data are presented instead. The description of the materials and methods is a joke, with ludicrous detail about things that don't matter, and zippo about things that very much do matter. Much boasting about multiple runs and reproducibility, but no data on this - just the same few traces presented multiple times on different axes. And so on. (And I've a feeling that if we had anyone here who was familiar with Raman spectroscopy, we'd find that these results are just as laughably wrong as the UV ones.)

If we start to get into all this, it begins to look like a blunderbuss attack, and the letter loses focus. In fact, the other points are so obvious to anyone who knows anything about writing a scientific paper that I woudn't be surprised if Peter Fisher gets other letters just about these. And if anyone here wants to do that, don't let me stop them! But I think what we've got is a detailed dissection of the UV data, and we should focus entirely on that, even if it does mean we feel we've missed a few easy tricks.

Rolfe.

[wilsontown]

Quote:

But I think what we've got is a detailed dissection of the UV data, and we should focus entirely on that, even if it does mean we feel we've missed a few easy tricks.

Yes, I think I agree with this. Pointing out all the problems with the paper would take another paper, and who has got time to do that? It seems reasonable to focus on the UV-vis stuff, because we're just about certain that it doesn't support the conclusions of the paper. This is the critical thing-that the conclusions (such as they are) are not supported by the data presented-and everything else is relatively minor.

[Pipirr]

Originally Posted by Rolfe

I did think about that, but actually Roy never disputes that he's just analysing multiple samples of ethanol. That's his entire point. He thinks he's found changes in the structure of ethanol. Even though there could theoretically be traces of mother tincture in the 6C preparations, he's not claiming that physical presence of solute is having any influence. I can change it if you like, but I think it's probably not a contentious point in this context.

No, fair enough. You're right, the paper is all about the 'bulk properties' of ethanol itself, so it shouldn't be a problem.

[Rolfe]

OK, new paragraph 6, trying to address the "envelopes of differences" thing. ("Address the envelopes" - oh I'm such a wit....) I've taken out the sentence about some of the lines crossing, as it didn't really add anything.

Quote:

Figure 2 purports to address this point, but then fails to present the necessary data. The legend declares that 10 samples of each of the six remedy preparations were analysed. The normal way to present such data would be as mean absorbance ± standard deviation for each wavelength point, or at least for a representative selection of wavelength points. Statistical analysis could then be used to demonstrate whether or not there was a real difference between any of the remedies or potencies. However, the authors have instead chosen to present only two traces for each preparation, as “envelopes of differences”. The derivation of these traces is not explained, although we surmise that “extreme” high and low traces for each preparation were chosen to provide an impression of the range of results obtained. This is not an appropriate method of handling data of this nature, as most of the information is lost and statistical analysis is rendered impossible.

Rolfe.

[wilsontown]

'Address the envelopes', cracking stuff.

I think the new version of paragraph 6 is fine. After all, it's really up to the authors to explain what on earth they mean, rather than for us to guess.

Just to draw your attention to this comment on the Bad Science journal club of the article, where someone who knows something about Raman spectroscopy weighs in.

http://www.badscience.net//?p=496 [look for comment No. 11]

[Rolfe]

Originally Posted by wilsontown

'Address the envelopes', cracking stuff.

I think the new version of paragraph 6 is fine. After all, it's really up to the authors to explain what on earth they mean, rather than for us to guess.

Just to draw your attention to this comment on the Bad Science journal club of the article, where someone who knows something about Raman spectroscopy weighs in.

http://www.badscience.net//?p=496 [look for comment No. 11]

I thought as much. I did look at the question of different scales (and even different units) on the different Raman spectra graphs as shown in the PowerPoint, but I couldn't find anyone to explain it in detail - although someone did give a helpful guide to how it works.

From Another John, yesterday evening on Bad Science (Wilsontown's link)

Quote:

Figure 4:
The range 750-975 cm-1 is only covered in Figure 4A as is the range 1560-1680 cm-1 making a mockery of their statement; “From the spectra shown in Figure 4, a clear distinction in the Raman active modes is noted between the TWO DIFFERENT REMEDIES as well as among the different potencies of the same remedy.”

Fig4A point (a) 6C and Fig4B point (a) 6C look to me to be the same intensity (1200 to 1300 arb units).

Fig4B point (b) 6C is probably no more of a peak then Fig4A point (d) 6C.
Looking at Figure 4 as a whole (bar Fig4A point (a) and Fig4B point (a)); every time a peak is highlighted the intensity around the peak is also greater. This is caused by just having a larger/bigger/fatter sample.**

Figure 5:
Assuming the peak positions a-f in Figure 5 to be real events, then I would suggest that the following “peaks” are not limited to the 6C sample as is stated by the authors:
a) there is a shoulder at around 275 cm-1 in all the samples .
b) the shoulder at around 380 cm-1 is definitely present in the “plain” sample (I admit there might be a slight shift in wavelength) and possibly in the 12 and 30C samples as well.
c) the shoulder at 415 cm-1 is present in all the “plain” sample. It is a little less intense, but then so is the main peak (probably due to different quantities of sample being scanned).
d) In the 6C sample, this peak at 625-630 cm-1 is around the same intensity as the peak at 650 cm-1. This is also true for the “plain sample”
e) is present in all the samples.
f) there is a shoulder in the plain sample at the same point as the peak.

I couldn't cope with adding a critique of the Raman spectra to what we've got, partly because it would be too long, and partly because I'd just be parroting - I know nothing about Raman spectra.

However, I wish Another John would come by here and elaborate. Maybe a separate letter with different signatories could be prepared, dealing with the rest of this duff data.

I think we need to give special, extra thanks to JJM though. It was he who spotted the filthy dirty nature of the "plain ethanol", and pointed us all at the Sigma catalogue specs for what ethanol should look like at 250-400nm. Nobody else seems to have picked up on this at all, and it's the absolute killer argument.

Rolfe.

[Rolfe]

I mentioned how few (non-Roy) returns I'd got when Googling "envelope of differences". Just enough to suggest Roy hadn't made it up, but very few indeed, and mostly very vague, almost as if the term just meant whatever the writer wanted it to mean at the time.

I just Googled "standard deviation". 24,300,000 returns, with the entire first page being links to pages defining the term.

Roy doesn't seem to have heard of it though....

Rolfe.