I thought I'd posted that yesterday, but it seems not. I probably hit the wrong button. I'll try to compose it again.
I'm not sure about Linda's assessment regarding homeopathic studies being categorized as "discovery-oriented exploratory research with massive testing" and setting R at 0.001. I'm not well acquainted with the literature on homeopathic studies, but the one homeopathic paper I've read was testing a specific substance for specific outcomes and thus it seems to me it would not fit into that category.
Finding a reasonable magnitude for R is the key issue. The typical controlled trial consists of testing one or a few homeopathic remedies in a clinical setting like "influenza" or "post-operative ileus". There is no established method by which the remedy or the condition is chosen for the testing. Rather, they are chosen based on the idea of homeopathy which has no supporting evidence as a valid method of choice.
If you are not familiar with homeopathy - it is based on an idea of "like cures like". A substance that causes certain symptoms when taken in pharmacologic doses, will cure those symptoms when taken in miniscule doses. A substance is "proven" by ingesting the substance and then meticulously recording all physical sensations the individual has for some period of time afterward (e.g. nausea and vomiting). Thereafter, if someone presents to a homeopathic practitioner with vomiting, they are given a very dilute preparation of the substance, to relieve their vomiting. The dilutions are made serially - the substance is put into solution in water and one part (e.g. one drop) of that is diluted with 100 parts of water ("C" potencies). Then one drop of that solution is diluted in 100 parts of water. This is repeated (commonly) 30 times. After 15 dilutions, the probability that the solution contains a single molecule of the original substance is effectively zero.
No part of this idea has ever been established. In fact, it contradicts all of our detailed knowledge about physiology, pharmacology, pathology and chemistry. And this is the critical part for determining R. Despite all the time and effort that has gone into developing homeopathy, the method of selection is no different from playing a game. The "rules" of the homeopathy game are arbitrary - there is nothing that prevents anyone from changing the rules. I could just as easily say that "like cures like" and choose a remedy that starts with the same letter of the alphabet as the symptom that I wish to treat.
Because the rules are arbitrary, there is no constraint on what substance could be chosen to treat what condition. It is no different from throwing a series of dice in order to make a choice. So, yes, you are testing one substance in one condition, but that combination has been (essentially randomly) chosen from among millions of possible combinations to test. And the totality of homeopathic clinical trials amounts to having chosen a few dozen combinations from among the millions, to test. That is why it is similar to exploratory research, which may test a few dozen combinations at one go.
[As an aside, in interpreting results of a situation with "massive testing", simultaneous confidence intervals should be used which keep the probability of a false positive from rising above alpha. I'm not sure how commonly simulataneous C.I.'s are used, but the author of the paper makes no adjustment for that situation in his analysis.]
That is one of the problems with homeopathic research. In many studies, multiple comparisons are made with no adjustments in alpha. If one of those comparisons is "statistically significant", it is concluded that the remedy is effective.
The use (or non-use) of commonly simultaneous C.I.'s would probably be covered under his "bias". But I agree, that it could benefit from further quantification.
And if you are not familiar with homeopathy research - the majority of trials are of poor-quality. A few dozen are of moderate to good quality. The better the quality the more likely they are to be negative. Metanalyses show that the demonstration of benefit can be accounted for by placebo-effect and bias, or by the use of active ingredients (i.e. non-dilute preparations acting by normal pharmacologic methods).
Since R respresents the ratio of "true relationships" to "no relationships", it can literally vary from 0 (no true relationships) to infinity (everything is related). Depending on your prior assessment of the ratio of true relationships to no relationships, the outcome can be just about anything. Unless you and Linda can agree upon a reasonable value for R, I don't think this analysis is useful to your discussion. On the other hand, discussing a reasonable value to set for R might be.
That's the issue. The analysis is useful to demonstrate that a reasonable value for R must be chosen. It cannot be assumed to be 1:1, which was what Rodney was assuming.
I am definitely open to other suggestions.
Linda