That is the picture I intended to invoke, but maybe it was expecting too much of you that you would recognize it as silly and giggle about it?
Linda
Ganzfeld Experiments
- Thread starter Uncayimmy
- Start date
That is the picture I intended to invoke, but maybe it was expecting too much of you that you would recognize it as silly and giggle about it?
Linda
Have you ever read any of the research?Nothing like an informed opinion and an open mind?
j x
Some, yes, and it tells me diddly-squat.
I don't rule out anything, including the chimera of psi, but up to now the "evidence," such as it is, has been thoroughly underwhelming.
Careful your mind's not so open that your brains fall out.
M.
Limbo
Jedi Consular
- Joined
- Feb 29, 2008
- Messages
- 3,077
That's too bad. It would be much pleasanter for you if you did.
Do you really think that'd work in a court of law? I'm not sure whether or not I should be worried or hopeful. I'll have to ask LossLeader.
Linda
I let it slide. I'm not here to win hearts and minds, just to work out what's going on.
Well, that's not true. There is. I often see people say "there's no evidence for psi", but in fact there is - it's just not very good.
Does that mean that you think it's unreasonable of me to point out when someone has used post-hoc group formation as a means of finding a significant difference when the main outcome is unremarkable?
I still need an answer to your comment about M&W's use of the Stouffer z. As far as I can see, it was formed as you described for its usual use, and it doesn't seem to give a significant result when formed that way. What am I missing?
Linda
Is there some part of that argument available online? I don't want to spoil your appetite.
Linda
Hmm, this is frustrating. I could’ve sworn I had more on this than I have. The first I heard of this was in an interview with Dean Radin on the podcast Skeptiko about a year ago.
I assumed, that the "method of statistical accumulation" was using the Stouffer z on separate experiments, which seemed to be borne out by Rodney (who emailed Daryl Bem about it). But I can’t find any published paper that makes this claim. I must’ve misremembered.
Thanks for checking that. It looks like Radin is talking about simply combining all the results (as though it were one giant experiment) in the manner that Rodney has talked about before (and maybe that's where Rodney got the idea), although I could have sworn that it was carefully explained to Rodney why this can lead to invalid results using (I think) batting averages as an example. Anyway, I need to confirm that this is what Bem was talking about. I'm harping on about this because I want to recommend that you use a method of combining studies that is familiar to parapsychologists, but not to the point of recommending something that obviously has the characteristics of 'likely to lead to invalid conclusions'. And I'd really be surprised and disturbed to discover that Bem and Radin would think that it was okay to do this with the ganzfeld data.
Linda
What Bem communicated to me in an e-mail was that he found Milton's & Wiseman's statistical methodology flawed, but went along with it in his, Palmer's, and Broughton's follow-up paper "Updating the Ganzfeld Database: A Victim of its Own Success?" because -- even using M&W's methodology -- the results were statistically significant and he, Palmer, and Broughton saw no need to fight an unnecessary statistical battle.
I was the one who suggested that the hits and misses from 26 of the 30 studies analyzed by M&W could simply be summed to calculate a binomial probability for those 26 studies because each of them used a hit or miss protocol. When that is done, p = 1.1%. I further noted that the other four studies comprised only 128 trials (I mistakenly said 148 previously) relative to 1,198 trials overall in the 30 studies, and the hit rate can be imputed for those 128 trials. Therefore, while there might be a better way to account for those four studies, it seemed reasonable as at least a good approximation to add the imputed hits in those four studies to the actual hits in the other 26 studies to calculate a binomial probability for the 30 studies. When that is done, p = 2.4%. If you have a better way of calculating the p value for the 30 studies analyzed by M&W, please advise.
Did he say what the flaw was?
(btw, the Kanthamani & Broughton experiments did report hit rates.)
Last edited:
He said that statistician Jessica Utts (from UC Davis) told him that Milton and Wiseman should have weighted each experiment by the number of sessions it contained.
Really? In the Bem, Broughton, and Palmer paper, there is a footnote attached to these experiments, as well as to the Stanford & Frank and Parker & Westerlund experiments, which states: "Hit rate not reported. Estimated from z score."
In the original papers Kanthamani wrote, there's no hit rate, but in the paper collecting all the Inst. for Parap. ganzfeld experiments (the Kathamani & Broughton one) it lists them in a table reporting hit rates. I guess it depends what paper they looked at.
What were the hit rates? In the Bem, Broughton, and Palmer paper, they are reported as follows:
1) Kanthamani & Khilji (1992) (Series 6b): 40 trials, 30% estimated hit rate.
2) Kanthamani & Broughton (1992) (Series 6a): 20 trials, 25% estimated hit rate.
3) Kanthamani et al. (1988) (Series 5b): 10 trials, 10% estimated hit rate.
But evidently not about me.So, can you enlighten me as to why summing the results for the 26 studies is invalid?
Because it can result in conclusions that are markedly different from the conclusions that can be drawn from individual studies. It introduces a bias that corresponds to differences in the size of the various groups studied.
Usually we use odds-ratios or risk-ratios for dichotomous data. Depending upon whether we think we are testing a fixed-effect or a random-effect, we may choose to combine them using Mantel-Hanzel or inverse variance methods - a sort of weighted average using variance to provide the weight.
Linda
That seems to contradict the batting average example you pointed me toward. As the Wikipedia article notes, a batting average for a small number of at-bats can be highly misleading, but is highly meaningful over several thousand at bats. Similarly, it seems to me, the results of a single Ganzfeld experiment with a small number of trials can be highly misleading, but, when grouped with a number of similar experiments, the results can be highly meaningful. One of the experiments analyzed by Milton and Wiseman had four trials with two hits -- what possible conclusion can be drawn from that one experiment, if it is not grouped with other experiments?
What do you believe is the bet way to combine the 30 studies analyzed by M&W, and what overall probability would that produce?
The Kanthamani/Broughton paper there's no distinction between to two halves of each experiemnt. Series 5 is reported as having 14 trials and 4 hits, series 6 is listed as having 66 trials and 18 hits (which is odd, since everywhere else it's said to have 60 trials).