From Suitbert Ertel:
1. Linda, what you write about randomness of ball positions is o. k., but ball positions are not essential. What counts in this experiment is only the randomness of numbers being picked. If this experiment were conducted with only 5 balls in a bag, one ball for each number 1, 2, 3, 4, and 5, then, after mixing the balls sufficiently, these numbers would have an equal chance of being picked. Now add another set of five balls 1 to 5, the probability of the five numbers being picked does not change. Add another set, nothing will change, continue adding sets of five balls, no change. No doubt, the probability of individual balls being picked will change, balls positioned on surface in the bag filled with hundreds of balls will have a greater chance to be picked than balls positioned at the bottom. But who cares?
I'm glad you agree that some positions are more likely to be chosen than
others. Like I said, I wasn't trying to say anything you wouldn't agree to.
As I mentioned in my last post, either each ball position must be chosen
randomly (which we agreed does not happen with human drawers), or each ball much have an equal chance of finding itself in one of those few ball
positions which get chosen. Who cares? I simply wanted to make it clear
that there is a distinction to be made, as it influences how we get
deviations from randomness.
As I told you before, just drawn numbers do not have a greater chance of being picked again than other numbers even though the chance of a just drawn individual ball being picked again, even in a bag with only 50 balls (the standard condition) does change to some extent. In your long criticism you do not address the distinction between balls and numbers which is crucial. Why not?
I think you are talking about the difference between ensuring randomness by
looking at whether the conditions are set up to be random (balls), and
whether or not the end result is random (numbers)? I agree that this is
what is at issue, if that wasn't clear from what I've said. In fact, that
is essentially my main criticism of your set-up. The purpose of good
experimental design is to create a situation where sources of bias have been
eliminated a priori. Instead you are arguing for allowing bias and then
attempting to discover these biases after the fact and appropriately
model/control for them. The problem is that this fails on both theoretical
and practical grounds. It means that you have to be extra clever - you have
to be able to think about all the ways in which the results may be biased,
and you have to figure out how to appropriately model those biases. Our
statistical techniques are designed to rule-in effects, not designed to
rule-out effects. Which means that our standard for ruling-out the effect
of a bias is far too low. For example, when Dean Radin went looking for a
particular bias in his pre-sentiment experiments as a result of some
criticisms and stated that he had ruled-out the possibility that it
contributed to the results, a power analysis showed that he allowed himself
a 40% chance of missing a real effect - that is, there was only a 60% chance
that he had ruled-out the possibility that bias was responsible for his
results. And this doesn't even take into account that there may be multiple
biases involved, any one of which is difficult to detect, which may add up
to a substantial effect (when you consider that your effect size is very
small).
The practical problem with your idea is that it has been demonstrated to be
wrong. In medicine, we frequently find that effects from observational
studies are over-turned by subsequent randomized, controlled studies. And
we try to do the same thing that you are doing with those observational
studies - we think of all the ways in which the results may be biased (age,
sex, socio-economic statues, etc.) and then model/control those sources of
bias to see if there is still an effect. And yet, we have discovered that
this simply isn't good enough. Observational studies led us to believe that
hormone replacement therapy or vitamin E reduced the risk of disease - even
taking into account potential influences. Yet, when subject to an
experimental design that ensured that bias was eliminated a priori - the
randomized controlled trial - it turned out that not only did these
therapies not have a benefit, they may even be harmful.
2. Linda, you said "Don't you see the difference?" [when you look at the frequencies of drawn numbers 1 to 5 in the 12345 condition demanding equal frequencies of drawn numbers]. The differences you and I also "see" are statistically not significant (a chi squared test has a p-value of .58). Our verdict must be based on calculation, not on visual impression.
I'm sorry. I wasn't clear. I wasn't talking about whether the frequencies
of the drawn numbers were different (I agreed that they were not). I was
asking you why you didn't seem to realize that there were substantial
differences in the study design.
3. You write: " Since the numbers were called with equivalent frequencies, then you'd expect the ball drawers to pick the numbers with roughly equivalent frequencies. In this case, there is an intention, a priori, to pick a particular number."
Yes, the participants are instructed to draw a particular number, in the standard condition they always have an a priori intention to draw a particular number. But intention does not explain the result which shows that – for a minority of participants – the desired numbers have a greater chance of being picked. What should be explained is how the success of these participants comes about. Since an explanation by sensory leakage or some other ordinary mechanism does not hold, psi is the only applicable concept. Psi, however, cannot presently be regarded yet as an explanation. parapsychologists want and expect psi to get explained by future science.
But as you admit in the video you provided of the Indian boy (if it is you
who provides the critical statement at the end of the video), sensory
leakage is not ruled-out since the boy simply looked into the bag on
multiple occasions. If the other participants in this experiment did the
same thing, there is a mundane explanation for the results, are they are
exactly what you would expect to find if the participants can sometimes see
which ball they want to pick.
4. I had considered the fact that in one instance of the 5x5 matrix test one participant obtained deviations from chance only for left bag numbers, not for right bag numbers and this was unexplainable by your “not-mixing-sufficiently” argument.
I don't understand how this is unexplainable. Even from the 12 sessions in
the video you provided one can see that the amount of shaking varies quite a bit from one session to the next, such that sometimes there did not seem to be much opportunity for mixing. Without documenting the amount of shaking on each trial for each participant in the 5x5 matrix, you simply don't know how much it varied, to what extent it was adequate, and to what extent it was associated with non-random results. You claim that your analyses after the fact rule it out, but you haven't documented that your analysis was sufficient to model or discover the effect, if present.
I had written: “Participants will hardly shake balls in the left bag more frequently or more intensely than the balls in the right bag, or less intensely or less frequently”. And you replied: “Why not? I will be more dextrous in my shaking of the right bag than I am of the left bag, simply because I am right-handed.”
So would you expect for standard ball tests (tests with intentions), in general, more hits for tests with balls drawn with the right hand than for tests with balls drawn with the left hand, because the great majority of participants are right-handed? Yes, you have to claim that.
Not at all. I am merely saying that this is a potential additional source
of variation. And we are not talking about "hits" in the 5x5 matrices. You
did not establish any particular intention a priori, so there is nothing to
"hit". All you are saying is that the matrices show greater variation than
you expected. I am asking, why did you expect so little variation
considering that you introduced extra variation with your research design?
I conducted a large number of standard tests with two bags, thus the condition was similar to the matrix test condition. But participants with two-bags had to wish to draw particular numbers right and left. Results: The total number of trials was 50,336. The number of hits expected by chance was 100,672. The observed number of hits, average of left and right bag hits, was 11,186. The crucial observation: The difference between right hand hits (11,227) and left hand hits (11,145) was statistically not significant (p = .58). Your argument, Linda, is thus invalid.
You are still misunderstanding what I said. The crucial difference is that
your standard experiments - those where you establish *a priori* the result
you want, are *different* from those where you are not looking for anything
in particular beforehand. If the reason for the increased hit rate is
chance and mundane sensory input, such as was demonstrated by the Indian
boy, then it is because you established the frequency *a priori* that you
observed a certain frequency (which happens to be consistent with "random") a posteriori. Your matrix experiments do not establish a frequency beforehand, so the information from your standard experiments does not help you one way or the other.
5. Linda, you wrote: “You seem to be claiming that your work does not encounter skepticism among your parapsychological colleagues.” „It is my impression that your colleagues' readiness to accept your work as factually possible, regardless of the absence of reasonable controls, is why the field has not shown any real progress in persuading anyone but themselves that psi can be demonstrated.”
I do not claim that my work does not encounter skepticism among my parapsychology colleagues. On the contrary, I have encountered much skepticism among my colleagues because I do not share their traditional dogmatic attitude regarding methodical precautions. Some editors, e.g. John Palmer, editor of the Journal of Parapsychology, rejected two submitted papers, the editor of European Journal of Parapsychology rejected two other papers.
I am very pleased to hear that. A lot of my impression of parapsychology
comes from those who have popularized it - such as Dean Radin, Honorton,
Jessica Utts, etc. It is reassuring to hear that the lax attitude to
methodological rigour is not shared by all in the profession, particularly
that it is not shared by some of the editors to the main journals of
publication.
I am curious, though. What is your opinion of those parapsychologists whose
attitude is that methodological rigour is useful and even crucial? Are they
similarly subject to "the power of human...disbelief"?
My strategy is “postcaution”, the idea that conceivable non-psi effects can be tested by comparing results with and without the seemingly incautious condition. So I tested the hypothetical just-drawn-number advantage which proved to be effectless. I tested the right hand over left hand advantage which was effectless. I tested many more conditions and did not find any hint that deviations from chance could be explained by ordinary mechanisms.
Suitbert
And I think that is the problem in a nutshell, as you have not demonstrated
that this method allows you to discover deviations from chance that could be
explained by ordinary mechanisms, and our previous experience with this
strategy in other fields of science shows it to be misguided.
Linda
