Chris Carter Exposes Richard Wiseman Misdeeds

[alex.tsakiris]

(Hi from your friends at www.skeptiko.com )

JSPR has published a penetrating critique by Chris Carter of Richard Wiseman's latest attack on parapsychology - his 'Heads I win, tails you lose' paper published in Skeptical Inquirer. Chris Carter's reply:
http://www.skepticalinvestigations.o...er_Wiseman.pdf

[Ersby]

This was mentioned on Rational Skepticism too, so I'll just post what I said there:

Chris Carter's paper is a bit strange. Instead of rebutting Wiseman's claims of what parapsychologists do to maintain significant results, he simply says "Well, you do it too!" which is fine as a playground squabble, but leaves the initial claims untouched.

He's misleading on a few things. He says Wiseman doesn't offer a shred of evidence regarding cherry-picking new methods, but in Wiseman's paper, he references a paper by Caroline Watt. So that's a shred of evidence, no?

Regarding the experimenter effect, Carter only references the Wiseman/Schlitz work and doesn’t even mention the follow-up paper in which the effect no longer seemed to occur (Schlitz, Wiseman et al, 2006) though I’d expect he’s heard of it.

Regarding the debate about Jay-tee, I’d say that Wiseman’s position isn’t very strong, and I do wonder what he was expecting to prove with just four trials. (I could hazard a guess, though)

The stuff Chris Carter writes on meta-analyses is just plain wrong. He says M&W used a statistical measure that didn’t take sample size into account, but Milton & Wiseman used z-scores (which includes the standard deviation which is linked to sample size) and the weighted z (or Effect Size ES – my least favourite statistical measure) which is the z-score divided by the square root of the number of trials. So it is taken into account, although a binomial distribution would’ve been better. (I should also point out that in the past Radin, Utts and the most recent meta-analysis by Storm et al have also used Effect Size ES on individual experiments, but he doesn't seem to be complaining about them!)

Meanwhile, he mentions that Dalton’s work was published two years before M&W’s meta-analysis was published. This isn’t quite true. It was presented at a PA Convention (it’s never been published in a peer-reviewed journal) in 1997, and so was Milton and Wiseman’s meta-analysis. Plus, the pdf of the PA article of M&W’s work states it was received in June 1997. (Of course, that leads into debates about the haste with which they went to print... etc etc.)

I find Chris Carter quite a tedious writer. He presents such an incomplete picture of the debate that it’s depressing just to wade through it all.

[Garrette]

I'm glad this thread is here. I read this article by Carter, though I have yet to go back to read Wiseman's original, and was wondering how much merit was in it. As I'm not by the slightest stretch a statistician, I am unable to dissect it as Ersby has begun.

That said, the comments on Sheldrake and Jaytee did seem to have some merit--not that the original experiments showed real significance but that Wiseman's replication was not properly handled/discussed.

[fls]

Originally Posted by alex.tsakiris

(Hi from your friends at www.skeptiko.com )

JSPR has published a penetrating critique by Chris Carter of Richard Wiseman's latest attack on parapsychology - his 'Heads I win, tails you lose' paper published in Skeptical Inquirer. Chris Carter's reply:
http://www.skepticalinvestigations.o...er_Wiseman.pdf

That's disappointing. Many of the claims are invalid or are directed at criticizing Wiseman, rather than addressing Wiseman's criticisms.

The issue about unpublished results (and the recent discussion of Bem's paper shows how extensive the 'exploratory' null results might be) is dismissed by quoting the Fail Safe N, which Carter must know by now is not valid. After all Scargle published the work showing the Fail Safe N to be a gross overestimation in almost all cases in a parapsychology journal (The Journal of Scientific Exploration) over ten years ago.

Carter continues to support the nonsensical idea that statistics based on chance are relevant to our expectations. Who cares whether Nadia performed in a way which would be unexpected if it were due to chance? Who would seriously propose that matching a person to a condition under non-blinded conditions wouldn't involve the use of clues? Or Sheldrake's experiments which were analyzed using the assumption that the dog's behaviour when Pam wasn't returning home was random, when their own observations showed it wasn't - his baseline behaviour matched the pattern they were searching for regardless of the movements of his owner.

The comments on the ganzfeld metanalysis are almost bizarre, as Wiseman simply used the same method of combining studies as had been used in prior metanalyses by parapsychologists (and continues to be used by them). Plus the measure does have some input from sample size. And the practise of combining all the hits and misses as one big study is an invalid technique anyway (the only way it could be treated as valid is if psi doesn't exist), especially since we have discovered that the base rate probably differs from study to study (per prior discussions).

And then we get to the bizarre conclusion at the end which seems to consist of "Wiseman is a jerk therefore psi is real".

Who exactly is this supposed to persuade?

Linda

[Rodney]

Originally Posted by Ersby

[i]The stuff Chris Carter writes on meta-analyses is just plain wrong. He says M&W used a statistical measure that didn’t take sample size into account, but Milton & Wiseman used z-scores (which includes the standard deviation which is linked to sample size) and the weighted z (or Effect Size ES – my least favourite statistical measure) which is the z-score divided by the square root of the number of trials. So it is taken into account, although a binomial distribution would’ve been better. (I should also point out that in the past Radin, Utts and the most recent meta-analysis by Storm et al have also used Effect Size ES on individual experiments, but he doesn't seem to be complaining about them!)

Maybe that's because the latter knew what they were doing? See http://www.internationalskeptics.com...5&postcount=21

[fls]

Originally Posted by Garrette

I'm glad this thread is here. I read this article by Carter, though I have yet to go back to read Wiseman's original, and was wondering how much merit was in it. As I'm not by the slightest stretch a statistician, I am unable to dissect it as Ersby has begun.

That said, the comments on Sheldrake and Jaytee did seem to have some merit--not that the original experiments showed real significance but that Wiseman's replication was not properly handled/discussed.

I have picked through Sheldrake's paper and Wiseman's paper, and there doesn't seem to be much merit in these criticisms. For one thing, the characterization of Wiseman's criteria is grossly misleading, and if they (Carter and Sheldrake) believe their characterizations, it shows a surprising ignorance of validity and reliability in the choice of outcome measures.

I can link to some threads where this was discussed.

http://www.internationalskeptics.com...81#post5379381

Here is one link, but I'm not sure how useful it will be since apparently you read it already.

Linda

[fls]

Originally Posted by Rodney

Maybe that's because the latter knew what they were doing? See http://www.internationalskeptics.com...5&postcount=21

Yeah, I remember in a prior discussion with you that the there were differences between my calculations and the numbers listed in these metanalyses (not just Wiseman's, as the same numbers were used in other analyses). I suggested we shouldn't trust the numbers as written without investigating them. But I haven't bothered since then as we can't draw any valid conclusions about whether psi exists from those studies anyway. Not that I seem to be averse to pointless statistical analyses.

Linda

[alex.tsakiris]

Originally Posted by Ersby

This was mentioned on Rational Skepticism too, so I'll just post what I said there:

Chris Carter's paper is a bit strange. Instead of rebutting Wiseman's claims of what parapsychologists do to maintain significant results, he simply says "Well, you do it too!" which is fine as a playground squabble, but leaves the initial claims untouched.

He's misleading on a few things. He says Wiseman doesn't offer a shred of evidence regarding cherry-picking new methods, but in Wiseman's paper, he references a paper by Caroline Watt. So that's a shred of evidence, no?

Regarding the experimenter effect, Carter only references the Wiseman/Schlitz work and doesn’t even mention the follow-up paper in which the effect no longer seemed to occur (Schlitz, Wiseman et al, 2006) though I’d expect he’s heard of it.

Regarding the debate about Jay-tee, I’d say that Wiseman’s position isn’t very strong, and I do wonder what he was expecting to prove with just four trials. (I could hazard a guess, though)

The stuff Chris Carter writes on meta-analyses is just plain wrong. He says M&W used a statistical measure that didn’t take sample size into account, but Milton & Wiseman used z-scores (which includes the standard deviation which is linked to sample size) and the weighted z (or Effect Size ES – my least favourite statistical measure) which is the z-score divided by the square root of the number of trials. So it is taken into account, although a binomial distribution would’ve been better. (I should also point out that in the past Radin, Utts and the most recent meta-analysis by Storm et al have also used Effect Size ES on individual experiments, but he doesn't seem to be complaining about them!)

Meanwhile, he mentions that Dalton’s work was published two years before M&W’s meta-analysis was published. This isn’t quite true. It was presented at a PA Convention (it’s never been published in a peer-reviewed journal) in 1997, and so was Milton and Wiseman’s meta-analysis. Plus, the pdf of the PA article of M&W’s work states it was received in June 1997. (Of course, that leads into debates about the haste with which they went to print... etc etc.)

I find Chris Carter quite a tedious writer. He presents such an incomplete picture of the debate that it’s depressing just to wade through it all.

Good starting points for a discussion. I'm going to email Chris Carter and see if he will weigh-in.

[Ersby]

Originally Posted by Rodney

Maybe that's because the latter knew what they were doing? See http://www.internationalskeptics.com...5&postcount=21

Or perhaps they didn't? http://www.internationalskeptics.com....php?p=5713571 (start at post 116 to watch Storm et al's work unravel)

As far as I can tell: Wiseman and Milton calculated the stouffer z for each experiment, and then calculated another stouffer z for all the individual stouffer zs. Usually people do a stouffer z of individual z-scores.

[alex.tsakiris]

Originally Posted by fls

I have picked through Sheldrake's paper and Wiseman's paper, and there doesn't seem to be much merit in these criticisms.

You better explain since your opinion seems to be in the minority... even among skepitcs.

[fls]

Originally Posted by alex.tsakiris

You better explain since your opinion seems to be in the minority... even among skepitcs.

I edited the post and added a link to the thread in which I discuss this in more detail.

Also, since I'm not sure that I explained it all that well, I will elaborate on this again.

Linda

[Rodney]

Originally Posted by Ersby

Or perhaps they didn't? http://www.internationalskeptics.com....php?p=5713571 (start at post 116 to watch Storm et al's work unravel)

As far as I can tell: Wiseman and Milton calculated the stouffer z for each experiment, and then calculated another stouffer z for all the individual stouffer zs. Usually people do a stouffer z of individual z-scores.

But all Wiseman's and Milton's negative Stouffer Z figures were wrong, and I demonstrated that, using the binomial distribution for the studies that they analyzed, the results were statistically significant.

[Ersby]

Their figures weren't wrong, given the statistical measure they'd chosen. But the statistical measure, it could be argued, was wrong.

EDIT: wait a sec... I remember now. There were some results that looked odd. I'll have to check in a couple of days. Not at home at the mo'.

[Paul C. Anagnostopoulos]

So here's something I think about frequently: Is statistics really this difficult? Or are there "master statisticians" who could take a look at a psi experiment or group of experiments and pass judgment on the statistical methods chosen? Masters who the rest of the community would trust?

Because if not, I can't help but be reminded of this:

http://lesswrong.com/lw/1ib/parapsyc...p_for_science/

~~ Paul

[fls]

Wiseman's selection of criteria:

The initial observation was that Pam's parents could tell when Pam was returning because of the way her dog behaved. This observation will be complicated by the fact that this occurs in the presence of knowledge about when Pam is expected to return and knowledge of when she has returned. Also, as the videotapes showed, there is wide variation in the behaviour of the dog depending upon time of day, which house he is in, how busy it is inside and outside the house, people coming and going to the door, some knowledge of Pam's routines, etc.

I'm a physician so I am familiar with looking for patterns in signs and symptoms in order to form diagnoses. And it is typical to start with the open-ended approach Sheldrake uses whereby you look for the pattern you expect to see, and if it is present, consider that your diagnosis is correct. This isn't a particularly relaible method, however. Even if you are careful about not letting your biases creep into the gathering of information, it turns out, once you apply an evidence-based approach, that you will be wrong often enough for it to be a problem. So instead, we depend on the criteria-based approach Wiseman uses to improve the reliability and validity of our conclusions. Instead of saying, if someone is pregnant, I expect them to have nausea and vomiting plus a missed period, we ask, what us the probability they are pregnant if they have nausea and vomiting plus a missed period? Instead of looking for the pattern we'd expect if the dog knew when its owner was returning, we ask, what is the probability their owner is returning when they exhibit this behavior?

Wiseman chose a criteria for success based on discussions with the family as to how they could tell when Pam was returning based on the behaviour of the dog. It turned out that this behavior was not associated with her return. He altered the criteria based on further discussion, and it still wasn't associated with a return. Now it may be that the criteria needed refinement.. And four experiments is a small number to completely rule out any effect. However, the impression given was that this behavior was consistent, not that it was so sporadic that it would easily be absent on four trials.

Regardless of whether or not the criteria were sufficiently refined and the number of trials sufficiently large, the approach chosen by Wiseman was a valid and reliable means of discovering whether the dog's behaviour predicted the owner's return compared to Sheldrake's confirmatory approach. So to call criteria based on the claims made by the parents "arbitrary" is a mischaracterization. And he certainly doesn't deserve the heavy criticism for choosing to stick with methods which are more reliable and valid, instead of discarding them in favor of less reliable methods because they give you the answer you want.

Linda

[fls]

Originally Posted by Paul C. Anagnostopoulos

So here's something I think about frequently: Is statistics really this difficult? Or are there "master statisticians" who could take a look at a psi experiment or group of experiments and pass judgment on the statistical methods chosen? Masters who the rest of the community would trust?

Because if not, I can't help but be reminded of this:

http://lesswrong.com/lw/1ib/parapsyc...p_for_science/

~~ Paul

Heh. I've said the very same thing about homeopathy.

This isn't a statistics problem though. It's more of a research methods problem and more about understanding exactly which hypothesis is being tested in order to choose the appropriate methods, rather than knowing how to apply the methods once chosen. I don't know what field that is. It wasn't part of my statistics courses. I took courses in research methodologies, epidemiology, etc. as part of my graduate studies in public health. I would presume that they are part of other graduate programs. I was taught by people from departments of anthropology, psychology, education, medicine, and economics, as far as I recall. No statisticians except for my statistics classes. No philosophers either. But I don't know if that's representative.

Linda

[fls]

Originally Posted by Ersby

Or perhaps they didn't? http://www.internationalskeptics.com....php?p=5713571 (start at post 116 to watch Storm et al's work unravel)

As far as I can tell: Wiseman and Milton calculated the stouffer z for each experiment, and then calculated another stouffer z for all the individual stouffer zs. Usually people do a stouffer z of individual z-scores.

They calculated an effect size for each experiment (the z score divided by N^1/2), but this isn't what they combined for the Stouffer z. The Stouffer z used the sum of individual z scores divided by the sqrt of the number of experiments. This is based on the numbers I get when performing those calculations matching up with the reported numbers (and not matching up when done the other way).

Linda

[Ersby]

Okay. I'm not at home, so I'm working from memory here, so there's every possibility I'm wrong.

[fls]

Originally Posted by Rodney

But all Wiseman's and Milton's negative Stouffer Z figures were wrong, and I demonstrated that, using the binomial distribution for the studies that they analyzed, the results were statistically significant.

It may interest you to know that the only way it can be presumed that combining all the hits and misses into one big study and testing for significance using the binomial distribution is valid, is if you presume that psi does not have an effect. Otherwise, it is an invalid test.

So which is it?

Linda

[fls]

Originally Posted by Ersby

Okay. I'm not at home, so I'm working from memory here, so there's every possibility I'm wrong.

Okay. I had to go back and check it myself. Too confusing to rely on memory.

Linda

[Rodney]

Originally Posted by fls

It may interest you to know that the only way it can be presumed that combining all the hits and misses into one big study and testing for significance using the binomial distribution is valid, is if you presume that psi does not have an effect. Otherwise, it is an invalid test.

1) How so?
2) Do you think what Wiseman and Milton did was valid?

[alex.tsakiris]

Originally Posted by Ersby

Chris Carter's paper is a bit strange...

Pls all ow me to re-post from the skeptiko forum:
http://forum.mind-energy.net/skeptik...html#post36805

Richard Wiseman writes ...

Quote:

Quote:
Heads I Win, Tails You Lose. How Parapsychologists Nullify Null Results

To my knowledge, only one paper has revealed an insight into the potential scale of this problem. Watt (2006) summarized all of the psi-related final-year undergraduate projects that have been supervised by staff at Edinburgh University’s Koestler Parapsychology Unit between 1987 and 2007.....

Note: Parapsychologist Caroline Watt is the long-term partner (or wife?) of Richard Wiseman.

Quote:

Quote:
...Interestingly, Watt’s paper also demonstrated a reporting bias. Only seven of the thirty-eight studies had made it into the public domain, presented as papers at conferences held by the Parapsychological Association. ...

- Richard Wiseman

Reporting bias against 'student' projects? Gosh, what a shocker. The PA might be biased against reporting student projects, in favor of qualified researchers? I'd be far more worried if it was publication bias against post graduates and staff studies.

Quote:

Quote:
.....Watt’s analysis, although informative, underestimates the total number of psi-related studies undertaken at Edinburgh University because it did not include projects undertaken by students prior to their final year ....

- Richard Wiseman

Is Richard Wiseman psychic? How does he know the projects by students prior to their final year were non-significant? Why didn't his partner Watt include the others?

Quote:

Quote:
it did not include projects ... experiments run by postgraduate students and staff,

- Richard Wiseman

What a shame because according to the Fortean Times article below ..... major studies by the postgraduate students and staff.... were successful.

Quote:

Quote:
Even stage magicians were consulted to help guard against trickery. Between 1993 and 2003, six out of nine major experimental studies produced statistically significant results. When asked directly if he personally believed in telepathy, Morris replied that he just did the research – but added that there was accumulating evidence indicating that it does occur. “We are perhaps studying nature, but in its fuller form,” he suggested. And then, in the late summer of 2004, Bob Morris went and died. Suddenly. Unexpectedly.

- Fortean Times

It seems there has been no 'major' reported experimental projects at the university since Caroline Watt and magician Peter Lamont took charge after 2004? Must be low on funding, I guess.

Quote:

Quote:
or any work conducted before 1987.

- Richard Wiseman

I think Wiseman must be referring to the early student work under Dr John Beloff?. Why is Wiseman assuming these were non-significant studies? Why were these not included?

Quote:

Quote:
To help the field move forward and rapidly reach closure on the psi question , parapsychologists need to make four important changes in the way they view null findings.

- Richard Wiseman

Rapidly reach closure? Why does Wiseman want to rapidly reach closure? Wiseman repeats this in the Skeptiko interview ...

Quote:

Quote:
'...So the article was really about how we might try to push that forward and try and get some closure, i.e., can we produce the level of evidence one would need, and if not, do we agree not to carry on with the endeavor? '

- Richard Wiseman

So Wiseman wants to reach closure on parapsychology. It seems Wiseman wants to 'rapidly reach closure' on his partner Watt's career as a parapsychologist?

[alex.tsakiris]

Originally Posted by fls

Wiseman's selection of criteria:

The initial observation was that Pam's parents could tell when Pam was returning because of the way her dog behaved. This observation will be complicated by the fact that this occurs in the presence of knowledge about when Pam is expected to return and knowledge of when she has returned...

This is silly. I've done about 5 hours of show/interviews with all the players in this experiment (Sheldrake and Wiseman twice each).

First off, this was Sheldrake's experiment. He had been working with Jay-tee and Pam for over a year when he invited Wiseman in (he even let him use his camera). Wiseman changed the rules/criteria of the experiment to his ridiculous (and arbitrary) 2 minute cut-off crtiera that he invented... he then hid from Shedlrake and published the data without consulting the lead investigator/collaborator... really bad form. 10 years later he admitted (on skeptiko) that his data matches Sheldrake's (in a very convincing way as the Carter paper shows).

The data speaks much louder and more clearly than anything else.

[fls]

Originally Posted by Rodney

1) How so?

To combine all the hits, you have to presume that the base rate is the same, that is, that the effect in each study is the same. This means that Bem has to be wrong. That the standardness of the study has no effect. It also means that any of the other measures which are associated with variations in hit rate are not measuring variations in psi.

Quote:

2) Do you think what Wiseman and Milton did was valid?

Yes, reasonably so. It isn't clear to me how the various studies should be weighted. I'm not sure that sample size is adequate, since it doesn't look like at least one of the larger studies is considered better quality. Since the Dalton study hasn't ever been published, it suggests that it's not up to snuff. Variance may be a better choice, but of course, Wiseman can't suggest this, otherwise he will be criticized. It's best he just goes along with what everyone else is doing.

Linda

[Wowbagger]

This is ridiculous! Either your results get stronger with more controls or weaker.

If they only get weaker, either the phenomenon you are trying to observe is not real; or you need to refine a testable hypothesis to account for those results, and see if you can strengthen the signal around that. No Testability = "Explained Away"

If you are actually able to isolate and measure a phenomenon. And, the signal for that phenomenon actually gets stronger with the more controls you apply: THEN some scientists might sit up and take notice. IF they are reliably replicable, by independent parties, you win big!

Until that happens, your criticism of the critics is pointless.

[Beth]

Originally Posted by fls

It may interest you to know that the only way it can be presumed that combining all the hits and misses into one big study and testing for significance using the binomial distribution is valid, is if you presume that psi does not have an effect. Otherwise, it is an invalid test.
So which is it?
Linda

That doesn't make sense. The null hypothesis assumes that there is no psi affect. That is, the statistics are analyzed presuming that there is no effect and if the p-value is sufficiently low, we can then reject that null hypothesis. There may be other issues, but I don't understand your complaint here.

Originally Posted by fls

To combine all the hits, you have to presume that the base rate is the same, that is, that the effect in each study is the same. This means that Bem has to be wrong. That the standardness of the study has no effect. It also means that any of the other measures which are associated with variations in hit rate are not measuring variations in psi.

Again, I'm not clear on what the issue is with doing this. Why would he not presume the base rate was the same for the null hypothesis?

If someone could quote this post, I'd appreciate it as I think Linda has me on ignore.

Beth

[Garrette]

Originally Posted by alex.tsakiris

You better explain since your opinion seems to be in the minority... even among skepitcs.

I'm a skeptic who has concerns raised by Carter's article, but don't count me in that number who make fls a minority (for the record, I don't think she is a minority).

[I wrote a lengthy post but erased it after some thought because there are really only two relevant points]

1. If you prove absolutely that Wiseman's critique was wrong and he is a monstrous hypocrite (something which is a long way from being demonstrated, let alone proven), then you still have done absolutely nothing to demonstrate the validity of Sheldrake's findings.

2. If you prove that Sheldrake performed his experiments and analysis correctly and his conclusions follow from the experiment, then you still are a long way from demonstrating psi; if the experiment is so ground-breaking, then why haven't the parapsychologists replicated it?

[fls]

Originally Posted by alex.tsakiris

This is silly. I've done about 5 hours of show/interviews with all the players in this experiment (Sheldrake and Wiseman twice each).

First off, this was Sheldrake's experiment. He had been working with Jay-tee and Pam for over a year when he invited Wiseman in (he even let him use his camera). Wiseman changed the rules/criteria of the experiment to his ridiculous (and arbitrary) 2 minute cut-off crtiera that he invented... he then hid from Shedlrake and published the data without consulting the lead investigator/collaborator... really bad form. 10 years later he admitted (on skeptiko) that his data matches Sheldrake's (in a very convincing way as the Carter paper shows).

The data speaks much louder and more clearly than anything else.

I'm sorry. I don't understand what this has to do with what I said. Regardless of whether any of that is true, it doesn't change the issue of which methodologies can reliably be used to discover if the dog's behaviour predicts Pam's return.

Linda

[fls]

Originally Posted by Beth

That doesn't make sense. The null hypothesis assumes that there is no psi affect. That is, the statistics are analyzed presuming that there is no effect and if the p-value is sufficiently low, we can then reject that null hypothesis. There may be other issues, but I don't understand your complaint here.

Again, I'm not clear on what the issue is with doing this. Why would he not presume the base rate was the same for the null hypothesis?

If someone could quote this post, I'd appreciate it as I think Linda has me on ignore.

Beth

I explain about the issues of combining studies here:

http://www.internationalskeptics.com...08#post4636708

Linda

[cj.23]

As I had already done this over on RatSkep, and it may be useful for those trying to follow the discussion a little reference for those who want to reach an educated opinion on the debate between Wiseman and Sheldrake on psychic dogs--


Latest article by Chris Carter from JSPR - http://www.sheldrake.org/D&C/controv...er_Wiseman.pdf

The background

http://www.sheldrake.org/Articles&Pa.../dog_video.pdf - Sheldrake's paper

http://www.richardwiseman.com/Jaytee.html is Wiseman's account

http://www.sheldrake.org/D&C/controversies/wiseman.html is Sheldrake's response


Listen to the participants speak

Skeptico podcasts on the issue featuring Wiseman & Sheldrake on the dog experiments
http://www.skeptiko.com/11-dr-richar...-dogsthatknow/
http://www.skeptiko.com/35-dr-steven...know-research/
http://www.skeptiko.com/rupert-sheld...wiseman-clash/

Read much more on the issue, and take part in designing replications - Open Source Science
http://www.opensourcescience.net/ind...%27s_Return%3F


Short Bibliography

Sheldrake, R. (1998). A dog that seems to know when his owner is returning: Preliminary investigations. Journal of the Society for Psychical Research, 62, 220-232.
http://www.sheldrake.org/Articles&Pa...f/dogknows.pdf

Wiseman, R., Smith, M. & Milton, J. (1998) Can animals detect when their owners are returning home? An experimental test of the 'psychic pet' phenomenon. British Journal of Psychology 89, 453-462.
https://uhra.herts.ac.uk/dspace/bits...5/1/902380.pdf

Sheldrake, R. (1999a) Commentary on a paper by Wiseman, Smith and Milton on the 'psychic pet' phenomenon. JSPR 63, 306-311.
http://www.sheldrake.org/Articles&Pa...df/comment.pdf

Sheldrake, R. (1999b) Dogs that Know When Their Owners are Coming Home. London: Hutchinson.

Wiseman, R., Smith, M. & Milton, J. (2000) The 'psychic pet' phenomenon: A reply to Rupert Sheldrake. JSPR 64, 46-49.
http://www.psy.herts.ac.uk/wiseman/p...icdogreply.pdf

Sheldrake, R, Smart, P (2000) A Dog That Seems To Know When His Owner is Coming Home:
Videotaped Experiments and Observations, Journal of Scientific Exploration 14, 233-255
http://www.sheldrake.org/Articles&Pa.../dog_video.pdf

Sheldrake, R., and Smart, P. (2000b). Testing a return-anticipating dog, Kane. Anthrozoös, 13(4), 203-212.
http://www.sheldrake.org/Articles&Pa...df/dogkane.pdf

Carter C (2010) Heads I win, Tails you lose, or how Richard Wiseman nullifoes positive results in parapsychology, and what to do about it JSPR 74, 156-2007
http://www.sheldrake.org/D&C/controv...er_Wiseman.pdf

Hope of interest
cj x

[fls]

Thanks cj x. That's handier than googling and combing through old threads.

Linda

[Beth]

Originally Posted by fls

I explain about the issues of combining studies here:

http://www.internationalskeptics.com...08#post4636708

Linda

Thanks for responding to my post. Yes, this old post of yours discusses some issues with combining studies. But this does not address the question that I asked. What problem do you perceive with the analysis as result of assuming the null hypothesis is true?

[Ersby]

It might be helpful to post a link to Wiseman's paper, too.
http://www.csicop.org/si/show/heads_..._null_results/

Originally Posted by alex.tsakiris

Is Richard Wiseman psychic? How does he know the projects by students prior to their final year were non-significant? Why didn't his partner Watt include the others?

He doesn't say they were non-significant.

Quote:

What a shame because according to the Fortean Times article below ..... major studies by the postgraduate students and staff.... were successful.

Now, usually I like to refer to first hand reports, so I'm acutely aware that I'm responding to someone who's quoting someone else who's quoting someone else who's reporting someone else's data (without a reference!). Nevertheless, I'll have a go.

The nine "major" experiments are probably the nine ganzfeld experiments. As such, I'd need to know where else these are called "major" and by what criteria.

There is an overlap in these "major" experiments and the experiments listed in Watt's paper, which is why I suspect this "major" label is fairly meaningless in this context. Also, it is important to bear in mind that both Watts and Wiseman are talking about all psi experiments, whereas these "major" experiments are all ganzfeld work.

Quote:

It seems there has been no 'major' reported experimental projects at the university since Caroline Watt and magician Peter Lamont took charge after 2004? Must be low on funding, I guess.

Again, until I know what you mean by "major", I can't comment.

[GnaGnaMan]

Originally Posted by alex.tsakiris

(Hi from your friends at www.skeptiko.com )

JSPR has published a penetrating critique by Chris Carter of Richard Wiseman's latest attack on parapsychology - his 'Heads I win, tails you lose' paper published in Skeptical Inquirer. Chris Carter's reply:
http://www.skepticalinvestigations.o...er_Wiseman.pdf

As has been been pointed out, Carter's reply is mainly a Tu Quoque fallacy. It does not address the meat of Wiseman's "attack".
BTW the criticisms that Wiseman expresses are by no means new. They have been variously expressed by other people in the past and in greater depth (and altogether better IMHO). For example Alcock's Give the null hypothesis a chance.

Publication Bias
Publication bias cannot be proven. A meta-analysis may detect certain tell-tale signs of it but that is far from conclusive. The skew of outcomes that is detected in the latest Storm et al. meta-analysis is certainly such a tell-tale sign.
The attempts to determine whether a file-drawer-effect may be responsible for the entirety of the observed effect do only that. They do not rule out availability bias being responsible for part of the effect. That is true for Blackmore's survey as well as statistical methods.
As an aside, Wiseman doesn't claim what Carter attempts to rebut. Quite the opposite.

Demkina case
It is not clear if Carter says that Wiseman should have changed the success criterion post hoc (a big NO, obviously) or if he thinks that Wiseman should have employed a different one from the start.
There are problems with the protocol, even though not of the kind that Carter claims.
Demkina was allowed to inspect the subjects will all normal senses. It is not surprising that she was able to assign the diseases better than would usually be expected if she had just acted randomly. She was simply not acting randomly and there is no reason to think she should have.
In favor of such an approach one could validly argue that even though she could use normal senses, a very great skill at diagnosing would be interesting even if not paranormal.
If Carter believes that what she displayed was really diagnosing skill great enough to be interesting he can go investigate it himself. I certainly don't think so.
http://www.csicop.org/si/show/natash...h_normal_eyes/
http://www.csicop.org/si/show/testing_natasha/
http://www.csicop.org/specialarticle...st_of_natasha/

The psychic dog
Again, Carter displays a a great unfamiliarity with the literature.
Wiseman's success criterion was not arbitrary but made in consultation with the owners of the dog. The claim that a psychic dog would respond telepathically to the owner's decision to return home had been made widely on television.
It has since been repeated. There has been an abortive, unpublished (and obviously unsuccesful) replication attempt by Alex Tsakiris. At the time he promoted 2 videos claiming that the dog reacted at the time the owner set out to go home.
In short there is no problem with Wiseman's criterion (even if there are some valid critcisms of the experiment).
Sheldrake's criterion is simply defunct. By that criterion we should judge an action made by the dog as influenced telepathically by the owner even if the owner has not made the decision yet that supposedly causes the action. Huh?

Ganzfeld Meta-analysis
The simple truth is that Wiseman did precisely the same thing that previous meta-analyses had done.
Again, there are actual problems with the study. It fails, for example, to make tests for availability bias and other such issues.


In summaryt, Carter's attack piece is an ill-informed and fallacious piece of fail.
It does little but making the idea of wide-spread incompetence and fraud within parapsychology more credible.

[Rodney]

Originally Posted by fls

To combine all the hits, you have to presume that the base rate is the same, that is, that the effect in each study is the same. This means that Bem has to be wrong. That the standardness of the study has no effect. It also means that any of the other measures which are associated with variations in hit rate are not measuring variations in psi.

I don't know if we're talking past one another, so let me ask a question: If ten parapsychology researchers simultaneously undertook ganzfeld experiments with identical protocols (including 25% probability of success) and each recorded, say, 28 hits in 100 trials, would it be valid or invalid to aggregate those results? If invalid, why?

Originally Posted by fls

Yes, reasonably so.

So why are all of Wiseman's and Milton's negative Stouffer Z figures wrong?

[Ersby]

Originally Posted by Ersby

The nine "major" experiments are probably the nine ganzfeld experiments. As such, I'd need to know where else these are called "major" and by what criteria.

Okay, someone on skeptiko has posted a link that he says was the reference for the Fortean Times article, which pretty much confirms that Morris was talking about the ganzfeld work.

http://www.scimednet.org/2003-smn-at...ity-of-salford

My source was a powerpoint written by Robert Morris in February 2004 which lists nine ganzfeld experiments, six of which were significant. I figured that the similarity between this and the results from these "major" experiments wasn't just coincidence, and so it turns out to be.

But I'm aware that we're drifting from the point. Chris Carter thought that this paper didn't count as a "shred of evidence." I think it does.

[fls]

Originally Posted by Rodney

I don't know if we're talking past one another, so let me ask a question: If ten parapsychology researchers simultaneously undertook ganzfeld experiments with identical protocols (including 25% probability of success) and each recorded, say, 28 hits in 100 trials, would it be valid or invalid to aggregate those results? If invalid, why?

If you set up something like a multicentre trial, then it is okay to pool the results. But why bring this up, since it doesn't describe the situation at hand or reflect what I said? Do you disagree with the parapsychologists who think that the standardness of the tests, or the creativity of the participants alters the hit rate which represents the effect of psi?

Quote:

So why are all of Wiseman's and Milton's negative Stouffer Z figures wrong?

I may have misunderstood what you were asking. I thought you were asking whether they had chosen a reasonably valid effect size and weighting method, not whether they had performed their calculations carefully. I don't know if their calculations are correct. As I mentioned previously, I found discrepancies between my calculations and the listed numbers (in several metanalyses using those studies, not just Milton and Wiseman) and I haven't followed up on where I think the problem lies. I wouldn't presume any of the numbers in these studies are correct (but I tend to do that anyway). It's not "negative Stouffer Z figures", though. It's "negative z scores". The Stouffer Z is what you get when you combine the z scores.

Linda

[alex.tsakiris]

Originally Posted by GnaGnaMan

The psychic dog
Again, Carter displays a a great unfamiliarity with the literature.
Wiseman's success criterion was not arbitrary but made in consultation with the owners of the dog.

It may have been base on a conversation with the owner, but it's still arbitrary (why a two minute cut off)... and a silly way to get an answer to Sheldrake's research question. Not even Wiseman offers much defense for this "success criteria" when simply measuring time-at-window/time-away-from-window gives you all you need to know.

Quote:

The claim that a psychic dog would respond telepathically to the owner's decision to return home had been made widely on television.
It has since been repeated. There has been an abortive, unpublished (and obviously unsuccesful) replication attempt by Alex Tsakiris. At the time he promoted 2 videos claiming that the dog reacted at the time the owner set out to go home.

I did publish them on Youtube (as you noted). I then funded research at U of Florida... they couldn't go very far with it. This experiemnt is hard to do... lotta people problems. In general, the experience greatly increased my confidence that the phenomena is real.

[Beth]

Originally Posted by alex.tsakiris

It may have been base on a conversation with the owner, but it's still arbitrary (why a two minute cut off)... and a silly way to get an answer to Sheldrake's research question. Not even Wiseman offers much defense for this "success criteria" when simply measuring time-at-window/time-away-from-window gives you all you need to know.

I agree. It's been a while since I looked at this, but my recollection is that Wiseman set up his criteria after the data had been collected rather than prior to setting up the experiment. Do you know if that is the case?

Quote:

I did publish them on Youtube (as you noted). I then funded research at U of Florida... they couldn't go very far with it. This experiemnt is hard to do... lotta people problems. In general, the experience greatly increased my confidence that the phenomena is real.

I find the data presented in the various papers, including Wiseman's data with Sheldrake's analysis, to be evidence for the hypothesis that something is going on. What, exactly, I'm not sure.

[cj.23]

Originally Posted by Beth

I agree. It's been a while since I looked at this, but my recollection is that Wiseman set up his criteria after the data had been collected rather than prior to setting up the experiment. Do you know if that is the case?
.

I'll email Matthew Smith and ask him.

cj x