ISF Logo   IS Forum
Forum Index Register Members List Events Mark Forums Read Help

Go Back   International Skeptics Forum » General Topics » General Skepticism and The Paranormal
 


Welcome to the International Skeptics Forum, where we discuss skepticism, critical thinking, the paranormal and science in a friendly but lively way. You are currently viewing the forum as a guest, which means you are missing out on discussing matters that are of interest to you. Please consider registering so you can gain full use of the forum features and interact with other Members. Registration is simple, fast and free! Click here to register today.
Tags Daryl Bem

Reply
Old 2nd November 2010, 05:07 AM   #81
fls
Penultimate Amazing
 
Join Date: Jan 2005
Posts: 10,226
Originally Posted by Arouet View Post
Here's a paper that has apparently been submitted for publication to the JPSP critically looking at the Bem paper. One proponent on Skeptiko has already called it "a load of crap dressed up in a nice pdf package", but I thought that I would post it anyway, to see what the thoughts are here.

http://dl.dropbox.com/u/1018886/Bem6.pdf
I've read it. They cover a number of points which have already been made here about Bem's paper specifically and parapsychology in general. I might quibble over some of their details (for example, the choice of prior probability at 10^-20 seems excessive), but it is otherwise generally sound. An interesting discussion could be had over the extent to which psychology needs an evidence-based overhaul. This is something where we would benefit from the input of those within the field here. For example, it's not enough to show that researchers write up their results in a way that puts them into a better light than they deserve. This happens in any field to some extent. What we want to know is the extent to which their colleagues are fooled by this. In medicine, part of the evidence-based overhaul included putting the ability to recognize this sort of trickery into the hands of physicians, so that the way in which research is written up doesn't necessarily reflect what practicing physicians take away from that research.

What specific criticisms have been leveled at it?

Linda
fls is offline   Quote this post in a PM   Nominate this post for this month's language award Copy a direct link to this post Reply With Quote Back to Top
Old 2nd November 2010, 09:37 AM   #82
Arouet
Thinker
 
Arouet's Avatar
 
Join Date: Apr 2010
Location: Toronto
Posts: 173
Originally Posted by fls View Post
What specific criticisms have been leveled at it?

Linda
There were only a few posts on this in that thread since it was a hijack. I may start a discussion thread on it once I've had a chance to read the paper in more detail.

Just wanted to say that I appreciate your analysis of these matters. Your detial is fantastic, even though you make my head spin sometimes!
Arouet is offline   Quote this post in a PM   Nominate this post for this month's language award Copy a direct link to this post Reply With Quote Back to Top
Old 2nd November 2010, 10:09 AM   #83
GnaGnaMan
Graduate Poster
 
Join Date: Sep 2008
Posts: 1,707
I, too, find nothing substantially wrong with it. Caveat: If there is anything wrong with the Bayes factors they calculate, I wouldn't have noticed it.
I'm eager to hear the counter arguments.

Two (scientist) bloggers have also written critiques which raise many of the same points, not quite coincidentally, I guess.

http://www.psychologytoday.com/blog/...n-precognition

http://www.psychologytoday.com/blog/...rs-really-mean

http://www.psychologytoday.com/blog/...omena-research
GnaGnaMan is offline   Quote this post in a PM   Nominate this post for this month's language award Copy a direct link to this post Reply With Quote Back to Top
Old 2nd November 2010, 10:20 AM   #84
Arouet
Thinker
 
Arouet's Avatar
 
Join Date: Apr 2010
Location: Toronto
Posts: 173
This discussion, as well as the one about the dog experiment seems to highlight just how vulnerable these types of studies are to statistical manipulation. Frankly, for a lay person, how are we to differentiate between these different statistical interpretations? Everyone seems to be writing from a position of authority and accusing the others of not knowing what they are doing!

Is it even feasible for the scientific community to reach a consensus on the proper way to proceed? I can't imagine that this problem lies only with paraspychology. It must permeate into the mainstream sciences as well.
Arouet is offline   Quote this post in a PM   Nominate this post for this month's language award Copy a direct link to this post Reply With Quote Back to Top
Old 2nd November 2010, 10:31 AM   #85
Beth
Philosopher
 
Beth's Avatar
 
Join Date: Dec 2004
Posts: 5,598
Originally Posted by Arouet View Post
This discussion, as well as the one about the dog experiment seems to highlight just how vulnerable these types of studies are to statistical manipulation. Frankly, for a lay person, how are we to differentiate between these different statistical interpretations? Everyone seems to be writing from a position of authority and accusing the others of not knowing what they are doing!

Is it even feasible for the scientific community to reach a consensus on the proper way to proceed? I can't imagine that this problem lies only with paraspychology. It must permeate into the mainstream sciences as well.
Well, they did title their paper Why Psychologists Must Change the Way They Analyze
Their Data: The Case of Psi


and conclude with this:

Quote:
our assessment suggests that something is deeply wrong with the way experimental psychologists design their studies and report their statistical results. It is a disturbing thought that many experimental findings, proudly and confidently reported in the literature as real, might in fact be based on statistical tests that are explorative and biased. We hope the Bem article will become a signpost for change, a writing on the wall: psychologists must change the way they analyze their data.
My conclusion is that BEM's evidence for psi in this paper is equal to or better than the evidence presented in many psychological papers for less controversial theories.
__________________
Beth
"You are not the stuff of which you are made."
Richard Dawkins, July 2005, 10:45

http://www.ted.com/talks/richard_daw..._universe.html
Beth is offline   Quote this post in a PM   Nominate this post for this month's language award Copy a direct link to this post Reply With Quote Back to Top
Old 2nd November 2010, 11:40 AM   #86
fls
Penultimate Amazing
 
Join Date: Jan 2005
Posts: 10,226
Originally Posted by Arouet View Post
This discussion, as well as the one about the dog experiment seems to highlight just how vulnerable these types of studies are to statistical manipulation. Frankly, for a lay person, how are we to differentiate between these different statistical interpretations?
That's a good question. I dont have an answer for you.

Quote:
Is it even feasible for the scientific community to reach a consensus on the proper way to proceed? I can't imagine that this problem lies only with paraspychology. It must permeate into the mainstream sciences as well.
Not so much. Speaking about medicine...you might find multiple comparisons in exploratory research, but it won't be reported as the main effect and will be followed up with confirmatory research, sometimes as part of the same article. Failure to include control groups is considered a fault. Analyses are based on the empirical information, rather than the expected results of random-number generators. For example, drug trials are analyzed based on how many people were actually randomized into each group, not on the 'mean chance expectation' that equal numbers would be in each group. Failure to include control groups is considered a fault. Much work is put into establishing prior probability so that experiments are generally performed in the setting of even odds. And a failure to include control groups is viewed as a fault.

I don't know the state of affairs in psychology, but it would be interesting to hear from people who would know.

Linda
fls is offline   Quote this post in a PM   Nominate this post for this month's language award Copy a direct link to this post Reply With Quote Back to Top
Old 3rd November 2010, 05:53 AM   #87
blobru
Philosopher
 
blobru's Avatar
 
Join Date: May 2007
Posts: 6,900
Originally Posted by Paul C. Anagnostopoulos View Post
I thought folks might be interested in Daryl Bem's recent experiments, to be published in Journal of Personality and Social Psychology.

http://dbem.ws/FeelingFuture.pdf

~~ Paul
Rereading Bem's write-up a little closer, I'm noticing what may be a basic flaw with experiments 3 & 4: ("Retroactive Priming I & II"). In each experiment, there are 32 trials: 16 congruous and 16 incongruous. The data seem to show that participants categorize quicker with congruous than incongruous 'retro-primes'. But note that, given the even split between congruous and incongruous pairings of picture-affect and retro-prime, it is more likely that a congruous pairing will have been preceded by an incongruous pairing, and vice versa. Allowing for the first pairing, which is either congruous or incongruous and preceded by nothing, on average the participants will experience an extra incongruity (incongruous to congruous or congruous to incongruous VS congruous to congruous or incongruous to incongruous) between pairings. Iow, each participant on average will see and be asked to categorize one more congruous pairing after seeing and categorizing an incongruous pairing than congruous after congruous; or, see and categorize one more incongruous pairing after a congruous pairing than incongruous after incongruous.

Note especially that this means that on average for every two participants there will have been one extra congruous pairing followed by an incongruous pairing, and one extra incongruous pairing followed by a congruous pairing.

So what? Well, given this, we can easily construct an alternate, non-psi hypothesis: that the difference in response times is not a psi effect; but rather, an averaged measure of the effect of categorizing the extra congruous after incongruous pairing and the extra incongruous after congruous pairing. That is, it may be that anomalies in participant response-times are tied to these extra incongruities between pairings (affect & prime) over the run of the experiment, and not to psi-retro-primed congruities or incongruities within the pairings themselves (as a possible non-psi mechanism: having just seen an incongruous pairing, where the picture and prime don't match, the participant may experience some tension, and be more likely to categorize the next picture -- which is more likely to be part of a congruous pairing -- more quickly in order to move on from the tension; conversely, having just seen a congruous pairing, the participant may be pleased, and be less likely to categorize the next picture -- more likely to be part of an incongruous pairing -- more quickly, not wanting to move past the lingering pleasure).
__________________
"Say to them, 'I am Nobody!'" -- Ulysses to the Cyclops

"Never mind. I can't read." -- Hokulele to the Easter Bunny

Last edited by blobru; 3rd November 2010 at 05:59 AM.
blobru is offline   Quote this post in a PM   Nominate this post for this month's language award Copy a direct link to this post Reply With Quote Back to Top
Old 3rd November 2010, 06:06 AM   #88
fls
Penultimate Amazing
 
Join Date: Jan 2005
Posts: 10,226
Originally Posted by blobru View Post
Rereading Bem's write-up a little closer, I'm noticing what may be a basic flaw with experiments 3 & 4: ("Retroactive Priming I & II"). In each experiment, there are 32 trials: 16 congruous and 16 incongruous. The data seem to show that participants categorize quicker with congruous than incongruous 'retro-primes'. But note that, given the even split between congruous and incongruous pairings of picture-affect and retro-prime, it is more likely that a congruous pairing will have been preceded by an incongruous pairing, and vice versa. Allowing for the first pairing, which is either congruous or incongruous and preceded by nothing, on average the participants will experience an extra incongruity (incongruous to congruous or congruous to incongruous VS congruous to congruous or incongruous to incongruous) between pairings. Iow, each participant on average will see and be asked to categorize one more congruous pairing after seeing and categorizing an incongruous pairing than congruous after congruous; or, see and categorize one more incongruous pairing after a congruous pairing than incongruous after incongruous.

Note especially that this means that on average for every two participants there will have been one extra congruous pairing followed by an incongruous pairing, and one extra incongruous pairing followed by a congruous pairing.

So what? Well, given this, we can easily construct an alternate, non-psi hypothesis: that the difference in response times is not a psi effect; but rather, an averaged measure of the effect of categorizing the extra congruous after incongruous pairing and the extra incongruous after congruous pairing. That is, it may be that anomalies in participant response-times are tied to these extra incongruities between pairings (affect & prime) over the run of the experiment, and not to psi-retro-primed congruities or incongruities within the pairings themselves (as a possible non-psi mechanism: having just seen an incongruous pairing, where the picture and prime don't match, the participant may experience some tension, and be more likely to categorize the next picture -- which is more likely to be part of a congruous pairing -- more quickly in order to move on from the tension; conversely, having just seen a congruous pairing, the participant may be pleased, and be less likely to categorize the next picture -- more likely to be part of an incongruous pairing -- more quickly, not wanting to move past the lingering pleasure).
That's a good point. It's all part of the larger picture that these experiments almost all include an effect occurring in the usual direction, in addition to the proposed pre-cognitive effect. And this obviously larger effect has not been accounted for in the analyses.

Linda
fls is offline   Quote this post in a PM   Nominate this post for this month's language award Copy a direct link to this post Reply With Quote Back to Top
Old 3rd November 2010, 10:15 AM   #89
Paul C. Anagnostopoulos
Nap, interrupted.
 
Paul C. Anagnostopoulos's Avatar
 
Join Date: Aug 2001
Posts: 19,145
On Monday I again asked Bem for the data from experiment 1. I have not heard back from him.

~~ Paul
__________________
Millions long for immortality who do not know what to do with themselves on a rainy Sunday afternoon. ---Susan Ertz

RIP Mr. Skinny, Tim
Paul C. Anagnostopoulos is offline   Quote this post in a PM   Nominate this post for this month's language award Copy a direct link to this post Reply With Quote Back to Top
Old 3rd November 2010, 10:55 AM   #90
Beth
Philosopher
 
Beth's Avatar
 
Join Date: Dec 2004
Posts: 5,598
Originally Posted by Paul C. Anagnostopoulos View Post
On Monday I again asked Bem for the data from experiment 1. I have not heard back from him.

~~ Paul
You might want to check your email address. I wrote him last week-end and got a form reply Monday. In it he states:

Originally Posted by BEM
Finally, some of you have asked for my raw data. I am a firm believer in providing access to all data—especially in such a controversial area. But it will take me a while to get to those requests. (And I have to strip the data files of all subject-identifying information). I am giving priority to the replication packages right now. So please be patient.
__________________
Beth
"You are not the stuff of which you are made."
Richard Dawkins, July 2005, 10:45

http://www.ted.com/talks/richard_daw..._universe.html
Beth is offline   Quote this post in a PM   Nominate this post for this month's language award Copy a direct link to this post Reply With Quote Back to Top
Old 3rd November 2010, 12:51 PM   #91
Ersby
Fortean
 
Join Date: Sep 2001
Posts: 1,881
That's good to hear. He sent me all the PRL ganzfeld data when I asked for them, too.
__________________
"Once a man admits complete and unshakeable faith in his own integrity, he is in an excellent frame of mind to be approached by con men." David W. Maurer, "The Big Con"
Ersby is offline   Quote this post in a PM   Nominate this post for this month's language award Copy a direct link to this post Reply With Quote Back to Top
Old 3rd November 2010, 12:54 PM   #92
Paul C. Anagnostopoulos
Nap, interrupted.
 
Paul C. Anagnostopoulos's Avatar
 
Join Date: Aug 2001
Posts: 19,145
Originally Posted by Beth
You might want to check your email address. I wrote him last week-end and got a form reply Monday.
The rest of the world can reply to my email address, so he may just have missed my emails or they never got to him. PM me the email address you used for him.

Why would the data files have subject-identifying information, other than subj1, subj2, etc.?

~~ Paul
__________________
Millions long for immortality who do not know what to do with themselves on a rainy Sunday afternoon. ---Susan Ertz

RIP Mr. Skinny, Tim

Last edited by Paul C. Anagnostopoulos; 3rd November 2010 at 12:55 PM.
Paul C. Anagnostopoulos is offline   Quote this post in a PM   Nominate this post for this month's language award Copy a direct link to this post Reply With Quote Back to Top
Old 10th November 2010, 03:28 PM   #93
Paul C. Anagnostopoulos
Nap, interrupted.
 
Paul C. Anagnostopoulos's Avatar
 
Join Date: Aug 2001
Posts: 19,145
Beth: Have you heard anything more from Bem?

~~ Paul
__________________
Millions long for immortality who do not know what to do with themselves on a rainy Sunday afternoon. ---Susan Ertz

RIP Mr. Skinny, Tim
Paul C. Anagnostopoulos is offline   Quote this post in a PM   Nominate this post for this month's language award Copy a direct link to this post Reply With Quote Back to Top
Old 11th November 2010, 05:33 AM   #94
Beth
Philosopher
 
Beth's Avatar
 
Join Date: Dec 2004
Posts: 5,598
Originally Posted by Paul C. Anagnostopoulos View Post
Beth: Have you heard anything more from Bem?

~~ Paul
No
__________________
Beth
"You are not the stuff of which you are made."
Richard Dawkins, July 2005, 10:45

http://www.ted.com/talks/richard_daw..._universe.html
Beth is offline   Quote this post in a PM   Nominate this post for this month's language award Copy a direct link to this post Reply With Quote Back to Top
Old 11th November 2010, 01:05 PM   #95
Almo
Masterblazer
 
Almo's Avatar
 
Join Date: Aug 2005
Location: Montreal, Quebec
Posts: 6,842
Researcher says we can see the future

Just saw this.

http://www.newscientist.com/article/...published.html

Quote:
In another study, Bem adapted research on "priming" – the effect of a subliminally presented word on a person's response to an image. For instance, if someone is momentarily flashed the word "ugly", it will take them longer to decide that a picture of a kitten is pleasant than if "beautiful" had been flashed. Running the experiment back-to-front, Bem found that the priming effect seemed to work backwards in time as well as forwards.
This has passed peer review at the Journal of Personality and Social Psychology. The guy himself says it sounds goofy.

I'm not sure I'm impressed by it, but I found it interesting anyway.
__________________
Almo!
My Music Blog
"No society ever collapsed because the poor had too much." — LeftySergeant
"It may be that there is no body really at rest, to which the places and motions of others may be referred." –Issac Newton in the Principia
Almo is offline   Quote this post in a PM   Nominate this post for this month's language award Copy a direct link to this post Reply With Quote Back to Top
Old 11th November 2010, 02:54 PM   #96
Brainache
Nasty Brutish and Tall
 
Brainache's Avatar
 
Join Date: Aug 2006
Location: Canberra
Posts: 17,271
Researcher says we can see the future ...

Tomorrow...
Brainache is offline   Quote this post in a PM   Nominate this post for this month's language award Copy a direct link to this post Reply With Quote Back to Top
Old 11th November 2010, 03:53 PM   #97
Gord_in_Toronto
Penultimate Amazing
 
Gord_in_Toronto's Avatar
 
Join Date: Jul 2006
Posts: 18,705
The Word of the Day is: Replication. I'll wait.

__________________
"Reality is what's left when you cease to believe." Philip K. Dick
Gord_in_Toronto is offline   Quote this post in a PM   Nominate this post for this month's language award Copy a direct link to this post Reply With Quote Back to Top
Old 11th November 2010, 04:42 PM   #98
Skeptical Greg
Agave Wine Connoisseur
 
Skeptical Greg's Avatar
 
Join Date: Jul 2002
Location: Just past ' Resume Speed ' .
Posts: 15,641
'Saw that coming..

Oh, wait..
__________________
" The main problem I have with the idea of heaven, is the thought of
spending eternity with most of the people who claim to be going there. "
Skeptical Greg is offline   Quote this post in a PM   Nominate this post for this month's language award Copy a direct link to this post Reply With Quote Back to Top
Old 11th November 2010, 05:38 PM   #99
rjh01
Gentleman of leisure
Tagger
 
rjh01's Avatar
 
Join Date: May 2005
Location: Flying around in the sky
Posts: 24,829
I wonder how far into the future they can see? Maybe a few hours? Maybe it can be used to predict tatts results? Big money for all if it works.
rjh01 is offline   Quote this post in a PM   Nominate this post for this month's language award Copy a direct link to this post Reply With Quote Back to Top
Old 11th November 2010, 05:57 PM   #100
Jeff Corey
New York Skeptic
 
Jeff Corey's Avatar
 
Join Date: Aug 2001
Posts: 13,714
There's another thread which was started a few weeks ago about that.
Jeff Corey is offline   Quote this post in a PM   Nominate this post for this month's language award Copy a direct link to this post Reply With Quote Back to Top
Old 11th November 2010, 06:20 PM   #101
Gord_in_Toronto
Penultimate Amazing
 
Gord_in_Toronto's Avatar
 
Join Date: Jul 2006
Posts: 18,705
Originally Posted by Jeff Corey View Post
There's another thread which was started a few weeks ago about that.
Are you sure? I thought it was going to start next week.
__________________
"Reality is what's left when you cease to believe." Philip K. Dick
Gord_in_Toronto is offline   Quote this post in a PM   Nominate this post for this month's language award Copy a direct link to this post Reply With Quote Back to Top
Old 11th November 2010, 09:22 PM   #102
Steve001
Graduate Poster
 
Join Date: Feb 2007
Posts: 1,533
A Replication of the Procedures from Bem (2010, Study 8)...


...and a Failure to Replicate the Same Results
http://papers.ssrn.com/sol3/papers.c...act_id=1699970

A preliminary study.

Last edited by Steve001; 11th November 2010 at 09:23 PM.
Steve001 is offline   Quote this post in a PM   Nominate this post for this month's language award Copy a direct link to this post Reply With Quote Back to Top
Old 11th November 2010, 11:17 PM   #103
Jeff Corey
New York Skeptic
 
Jeff Corey's Avatar
 
Join Date: Aug 2001
Posts: 13,714
Originally Posted by Steve001 View Post

...and a Failure to Replicate the Same Results
http://papers.ssrn.com/sol3/papers.c...act_id=1699970

A preliminary study.
Thanks for the reference. Ever since Linda asked me to check this out a few weeks ago, I've been puzzling over a cleaner way to falsify this. I'm going to pick one of his tests and see if I can get some students interested in helping me run the systematic replication.
But I do have a niggle. Who will publish all the negative results?

Last edited by Jeff Corey; 11th November 2010 at 11:25 PM.
Jeff Corey is offline   Quote this post in a PM   Nominate this post for this month's language award Copy a direct link to this post Reply With Quote Back to Top
Old 11th November 2010, 11:53 PM   #104
Robin
Penultimate Amazing
 
Join Date: Apr 2004
Posts: 10,665
Originally Posted by Gord_in_Toronto View Post
The Word of the Day is: Replication. I'll wait.

Not to mention better experimental design.
__________________
The non-theoretical character of metaphysics would not be in itself a defect; all arts have this non-theoretical character without thereby losing their high value for personal as well as for social life. The danger lies in the deceptive character of metaphysics; it gives the illusion of knowledge without actually giving any knowledge. This is the reason why we reject it. - Rudolf Carnap "Philosophy and Logical Syntax"
Robin is offline   Quote this post in a PM   Nominate this post for this month's language award Copy a direct link to this post Reply With Quote Back to Top
Old 12th November 2010, 05:51 AM   #105
Paul C. Anagnostopoulos
Nap, interrupted.
 
Paul C. Anagnostopoulos's Avatar
 
Join Date: Aug 2001
Posts: 19,145
Originally Posted by Jeff Corey
Thanks for the reference. Ever since Linda asked me to check this out a few weeks ago, I've been puzzling over a cleaner way to falsify this. I'm going to pick one of his tests and see if I can get some students interested in helping me run the systematic replication.
But I do have a niggle. Who will publish all the negative results?
Well, you can certainly publish them here. But submit them to the same journal as Bem.

~~ Paul
__________________
Millions long for immortality who do not know what to do with themselves on a rainy Sunday afternoon. ---Susan Ertz

RIP Mr. Skinny, Tim
Paul C. Anagnostopoulos is offline   Quote this post in a PM   Nominate this post for this month's language award Copy a direct link to this post Reply With Quote Back to Top
Old 12th November 2010, 10:13 AM   #106
GnaGnaMan
Graduate Poster
 
Join Date: Sep 2008
Posts: 1,707
Originally Posted by Jeff Corey View Post
Thanks for the reference. Ever since Linda asked me to check this out a few weeks ago, I've been puzzling over a cleaner way to falsify this. I'm going to pick one of his tests and see if I can get some students interested in helping me run the systematic replication.
But I do have a niggle. Who will publish all the negative results?
The original journal will be hard pressed to come up with an excuse not to follow up.

There's also this:
http://failuretoreplicate.com/
Failure to Replicate is, as the name suggests, a database of failed attempts to replicate previously-reported psychological findings.

It seems to have completely failed to take off. Quite a pity.
GnaGnaMan is offline   Quote this post in a PM   Nominate this post for this month's language award Copy a direct link to this post Reply With Quote Back to Top
Old 12th November 2010, 05:40 PM   #107
Peristarkawan
New Blood
 
Join Date: Aug 2008
Posts: 7
Originally Posted by Woomaster View Post
If it were true that people could indeed foresee future events, could this be explained within a materialistic worldview? What do you think?
Possibly. You should visit Wikipedia and read up on the arrow of time. The basic gist is that there are many different methods we can use to distinguish the past from the future, but as far as we can tell, they all essentially boil down to thermodynamics.

If that is true, then it logically follows that entropy is the only real obstacle preventing us from predicting the future. We can easily see into the past because the past has lower entropy than the present, but it is difficult to see into the future because the future has higher entropy than the present. As an analogy, it would be like riding a bike -- going downhill is easy, but climbing up a steep hill takes a lot of work.
Peristarkawan is offline   Quote this post in a PM   Nominate this post for this month's language award Copy a direct link to this post Reply With Quote Back to Top
Old 12th November 2010, 10:32 PM   #108
Rramjet
Guest
 
Join Date: Nov 2008
Posts: 3,046
Originally Posted by Jeff Corey View Post
Thanks for the reference. Ever since Linda asked me to check this out a few weeks ago, I've been puzzling over a cleaner way to falsify this. I'm going to pick one of his tests and see if I can get some students interested in helping me run the systematic replication.
But I do have a niggle. Who will publish all the negative results?
Yes of course! That is precisley how science should be conducted. Announce the predetermined results and only then undertake an experiment to find exactly according to what the experimental bias has been set up to predetermine. Nice. I knew there was something missing from my concept of scientific methodology!
Rramjet is offline   Quote this post in a PM   Nominate this post for this month's language award Copy a direct link to this post Reply With Quote Back to Top
Old 13th November 2010, 04:18 AM   #109
GnaGnaMan
Graduate Poster
 
Join Date: Sep 2008
Posts: 1,707
Originally Posted by blobru View Post
Rereading Bem's write-up a little closer, I'm noticing what may be a basic flaw with experiments 3 & 4: ("Retroactive Priming I & II"). In each experiment, there are 32 trials: 16 congruous and 16 incongruous.
I think you misunderstand Bem on this point. My understanding is that the number of congruous vs. incongruous is not predetermined.

Nevertheless, that's an interesting idea you introduce. I'm pretty positive that it is possible to come up with an alternative explanation based on making the right assumptions about between trials priming.
The tricky thing is that these assumptions would have to match what is known about priming. The implication would be that a lot of mainstream work done, using a similar priming paradigm, would become worthless.
GnaGnaMan is offline   Quote this post in a PM   Nominate this post for this month's language award Copy a direct link to this post Reply With Quote Back to Top
Old 13th November 2010, 09:13 AM   #110
Garrette
Penultimate Amazing
 
Join Date: Aug 2001
Posts: 14,763
Originally Posted by Rramjet View Post
Yes of course! That is precisley how science should be conducted. Announce the predetermined results and only then undertake an experiment to find exactly according to what the experimental bias has been set up to predetermine. Nice. I knew there was something missing from my concept of scientific methodology!
Not that I am a scientist, but I suppose I approximate one as closely as you, but I think you completely misunderstand Jeff Corey's use of the term "falsify."
__________________
My kids still love me.
Garrette is offline   Quote this post in a PM   Nominate this post for this month's language award Copy a direct link to this post Reply With Quote Back to Top
Old 13th November 2010, 09:24 AM   #111
roger
Penultimate Amazing
 
roger's Avatar
 
Join Date: May 2002
Posts: 11,465
Originally Posted by Garrette View Post
Not that I am a scientist, but I suppose I approximate one as closely as you, but I think you completely misunderstand Jeff Corey's use of the term "falsify."
To be fair, I think he was referring to the last sentence, where Jeff questioned who would publish the negative results.

In reality, the world many of us try to live in, scientists very often have a very good idea about how an experiment is going to turn out. They, of course, are open to being wrong about it. Who hasn't read in a press release "We are very excited. This experiment confirmed our expectation that the quasar ....".

It's just a cheap shot at a professional.
__________________
May your trails be crooked, winding, lonesome, dangerous, leading to the most amazing view. May your mountains rise into and above the clouds. - Edward Abbey

Climb the mountains and get their good tidings.
Nature's peace will flow into you as sunshine flows into trees. The winds will blow their own freshness into you, and the storms their energy, while cares will drop off like autumn leaves. - John Muir
roger is offline   Quote this post in a PM   Nominate this post for this month's language award Copy a direct link to this post Reply With Quote Back to Top
Old 13th November 2010, 09:28 AM   #112
Garrette
Penultimate Amazing
 
Join Date: Aug 2001
Posts: 14,763
Originally Posted by roger View Post
To be fair, I think he was referring to the last sentence, where Jeff questioned who would publish the negative results.
On re-reading, I have to agree.

My apologies to Rramjet.


Originally Posted by roger
In reality, the world many of us try to live in, scientists very often have a very good idea about how an experiment is going to turn out. They, of course, are open to being wrong about it. Who hasn't read in a press release "We are very excited. This experiment confirmed our expectation that the quasar ....".

It's just a cheap shot at a professional.
I can't improve on this response.
__________________
My kids still love me.
Garrette is offline   Quote this post in a PM   Nominate this post for this month's language award Copy a direct link to this post Reply With Quote Back to Top
Old 13th November 2010, 09:47 AM   #113
technoextreme
Illuminator
 
technoextreme's Avatar
 
Join Date: Jun 2007
Posts: 3,785
Originally Posted by Jeff Corey View Post
Thanks for the reference. Ever since Linda asked me to check this out a few weeks ago, I've been puzzling over a cleaner way to falsify this. I'm going to pick one of his tests and see if I can get some students interested in helping me run the systematic replication.
You probably don't have to. Check the statistical tests used to reject the null hypothesis. If the criticism in Psychology Today is correct I think he ********** up the statistics.
EDIT:
SERIOUSLY?????? I was right. Really????? How the hell does an electrical engineer who has very limited experience with statistical analysis know more than a supposed scientist??? Wait I know.
Originally Posted by Gord_in_Toronto View Post
The Word of the Day is: Replication. I'll wait.

Actually the word of the day is make sure there is an effect before replication.
__________________
It's amazing how many of these "paranormal" icons seem to merge together. There always seem to be theories about how they link together in some way. I'm sure someone has a very good explanation as to how Bigfoot killed JFK to help cover Roswell.-Mark Mekes
This isn't rocket surgery.-Bill Nye

Last edited by technoextreme; 13th November 2010 at 10:08 AM.
technoextreme is offline   Quote this post in a PM   Nominate this post for this month's language award Copy a direct link to this post Reply With Quote Back to Top
Old 13th November 2010, 11:06 AM   #114
Beth
Philosopher
 
Beth's Avatar
 
Join Date: Dec 2004
Posts: 5,598
Originally Posted by technoextreme View Post
You probably don't have to. Check the statistical tests used to reject the null hypothesis. If the criticism in Psychology Today is correct I think he ********** up the statistics.
EDIT:
SERIOUSLY?????? I was right. Really????? How the hell does an electrical engineer who has very limited experience with statistical analysis know more than a supposed scientist??? Wait I know.
What criticism of the statistics are you talking about? The worst thing the linked article said was "some of the null hypotheses are rejected only by way of the flat-footed one-tailed test".
__________________
Beth
"You are not the stuff of which you are made."
Richard Dawkins, July 2005, 10:45

http://www.ted.com/talks/richard_daw..._universe.html
Beth is offline   Quote this post in a PM   Nominate this post for this month's language award Copy a direct link to this post Reply With Quote Back to Top
Old 13th November 2010, 11:17 AM   #115
technoextreme
Illuminator
 
technoextreme's Avatar
 
Join Date: Jun 2007
Posts: 3,785
Originally Posted by Beth View Post
What criticism of the statistics are you talking about? The worst thing the linked article said was "some of the null hypotheses are rejected only by way of the flat-footed one-tailed test".
Yeah if you knew anything about statistics its almost tantamount to faking the results. When you use the one tailed test you are making assumptions that your data will show you a specific result. In general you can't make that assumption. For example in medicine you can't assume that a new medicine will make you better. It could do worst. That in turn means you have to use a two tailed test or something similar than that. The issue with the two tailed test is that its actually harder to reject the null hypothesis as opposed to the one tailed test. There are lies, dam lies, and statistics and in this case he went for the easiest trick in the book to make your data look legitimate.
__________________
It's amazing how many of these "paranormal" icons seem to merge together. There always seem to be theories about how they link together in some way. I'm sure someone has a very good explanation as to how Bigfoot killed JFK to help cover Roswell.-Mark Mekes
This isn't rocket surgery.-Bill Nye

Last edited by technoextreme; 13th November 2010 at 11:21 AM.
technoextreme is offline   Quote this post in a PM   Nominate this post for this month's language award Copy a direct link to this post Reply With Quote Back to Top
Old 13th November 2010, 11:20 AM   #116
Beth
Philosopher
 
Beth's Avatar
 
Join Date: Dec 2004
Posts: 5,598
Originally Posted by technoextreme View Post
Yeah if you knew anything about statistics its almost tantamount to faking the results.
Why would you say that? I'm reasonably well versed in statistics and I don't see it that way.
__________________
Beth
"You are not the stuff of which you are made."
Richard Dawkins, July 2005, 10:45

http://www.ted.com/talks/richard_daw..._universe.html
Beth is offline   Quote this post in a PM   Nominate this post for this month's language award Copy a direct link to this post Reply With Quote Back to Top
Old 13th November 2010, 11:26 AM   #117
technoextreme
Illuminator
 
technoextreme's Avatar
 
Join Date: Jun 2007
Posts: 3,785
Originally Posted by Beth View Post
Why would you say that? I'm reasonably well versed in statistics and I don't see it that way.
You make assumptions in a one tailed test that I can't possibly fathom being possible in this scenario especially when he admits he has no *********** clue what is going on. In that case the test should revert to a two tailed test but he did the statistics using the one tailed test.
__________________
It's amazing how many of these "paranormal" icons seem to merge together. There always seem to be theories about how they link together in some way. I'm sure someone has a very good explanation as to how Bigfoot killed JFK to help cover Roswell.-Mark Mekes
This isn't rocket surgery.-Bill Nye

Last edited by technoextreme; 13th November 2010 at 11:28 AM.
technoextreme is offline   Quote this post in a PM   Nominate this post for this month's language award Copy a direct link to this post Reply With Quote Back to Top
Old 13th November 2010, 12:25 PM   #118
Gord_in_Toronto
Penultimate Amazing
 
Gord_in_Toronto's Avatar
 
Join Date: Jul 2006
Posts: 18,705
Originally Posted by technoextreme View Post

<SNIP>

Actually the word of the day is make sure there is an effect before replication.
It's sure going to be damn hard to replicate if there is no effect.
__________________
"Reality is what's left when you cease to believe." Philip K. Dick
Gord_in_Toronto is offline   Quote this post in a PM   Nominate this post for this month's language award Copy a direct link to this post Reply With Quote Back to Top
Old 13th November 2010, 01:23 PM   #119
blobru
Philosopher
 
blobru's Avatar
 
Join Date: May 2007
Posts: 6,900
Originally Posted by GnaGnaMan View Post
I think you misunderstand Bem on this point. My understanding is that the number of congruous vs. incongruous is not predetermined.
On rerereading Bem's write-up even a little more closely... you're right. For "Retroactive Priming I" the number of positive and negative affects and primes is evenly split at 16 apiece, but the mix is randomized (which does make it possible to anticipate congruous or incongruous towards the end of the trials by 'card-counting' so to speak, but "Retroactive Priming II" appears to correct for this with open deck sampling).

Quote:
Nevertheless, that's an interesting idea you introduce. I'm pretty positive that it is possible to come up with an alternative explanation based on making the right assumptions about between trials priming.
The tricky thing is that these assumptions would have to match what is known about priming. The implication would be that a lot of mainstream work done, using a similar priming paradigm, would become worthless.
Possibly, but if so the alternate non-psi mechanism is not as obvious as my misreading implied (I'll have to think about it, but I think randomization should balance out the between-trials priming I was complaining about). In any event, the other experiments in Bem's study don't suggest any non-psi mechanism (that I can see), experimenters having moved on to replication (with one failure to already cited by Steve001).
__________________
"Say to them, 'I am Nobody!'" -- Ulysses to the Cyclops

"Never mind. I can't read." -- Hokulele to the Easter Bunny
blobru is offline   Quote this post in a PM   Nominate this post for this month's language award Copy a direct link to this post Reply With Quote Back to Top
Old 13th November 2010, 02:29 PM   #120
Beth
Philosopher
 
Beth's Avatar
 
Join Date: Dec 2004
Posts: 5,598
Originally Posted by technoextreme View Post
You make assumptions in a one tailed test that I can't possibly fathom being possible in this scenario especially when he admits he has no *********** clue what is going on. In that case the test should revert to a two tailed test but he did the statistics using the one tailed test.
The only different assumption is whether you are predicting a particular direction for the change. It's arguable about whether or not he made the prediction in advance, but he did cite previous non-retroactive experiments that established the expected direction. I don't think that sort of difference of opinion on technical details is in any way "tantamount to faking the results".
__________________
Beth
"You are not the stuff of which you are made."
Richard Dawkins, July 2005, 10:45

http://www.ted.com/talks/richard_daw..._universe.html
Beth is offline   Quote this post in a PM   Nominate this post for this month's language award Copy a direct link to this post Reply With Quote Back to Top
Reply

International Skeptics Forum » General Topics » General Skepticism and The Paranormal

Bookmarks

Thread Tools

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Forum Jump


All times are GMT -7. The time now is 08:52 PM.
Powered by vBulletin. Copyright ©2000 - 2019, Jelsoft Enterprises Ltd.

This forum began as part of the James Randi Education Foundation (JREF). However, the forum now exists as
an independent entity with no affiliation with or endorsement by the JREF, including the section in reference to "JREF" topics.

Disclaimer: Messages posted in the Forum are solely the opinion of their authors.