IS Forum
Forum Index Register Members List Events Search Today's Posts Mark Forums Read Help

Go Back   International Skeptics Forum » General Topics » General Skepticism and The Paranormal
 


Welcome to the International Skeptics Forum, where we discuss skepticism, critical thinking, the paranormal and science in a friendly but lively way. You are currently viewing the forum as a guest, which means you are missing out on discussing matters that are of interest to you. Please consider registering so you can gain full use of the forum features and interact with other Members. Registration is simple, fast and free! Click here to register today.
Tags Daryl Bem

Reply
Old 19th December 2010, 04:31 PM   #361
Rramjet
Guest
 
Join Date: Nov 2008
Posts: 3,046
Originally Posted by marplots View Post
Rramjet, your PSI-Q is quite impressive.
Perhaps... If you accept the assumptions underlying the trial that is ... but I have experienced some "anomalous" things in my life (that is, I seem to be one of those people to whom "truly" anomalous phenomenon occasionally manifest - as opposed to the majority who seem rarely or never to experience such things).
Rramjet is offline   Quote this post in a PM   Nominate this post for this month's language award Copy a direct link to this post Reply With Quote Back to Top
Old 20th December 2010, 09:42 AM   #362
Ersby
Fortean
 
Join Date: Sep 2001
Posts: 1,881
Thanks to Nemo85 on another forum, I see that Bem has written up a response to Alcock's article.

http://dl.dropbox.com/u/8290411/ResponseToAlcock.pdf
__________________
"Once a man admits complete and unshakeable faith in his own integrity, he is in an excellent frame of mind to be approached by con men." David W. Maurer, "The Big Con"
Ersby is offline   Quote this post in a PM   Nominate this post for this month's language award Copy a direct link to this post Reply With Quote Back to Top
Old 20th December 2010, 02:29 PM   #363
Robin
Penultimate Amazing
 
Robin's Avatar
 
Join Date: Apr 2004
Posts: 14,718
Originally Posted by Beth View Post
I'm still not clear on how to interpret your analysis and graph. What is it the graph you posted telling us about BEM's analysis?
Essentially the graphs say that the significance of any DR is closely related to the average number of words recalled during the experiment.

For example in the first experiment, n=99 and DR=2.27, this would be significant at the level Bem claimed (p=0.029) if the average percentage of words recalled was 41%. If the average percentage of words was, for example, 50% then p would be 0.06.

No problem there as far as I know, the average number of words recalled may well be 41%.

In the second experiment Bem halves the number of subjects and adds an exercise to boost the number of words recalled through conventional means.

But according to my analysis the results n=49, DR=4.41 p=0.002 would be consistent with an average recall rate of about 37% of total words, less than the first experiment.

So my suspicion is that Bem's method of analysis does not take into account that statistical significance of DR is dependent upon average number of words recalled.

And I am interested to find the average number of words recalled in both experiments and whether Bem did, in fact, manage to boost the average number of words recalled in the second experiment.

He has not released the data yet, so I have downloaded the source code so that I can see the precise calculation.
__________________
We all hate poverty, war, and injustice
Unlike the rest of you squares.

Tom Lehrer - Folk Song Army
Robin is online now   Quote this post in a PM   Nominate this post for this month's language award Copy a direct link to this post Reply With Quote Back to Top
Old 20th December 2010, 04:34 PM   #364
fls
Penultimate Amazing
 
Join Date: Jan 2005
Posts: 10,226
Originally Posted by Ersby View Post
Thanks to Nemo85 on another forum, I see that Bem has written up a response to Alcock's article.

http://dl.dropbox.com/u/8290411/ResponseToAlcock.pdf
Well, I suppose we're obliged to trust him, but I find it hard to believe that he wouldn't have reported a difference on non-erotic pictures as a confirmation of his hypothesis.

Linda
fls is offline   Quote this post in a PM   Nominate this post for this month's language award Copy a direct link to this post Reply With Quote Back to Top
Old 21st December 2010, 03:52 AM   #365
GnaGnaMan
Graduate Poster
 
Join Date: Sep 2008
Posts: 1,707
Originally Posted by Robin View Post
Essentially the graphs say that the significance of any DR is closely related to the average number of words recalled during the experiment.

For example in the first experiment, n=99 and DR=2.27, this would be significant at the level Bem claimed (p=0.029) if the average percentage of words recalled was 41%. If the average percentage of words was, for example, 50% then p would be 0.06.

No problem there as far as I know, the average number of words recalled may well be 41%.

In the second experiment Bem halves the number of subjects and adds an exercise to boost the number of words recalled through conventional means.

But according to my analysis the results n=49, DR=4.41 p=0.002 would be consistent with an average recall rate of about 37% of total words, less than the first experiment.

So my suspicion is that Bem's method of analysis does not take into account that statistical significance of DR is dependent upon average number of words recalled.

And I am interested to find the average number of words recalled in both experiments and whether Bem did, in fact, manage to boost the average number of words recalled in the second experiment.

He has not released the data yet, so I have downloaded the source code so that I can see the precise calculation.
The average % of words recalled is given in the paper.
In Exp 8:
In both conditions, participants recalled 18.4 (38.3%) of the 48 words in the original list.

In Exp 9:
Similarly, overall recall was again not significantly higher in the experimental sessions (45%) than in the control sessions (44%).

It is also remarked on exp 9 that:
The range of DR% scores was 42% smaller in the replication than in the
original experiment; the variance was 27% smaller.



I would suggest that you try to filter out those simulated experiments that meet these criteria.
GnaGnaMan is offline   Quote this post in a PM   Nominate this post for this month's language award Copy a direct link to this post Reply With Quote Back to Top
Old 21st December 2010, 04:33 AM   #366
cj.23
Master Poster
 
cj.23's Avatar
 
Join Date: Dec 2006
Posts: 2,827
Originally Posted by fls View Post
Well, I suppose we're obliged to trust him, but I find it hard to believe that he wouldn't have reported a difference on non-erotic pictures as a confirmation of his hypothesis.

Linda

He did as I recall in his presentation at Liverpool Hope? I'll try to check means finding the book, though my review of it is in the latest JSPR and might mention this.
__________________
I'm an Anglican Christian, so I declare my prejudice here. Please take it in to account when reading my posts. "Most people would rather die than think: many do." - Betrand Russell

My dull life blogged http://jerome23.wordpress.com
cj.23 is offline   Quote this post in a PM   Nominate this post for this month's language award Copy a direct link to this post Reply With Quote Back to Top
Old 21st December 2010, 08:31 AM   #367
GnaGnaMan
Graduate Poster
 
Join Date: Sep 2008
Posts: 1,707
Evidence of Scientific Misconduct in "Feeling the Future"

The issues presented below are present in the pre-print of Feeling the Future: Experimental Evidence for Anomalous Retroactive Influences on Cognition and Affect by Daryl Bem as retrieved on 21.12.2010 from http://dbem.ws/FeelingFuture.pdf


Misrepresenting other's work
Bem cites a conceptual replication of the experiments on precognitive habituation by Savva et al*1. Bem describes the results as significant above chance with t(24) = 1.70, p = .05
However, the precise p-value is in fact .051 and therefore non-significant. Savva et al. themselves give this more precise figure and correctly describe their findings as non-significant both in the text and the abstract.
Furthermore Savva et al performed further experiments which they presented in 2005 and which are not mentioned by Bem*2. These unambigously failed to replicate any effect. Bem must have been aware of these experiments as Savva et al have cooperated with him. They credit him with providing a version of his software specially modified for him, as well as with developing the spider fear scale which the later, failed experiments used.

Savva et al's work on precognitive habituation failed to produce evidence for an effect, yet Bem presents it quite differently.

Savva has since become disillusioned by attempts to prove the paranormal and has left parapsychology.*3

*1 Savva, L., Child, R., & Smith, M. D. (2004). The precognitive habituation effect: An adaptation using spider stimuli. Paper presented at the 47th Annual Convention of the Parapsychological Association
*2 Savva, L., Child, R., & Smith, M. D. (2005). Further testing of the precognitive habituation effect using spider stimuli. Paper presented at the 48th Annual Convention of the Parapsychological Association
*3 http://www.everythingispointless.com...sychology.html

Misleading statistics
In a reply to James Alcock, Daryl Bem writes the following*1:
"As [Alcock] correctly notes, it is illegitimate and misleading to perform multiple tests on a set of data without adjusting the resulting significance levels to take into account the number of separate analyses conducted. This is well known to experimental psychologists, but, in fact, it does not apply to any of the analyses in my article."

In 2005, Daryl Bem presented a paper on an experiment at a convenction*2. This experiment matches in all details experiment 7 in "Feeling the Future" except in presenting different statistics. None of the tests reported in 2005 are reported in 2010 and vice versa.
The presentation is not cited in "Feeling the Future".
Not only has Bem used misleading statistics on at least this one occasion, he also has falsely denied it.

*1 http://dl.dropbox.com/u/8290411/ResponseToAlcock.pdf
*2 Bem, D. (2005). Precognitive Aversion. Paper presented at the 48th Annual Convention of the Parapsychological Association

Detailed explanation
According to the 2005 account, participants were divided into high and low in arousability.
"To identify participants who were low in Arousability, I used the same question used
in the PH experiments to identify participants who were high in Arousability (“In general, how intense are your emotional reactions to movies, videos, or photographs that are violent, scary, or gruesome?”). Those who scored below 3 on the 5-point scale were defined as low in Arousability."
Arousability played a prominent role in 2005 but is not mentioned in 2010.

The 2005 account also relates that and how subjects with a low boredom tolerance were identified.
"To identify participants who were low in their tolerance for boredom, responses to two questions were averaged: “I get bored easily” [scored in the reverse direction] and “I often enjoy seeing movies that I’ve seen before.” Those who scored below 3 on the combined scale were defined as low in Boredom Tolerance."

In 2010 these question are said to underly a stimulus seeking scale which plays a prominent role in the entire paper.
"To assess stimulus seeking as a correlate of psi performance in our experiments, I constructed a scale comprising the following two statements: “I am easily bored” and “I often enjoy seeing movies I’ve seen before” (reverse scored). Responses were recorded on 5-point scales that ranged from Very Untrue to Very True and averaged into a single score ranging from 1 to 5."
Those who score above the midpoint of the scale are deemed high in stimulus seeking.

The most important difference is that the high stimulus seeking group does not equal the low Boredom Tolerance group. The first includes the latter but also more individuals on top.

In 2005, these hit rates are reported.
1. the overall hit rate (49.1%)
2. for those low in Arousability and Boredom Tolerance on all trials(47.3%, p = .006)
3. for participants low in Arousability on trials with negative targets (46.9%, p = .036)
4. for participants low in Boredom Tolerance on trials with positive targets (44.4%, p = .005)

In 2010 none of these is reported except for the overall hit rate.
Instead reported is a correlation between the now so-called stimulus seeking scale and psi performance.
We also learn that participants "high in stimulus seeking obtained a hit rate significantly below chance, 47.9%, t(95) = -2.11, p = .019, d = .22, binomial z = -1.94, p = .026"
There is no mention of differences related to targets being positive or negative.


Unknown issues
According to a footnote in "Feeling the Future", preliminary results experiments 5 and 6 had previously been presented in 2003*1. A comparison of the accounts given in 2003 and 2010 shows irreconcilable differences. The experiments cannot have happened as described in either or both of the accounts.

*1 Bem, D. Precognitive Habituation: Replicable Evidence for a Process Anomalous Cognition
modified version of a presentation given at the 46th Annual Convention of the Parapsychological Association as retrieved from http://citeseerx.ist.psu.edu/viewdoc...=rep1&type=pdf


Detailed explanation
Experiments 5 and 6 have 48 trials per subject. Experiment 5 does not use erotic images. These are "introduced" in Experiment 6. Experiment 6 also measures erotic stimulus seeking among the participants but for some reason the scale is administered to only 100 of the 150 subjects.
The number of exposures varies between 4 and 10 in both experiments, for no reason that is given.

The 2003 account described 3 experimental series (termed 100, 200 and 300) with 8 experiments in total.
Experimental series 300 (consisting of 2 experiments termed 301 and 302) apparently is the "small retroactive habituation experiment that used supraliminal rather than subliminal exposures" that is mentioned in the File-Drawer section of "Feeling the Future".

Further, experiment 102 had 60 trials for each subject and must have been excluded if we are to trust the description in "Feeling the Future". It is not mentioned how many trials per subject were conducted in experimental series 200 (Experiments 201, 202 and 203).

Strikingly all the individual experiments described in 2003 had a fixed number of exposures which would imply that the experiments described in 2010 are an amalgam of different experiments.
This works for Experiment 6. In 2003 4 experiments (103, 201,202,203) were said to contain erotic pictures. Together these have 150 participants, like experiment 6. Also only 100 of the participants were given a test to measure erotic stimulus seeking, namely those in 201, 202 and 203.

However, only Experiment 101 did not include erotic pictures and it only had 50 participants, rather than 100 as Experiment 5. Also it used a uniform 4 exposures.


URLs for papers mentioned
Savva, L., Child, R., & Smith, M. D. (2004). The precognitive habituation effect: An adaptation using spider stimuli. Paper presented at the 47th Annual Convention of the Parapsychological Association
http://www.parapsych.org/papers/19.pdf

Savva, L., Child, R., & Smith, M. D. (2005). Further testing of the precognitive habituation effect using spider stimuli. Paper presented at the 48th Annual Convention of the Parapsychological Association
http://74.125.155.132/scholar?q=cach...ar.google.com/

Bem, D. (2005). Precognitive Aversion. Paper presented at the 48th Annual Convention of the Parapsychological Association
http://www.psych.cornell.edu/sec/pub...ogAversion.pdf
GnaGnaMan is offline   Quote this post in a PM   Nominate this post for this month's language award Copy a direct link to this post Reply With Quote Back to Top
Old 21st December 2010, 02:57 PM   #368
Robin
Penultimate Amazing
 
Robin's Avatar
 
Join Date: Apr 2004
Posts: 14,718
Originally Posted by GnaGnaMan View Post
The average % of words recalled is given in the paper.
In Exp 8:
In both conditions, participants recalled 18.4 (38.3%) of the 48 words in the original list.

In Exp 9:
Similarly, overall recall was again not significantly higher in the experimental sessions (45%) than in the control sessions (44%).

It is also remarked on exp 9 that:
The range of DR% scores was 42% smaller in the replication than in the
original experiment; the variance was 27% smaller.



I would suggest that you try to filter out those simulated experiments that meet these criteria.
Thanks for that, I missed the averages. You were right earlier about the distribution affecting the significance - but just not to such a great extent as the mean.

An initial look at this suggests that Bem has overstated the significance of the result in experiment 9 - although the result may still be significant.
__________________
We all hate poverty, war, and injustice
Unlike the rest of you squares.

Tom Lehrer - Folk Song Army
Robin is online now   Quote this post in a PM   Nominate this post for this month's language award Copy a direct link to this post Reply With Quote Back to Top
Old 21st December 2010, 03:13 PM   #369
Robin
Penultimate Amazing
 
Robin's Avatar
 
Join Date: Apr 2004
Posts: 14,718
Originally Posted by Robin View Post
Thanks for that, I missed the averages. You were right earlier about the distribution affecting the significance - but just not to such a great extent as the mean.

An initial look at this suggests that Bem has overstated the significance of the result in experiment 9 - although the result may still be significant.
And in fact I think that this is a fairly important point.

All the replication packages are going out with the extra exercise to boost the number of words recalled and all these packages are hard coded with Bem's original calculation.

If I am right, this means that even if none of the replications come up with a statistically significant result any meta analysis will show a massive, spurious significance.
__________________
We all hate poverty, war, and injustice
Unlike the rest of you squares.

Tom Lehrer - Folk Song Army
Robin is online now   Quote this post in a PM   Nominate this post for this month's language award Copy a direct link to this post Reply With Quote Back to Top
Old 21st December 2010, 03:18 PM   #370
Rramjet
Guest
 
Join Date: Nov 2008
Posts: 3,046
Originally Posted by GnaGnaMan View Post
Bem cites a conceptual replication of the experiments on precognitive habituation by Savva et al*1. Bem describes the results as significant above chance with t(24) = 1.70, p = .05
However, the precise p-value is in fact .051 and therefore non-significant.
I cannot let this past without comment. You should consult:

Cohen, J. (1994) The Earth is Round (p < .06). America Psychologist. Vol. 49, No. 12. 997-1003

Anyone who understands anything about statistics will recognise Cohen as a giant in the field. He states:

After 4 decades of severe criticism, the ritual of null hypothesis significance testing – mechanical dichotomous decisions based around a sacred .05 criterion – still persists. This article views the problems with this practice, including its near-universal misinterpretation of p as the probability that H0 is false, the misinterpretation that its compliment is the probability of a successful replication, and the mistaken assumption that if one rejects H0 one thereby affirms the theory that led to the test.”

Cohen goes on to describe (among other objections) – in effect – why the difference between (for example your) .05 and .051 as a scientific decision making tool of the type you delineate is utterly erroneous.

See also:

Kirk, E. (1996) Practical Significance: A concept whose time has come. Educational and Psychological Measurement. Vol. 56, No. 5. 746-759

I will refrain from comment on the rest of your critique, merely to point out that your characterisation as “Misrepresenting other’s work” is scurrilous and your “Scientific misconduct” is itself a misrepresentation - at best.

Last edited by Rramjet; 21st December 2010 at 03:19 PM.
Rramjet is offline   Quote this post in a PM   Nominate this post for this month's language award Copy a direct link to this post Reply With Quote Back to Top
Old 21st December 2010, 05:53 PM   #371
wielange
New Blood
 
Join Date: Nov 2010
Posts: 9
GnaGnaMan, scientific misconduct is quite an accusation, even if, or especially if, done anonymously on this board. Bem seems to be quite ready to reply to emails, and if you really believed that Bem, or anyone else, was committing outright scientific fraud, you might want to contact these people directly before publicly claiming that they indeed committed fraud. Thus is the usual procedure, and it is very basic: check first, accuse later.

I cannot claim that I am entirely convinced by Bem's work, but I don't find it very encouraging if you, or, for that matter, Alcock, just shoot blindly at these findings in the hope that one shot might hit. You are trying to convince ... whom? Yourself? Others? I find Bem's proposal to attempt replications much more rational than any of these armchair attempts to "undo" the findings. Frankly, it reminds me of creationist apologetics, and I find it scary that skepticism would have come to that.

Besides, your claim that Savva, Child, and Smith "unambigously" failed to find any evidence suggestive of what looks like precognitive habituation does not square well with what the content of their papers; have a check first. Hint: read the whole papers, including the 2005 one, not just the abstract. The discussion session of the 2005 paper particularily.
Generally, you may not like all of this, but this itself is not a free ticket to twist and turn things.

Last edited by wielange; 21st December 2010 at 06:36 PM.
wielange is offline   Quote this post in a PM   Nominate this post for this month's language award Copy a direct link to this post Reply With Quote Back to Top
Old 21st December 2010, 08:10 PM   #372
Arouet
Thinker
 
Arouet's Avatar
 
Join Date: Apr 2010
Location: Toronto
Posts: 173
I appreciate reading critique - whether positive or negative - of these studies, though I get lost in much of the science lingo, but I agree with the poster above: it serves little to throw out such inflammatory (and perhaps unfair) language in an otherwise seemingly well-reasoned critique. All it will do is take attention away from the points being raised and result in wasted digression into the DBAD debate.

Why don't you remove the insults, and post it on Dean Radin's blog? Will be interesting to see the response.
__________________
Doubt is not a pleasant condition, but certainty is absurd - Voltaire
Arouet is offline   Quote this post in a PM   Nominate this post for this month's language award Copy a direct link to this post Reply With Quote Back to Top
Old 21st December 2010, 10:31 PM   #373
Robin
Penultimate Amazing
 
Robin's Avatar
 
Join Date: Apr 2004
Posts: 14,718
Originally Posted by fls View Post
Transformation of the data is done to bring the data more in line with the assumptions underlying a specific analysis - normal distribution, equality of variability, or linearity of a realtionship. In this case (T-test), the assumption is that the data follows a normal distribution with similar variability. If the data is skewed, then estimates or approximations as to the probability of specific events will be off. Log or inverse transformations are typically helpful to return positively skewed data to normality, but the exact choice may be a matter of trial and error.

The results of the analysis are transformed back for the purposes of reporting them in units which make sense.

Linda
In fact it appears to be a very robust test indeed.

I had been somewhat skeptical that a 15 ms difference - a little more than a hundredth of a second - in averages between two sets of response time data could be reliably classified as significant.

But I am having to rethink that.
__________________
We all hate poverty, war, and injustice
Unlike the rest of you squares.

Tom Lehrer - Folk Song Army

Last edited by Robin; 21st December 2010 at 10:40 PM.
Robin is online now   Quote this post in a PM   Nominate this post for this month's language award Copy a direct link to this post Reply With Quote Back to Top
Old 21st December 2010, 10:42 PM   #374
Robin
Penultimate Amazing
 
Robin's Avatar
 
Join Date: Apr 2004
Posts: 14,718
I would have to agree that "scientific misconduct" is not an appropriate appellation for these issues.
__________________
We all hate poverty, war, and injustice
Unlike the rest of you squares.

Tom Lehrer - Folk Song Army
Robin is online now   Quote this post in a PM   Nominate this post for this month's language award Copy a direct link to this post Reply With Quote Back to Top
Old 22nd December 2010, 04:32 AM   #375
GnaGnaMan
Graduate Poster
 
Join Date: Sep 2008
Posts: 1,707
Originally Posted by Rramjet View Post
I cannot let this past without comment. You should consult:
Thank you but a discussion of the pros and cons of NHST would be off-topic.
GnaGnaMan is offline   Quote this post in a PM   Nominate this post for this month's language award Copy a direct link to this post Reply With Quote Back to Top
Old 22nd December 2010, 04:39 AM   #376
GnaGnaMan
Graduate Poster
 
Join Date: Sep 2008
Posts: 1,707
Originally Posted by wielange View Post
GnaGnaMan, scientific misconduct is quite an accusation, even if, or especially if, done anonymously on this board. Bem seems to be quite ready to reply to emails, and if you really believed that Bem, or anyone else, was committing outright scientific fraud, you might want to contact these people directly before publicly claiming that they indeed committed fraud. Thus is the usual procedure, and it is very basic: check first, accuse later.
I thought hard about what I should do. I considered contacting the relevant people but then decided that I should check first and accuse later and therefore posted this here to have people check it.

Quote:
I cannot claim that I am entirely convinced by Bem's work, but I don't find it very encouraging if you, or, for that matter, Alcock, just shoot blindly at these findings in the hope that one shot might hit. You are trying to convince ... whom? Yourself? Others? I find Bem's proposal to attempt replications much more rational than any of these armchair attempts to "undo" the findings. Frankly, it reminds me of creationist apologetics, and I find it scary that skepticism would have come to that.
Bravo. Bravo. A splendid demonstration of a reasonable response.
Ignore any substantive issues and simply launch insults. I wish I had thought of that. It would have save me a lot of time.
..nice projecting.

Quote:
Besides, your claim that Savva, Child, and Smith "unambigously" failed to find any evidence suggestive of what looks like precognitive habituation does not square well with what the content of their papers; have a check first. Hint: read the whole papers, including the 2005 one, not just the abstract. The discussion session of the 2005 paper particularily.
Generally, you may not like all of this, but this itself is not a free ticket to twist and turn things.
You think I would post the URLs if I had not read them?
How about you do some actual work instead of just insulting me.
GnaGnaMan is offline   Quote this post in a PM   Nominate this post for this month's language award Copy a direct link to this post Reply With Quote Back to Top
Old 22nd December 2010, 04:47 AM   #377
GnaGnaMan
Graduate Poster
 
Join Date: Sep 2008
Posts: 1,707
Originally Posted by Robin View Post
I would have to agree that "scientific misconduct" is not an appropriate appellation for these issues.
I considered calling it dishonesty but decided that misconduct was more neutral. Oh well....

Originally Posted by Arouet View Post
I appreciate reading critique - whether positive or negative - of these studies, though I get lost in much of the science lingo, but I agree with the poster above: it serves little to throw out such inflammatory (and perhaps unfair) language in an otherwise seemingly well-reasoned critique. All it will do is take attention away from the points being raised and result in wasted digression into the DBAD debate.
I posted these issues a few pages back without discussing the implications. They were almost completely ignored. In fact noone who now jerks their knees commented at the time.

Quote:
Why don't you remove the insults, and post it on Dean Radin's blog? Will be interesting to see the response.
What do you see as insults and why?
Arguments please.
GnaGnaMan is offline   Quote this post in a PM   Nominate this post for this month's language award Copy a direct link to this post Reply With Quote Back to Top
Old 22nd December 2010, 06:00 AM   #378
wielange
New Blood
 
Join Date: Nov 2010
Posts: 9
GnaGnaMan,
I didn't mean to insult you. That your mail reminds me of much one can read in apologetics is not an insult, though, but a personal impression I cannot help having. I could have called it by the mechanism, dissonance resolution, and then it perhaps would have been less insulting (therefore my question whom you are trying to convince).

Still, I am amazed how easily you get insulted vis-a-vis the fact that you are publicly accusing Bem of scientific misconduct (or dishonesty, or fraud). Again, the usual procedure is to ask Bem first about what you believe to be irregularities. Do it. It would be interesting to see what he has to say.

Just for the record (since you mention projection) - I am undecided what Bem's findings mean, how they came about and to which degree they provide evidence suggestive of an anomalous effect. I believe that replications are the only way to resolve this, and since these take a while, some patience is needed.
My impression was that you didn't want to wait; and that you were really overdoing it with your misconduct charge. It's shrill, and your evidence is, imho, not overly convincing. That's why I wondered whether you really read the Savva papers.

Last edited by wielange; 22nd December 2010 at 06:32 AM.
wielange is offline   Quote this post in a PM   Nominate this post for this month's language award Copy a direct link to this post Reply With Quote Back to Top
Old 22nd December 2010, 06:50 AM   #379
EHocking
Philosopher
 
EHocking's Avatar
 
Join Date: Apr 2004
Posts: 8,592
Originally Posted by Rwalsh View Post
I beleive we have common telecommunications skills that most people dont think much of, but infact everyone has some form of telecommunication.
Not hugely surprising since Alexander Graham Bell invented the telephone 134 years ago.
__________________
Vote like you’re poor.

A closed mouth gathers no feet"
"Ignorance is a renewable resource" P.J.O'Rourke
"It's all god's handiwork, there's little quality control applied", Fox26 reporter on Texas granite
EHocking is offline   Quote this post in a PM   Nominate this post for this month's language award Copy a direct link to this post Reply With Quote Back to Top
Old 22nd December 2010, 06:56 AM   #380
fls
Penultimate Amazing
 
Join Date: Jan 2005
Posts: 10,226
GnaGnaMan, the claim of misrepresenting others' work is over-stated. It is perfectly reasonable to draw different conclusions from the original author so long as the data support this conclusion. An invested researcher may be likely to over-state what conclusions are drawn, while other readers draw more conservative conclusions. So merely coming to a different conclusion is not misrepresentation.

And while it is a somewhat questionable practice to round off the p-value to 0.05, a difference that small cannot alter the conclusions which might reasonably be drawn. The much bigger problem in this case is that the p-value of 0.051 is one-tailed, when Bem is clearly operating under two-tailed assumptions. When reading the Savva papers, it becomes clear that between precognitive habituation/aversion/boredom, a difference in any direction on any kind of stimuli is fitted with a post-hoc explanation of why it supports the hypothesis (not so much from Savva et. al., but from their discussion of Bem). It is this which makes Bem's claim of a successful replication weak, since two-tailed testing would render it non-significant. But even then, it doesn't quite rise to the level of misconduct.

Also, Bem may have decided that the second set of experiments did not represent a replication attempt because they were supraliminal, instead of subliminal.

I think you should query Bem about the issue of the relationship between those experiments he has previously reported upon and these reports. As you say, the accounts are contradictory and I cannot see how it can be anything other than a gross misrepresentation. So it is important to see how Bem reconciles this. I have to assume that we are missing important information (which doesn't get him off the hook as it also represents misrepresentation) because he couldn't have expected to get away with this when both accounts are in the public record.

One of the other issues I have is that he seems to have misrepresented the form of the normal experiments he is time-reversing. I'm not familiar with these experiments, but I've been doing some background reading plus reading about some of the original experiments he refers to in his paper. The habituation/mere exposure effect he refers to uses non-arousing stimuli, not the arousing (positive and negative) stimuli he uses. And he fails to find the effect which is seen in the original experiments as his non-arousing trials fail to show an effect. I realize that he talks about this to some degree, but I think he downplays the significance. Other experiments, such as the facilitated recall experiments, used control groups in the original experiments, which he fails to include. And we've already talked about his choice of different analysis techniques than those used by the original experimenters. He makes a big point of using established procedures, but I haven't yet found an original experiment which actually corresponds to what he did in a time-reversed manner. I still have a lot of looking to do though.

Linda

Last edited by fls; 22nd December 2010 at 06:57 AM.
fls is offline   Quote this post in a PM   Nominate this post for this month's language award Copy a direct link to this post Reply With Quote Back to Top
Old 22nd December 2010, 08:48 AM   #381
GnaGnaMan
Graduate Poster
 
Join Date: Sep 2008
Posts: 1,707
Originally Posted by fls View Post
GnaGnaMan, the claim of misrepresenting others' work is over-stated. It is perfectly reasonable to draw different conclusions from the original author so long as the data support this conclusion. An invested researcher may be likely to over-state what conclusions are drawn, while other readers draw more conservative conclusions. So merely coming to a different conclusion is not misrepresentation.
I agree that it is a minor case of misrepresentation and I would not have thought it relevant if it was in isolation. However, I do not think it is a legitimate reinterpretation.
Savva et al performed 3 experiments. Bem cites only 1 and fudges the numbers on that one. I don't see that as reinterpretation.

I would agree that this is minor. It can be said to be just putting a little spin on rather than really being dishonest. However, since there are other instances where the line, IMO, is clearly crossed it seemed wrong to omit this point. It seems to me that one should produce a complete account to the best of one's knowledge.

Originally Posted by fls View Post
I think you should query Bem about the issue of the relationship between those experiments he has previously reported upon and these reports. As you say, the accounts are contradictory and I cannot see how it can be anything other than a gross misrepresentation. So it is important to see how Bem reconciles this. I have to assume that we are missing important information (which doesn't get him off the hook as it also represents misrepresentation) because he couldn't have expected to get away with this when both accounts are in the public record.
I have sent a (neutrally worded) request to Bem over a week ago. I do not regard it as suspicious that he has not replied but given that he has apparently not replied to requests for data made by other posters here much sooner I don't have much hope for a reply anytime soon.
What pushed me toward rewriting these issues and now spelling out the implications was really the reply to Alcock. I don't see how his denial there can be innocently interpreted.
GnaGnaMan is offline   Quote this post in a PM   Nominate this post for this month's language award Copy a direct link to this post Reply With Quote Back to Top
Old 22nd December 2010, 09:54 AM   #382
Arouet
Thinker
 
Arouet's Avatar
 
Join Date: Apr 2010
Location: Toronto
Posts: 173
Originally Posted by GnaGnaMan View Post
What do you see as insults and why?
Arguments please.
Sorry, I wrote too quickly: unnecessarily inflamatory might be more accurate. I think it serves as a distraction at this stage. If the critiques pan out, and it does look like misconduct, that can come out over the discussion, as consensus develops. But just throwing out that accusation I believe will only serve as distraction, and worse, might be as unfounded attack on a researcher's character.

I would be interested in their response, which is why I suggested posting it on Radin's blog. He tends to reply to posts, and it is likely to garner some attention.
__________________
Doubt is not a pleasant condition, but certainty is absurd - Voltaire
Arouet is offline   Quote this post in a PM   Nominate this post for this month's language award Copy a direct link to this post Reply With Quote Back to Top
Old 22nd December 2010, 01:29 PM   #383
Rwalsh
Banned
 
Join Date: Jan 2009
Posts: 305
Originally Posted by Robin View Post
I don't believe that anyone has any of those things.

But it would be cool to be proved wrong.

The one I would want is sensing where I lost something.

In fact right now I wish I could sense where my house keys are.
My dog is telecommunicative, Its hard to prove this statement.

Lets see if I can try to explain.

Say you stop at a 4 way Intersection, Just stop signs. A couple drivers come driving up and stop at the same time. Some drivers become suddenly confused at times. Perhaps the beginners. Even in Night time conditions, its easy to see a in-expirenced driver. Often enough they hesitate. Your swearing in your *********** brain to go- and they acknowledge then of there of there mistakes. Sometimes these are short body language too.

Personally Telecommunication is on a sub level of the conciousness where most people dont fully understand to see there foresight.

Telecommunication involves no speakin to be heard.

I remember When I was young boy, and we owned a dog. He could sense A thunderstorm comming when the sky was clear. He would always run and hide when he sensed it coming.
Rwalsh is offline   Quote this post in a PM   Nominate this post for this month's language award Copy a direct link to this post Reply With Quote Back to Top
Old 22nd December 2010, 01:40 PM   #384
Rwalsh
Banned
 
Join Date: Jan 2009
Posts: 305
Its probably a combination of a bunch of things as well that the brain calculates. To me the brain is like a calculator processing information, gatherin information., Inputs and outputs of information. Coming and going. Doing all sorts of things.

Its possible with a bunch of things. The
Inclind Notion
Sensory vision.
telepathy-big or small
your own personal feelings which are like Sensory Visions as well.
Inclind notion which produces Telepathy communication

Im sure we all could be very powerful with our minds if we, Gather enough proof of its existance and finding the scientific ways to produce them. A faculty of such.

Like men who stare at goats

http://www.youtube.com/watch?v=SreufFevUSw
Rwalsh is offline   Quote this post in a PM   Nominate this post for this month's language award Copy a direct link to this post Reply With Quote Back to Top
Old 22nd December 2010, 01:49 PM   #385
Akhenaten
Heretic Pharaoh
 
Akhenaten's Avatar
 
Join Date: Nov 2007
Location: Pi-Broadford, Australia
Posts: 29,692
Originally Posted by Rwalsh View Post
My dog is telecommunicative, Its hard to prove this statement.


Let me guess. You had your proof all written out but then he ate it.


Originally Posted by Rwalsh View Post
Lets see if I can try to explain.

<snip abysmal failure to explain anything at all>


Perhaps one should master normal, written communication before moving on to more complicated methods of broadcasting information.


Originally Posted by Rwalsh View Post
Personally Telecommunication is on a sub level of the conciousness where most people dont fully understand to see there foresight.


You really should use a dictionary and look up 'telecommunications' because it doesn't mean what you appear to think it means.


Originally Posted by Rwalsh View Post
Telecommunication involves no speakin to be heard.


So the purpose of television, radio, telephones, etcetera is to broadcast silence, is it? That's news.


Originally Posted by Rwalsh View Post
I remember When I was young boy, and we owned a dog. He could sense A thunderstorm comming when the sky was clear. He would always run and hide when he sensed it coming.


Did he actually tell you this or did he telecommunicate it to you?
__________________


Life is mostly Froth and Bubble - Adam Lindsay Gordon
Akhenaten is offline   Quote this post in a PM   Nominate this post for this month's language award Copy a direct link to this post Reply With Quote Back to Top
Old 22nd December 2010, 02:01 PM   #386
Robin
Penultimate Amazing
 
Robin's Avatar
 
Join Date: Apr 2004
Posts: 14,718
Originally Posted by Rwalsh View Post
I remember When I was young boy, and we owned a dog. He could sense A thunderstorm comming when the sky was clear. He would always run and hide when he sensed it coming.
Could be that he heard the thunder before any human member of the family could.
__________________
We all hate poverty, war, and injustice
Unlike the rest of you squares.

Tom Lehrer - Folk Song Army
Robin is online now   Quote this post in a PM   Nominate this post for this month's language award Copy a direct link to this post Reply With Quote Back to Top
Old 22nd December 2010, 02:04 PM   #387
Robin
Penultimate Amazing
 
Robin's Avatar
 
Join Date: Apr 2004
Posts: 14,718
Originally Posted by Rwalsh View Post
Lets see if I can try to explain.

Say you stop at a 4 way Intersection, Just stop signs. A couple drivers come driving up and stop at the same time. Some drivers become suddenly confused at times. Perhaps the beginners. Even in Night time conditions, its easy to see a in-expirenced driver. Often enough they hesitate. Your swearing in your *********** brain to go- and they acknowledge then of there of there mistakes. Sometimes these are short body language too.

Personally Telecommunication is on a sub level of the conciousness where most people dont fully understand to see there foresight.

Telecommunication involves no speakin to be heard.
But this does not necessarily involve any anomalous event.

I would agree that much of human communication happens at a non verbal level.
__________________
We all hate poverty, war, and injustice
Unlike the rest of you squares.

Tom Lehrer - Folk Song Army
Robin is online now   Quote this post in a PM   Nominate this post for this month's language award Copy a direct link to this post Reply With Quote Back to Top
Old 22nd December 2010, 02:05 PM   #388
Akhenaten
Heretic Pharaoh
 
Akhenaten's Avatar
 
Join Date: Nov 2007
Location: Pi-Broadford, Australia
Posts: 29,692
Originally Posted by Rwalsh View Post
Its probably a combination of a bunch of things as well that the brain calculates. To me the brain is like a calculator processing information, gatherin information., Inputs and outputs of information. Coming and going. Doing all sorts of things.


GIGO


Originally Posted by Rwalsh View Post
Its possible with a bunch of things. The
Inclind Notion
Sensory vision.
telepathy-big or small
your own personal feelings which are like Sensory Visions as well.
Inclind notion which produces Telepathy communication


"Inclind notion" ???

Just on the off-chance that "inclined notion" actually meant something I did a Google search for the phrase and unbelievably (?) discovered that it doesn't, although it's been used before in this very forum, in a thread about animal telepathy by a poster by the name of . . . oh . . . never mind.


Originally Posted by Rwalsh View Post
Im sure we all could be very powerful with our minds if we, Gather enough proof of its existance and finding the scientific ways to produce them. A faculty of such.


You could call it the Gathering.


Originally Posted by Rwalsh View Post
Like men who stare at goats

http://www.youtube.com/watch?v=SreufFevUSw


You do realise that film is a comedy, and is only very loosely based on the 2004 book of the same name, don't you?

I probably have some bad news for you about Skippy, Flipper and Bambi as well.
__________________


Life is mostly Froth and Bubble - Adam Lindsay Gordon
Akhenaten is offline   Quote this post in a PM   Nominate this post for this month's language award Copy a direct link to this post Reply With Quote Back to Top
Old 22nd December 2010, 02:06 PM   #389
Rwalsh
Banned
 
Join Date: Jan 2009
Posts: 305
Then how do you explain

men who stare at goats

http://www.youtube.com/watch?v=SreufFevUSw
Rwalsh is offline   Quote this post in a PM   Nominate this post for this month's language award Copy a direct link to this post Reply With Quote Back to Top
Old 22nd December 2010, 02:18 PM   #390
Akhenaten
Heretic Pharaoh
 
Akhenaten's Avatar
 
Join Date: Nov 2007
Location: Pi-Broadford, Australia
Posts: 29,692
Originally Posted by Rwalsh View Post
Then how do you explain

men who stare at goats


It's a documentary cast in the same mold as True Lies or Last Action Hero. I'd recommend it highly to anyone wanting to learn more about the secrets of Inclind Notion.
__________________


Life is mostly Froth and Bubble - Adam Lindsay Gordon
Akhenaten is offline   Quote this post in a PM   Nominate this post for this month's language award Copy a direct link to this post Reply With Quote Back to Top
Old 22nd December 2010, 03:05 PM   #391
Jeff Corey
New York Skeptic
 
Jeff Corey's Avatar
 
Join Date: Aug 2001
Posts: 13,714
Originally Posted by Rwalsh View Post
Then how do you explain

men who stare at goats

http://www.youtube.com/watch?v=SreufFevUSw
I'll bet you tried that "running through walls" stunt a number of times. Do you remember how many?
Jeff Corey is offline   Quote this post in a PM   Nominate this post for this month's language award Copy a direct link to this post Reply With Quote Back to Top
Old 22nd December 2010, 03:37 PM   #392
Robin
Penultimate Amazing
 
Robin's Avatar
 
Join Date: Apr 2004
Posts: 14,718
On further investigation I find I am wrong about experiment 9 - the p value of 0.002 claimed is accurate.

There is a 0.02 probability a deviation of 4.41 or greater from the mean, but on a 1 tailed t test p is 0.002

This would even be significant on a two-tailed analysis (experiment 8 would not). There appear to be a number of results that fall into this category.

This brings me back to Paul's earlier question:
Originally Posted by Paul C. Anagnostopoulos View Post
Does anyone want to hazard a guess as to what's happening?
I can think of only three possibilities:

1. Carelessness
2. Fraud
3. Precognition

Any comments?
__________________
We all hate poverty, war, and injustice
Unlike the rest of you squares.

Tom Lehrer - Folk Song Army
Robin is online now   Quote this post in a PM   Nominate this post for this month's language award Copy a direct link to this post Reply With Quote Back to Top
Old 23rd December 2010, 12:45 AM   #393
Kid Eager
Philosopher
 
Kid Eager's Avatar
 
Join Date: Nov 2010
Posts: 7,296
Originally Posted by Rwalsh View Post
Then how do you explain

men who stare at goats

http://www.youtube.com/watch?v=SreufFevUSw
Sure. Do you want me to explain the movie plot, or to explain strange men?
__________________
What do Narwhals, Magnets and Apollo 13 have in common? Think about it....
Kid Eager is offline   Quote this post in a PM   Nominate this post for this month's language award Copy a direct link to this post Reply With Quote Back to Top
Old 23rd December 2010, 06:09 AM   #394
fls
Penultimate Amazing
 
Join Date: Jan 2005
Posts: 10,226
Originally Posted by Robin View Post
I can think of only three possibilities:

1. Carelessness
2. Fraud
3. Precognition

Any comments?
Considering that we already know that 1 and 2 are present (for a loose consideration of "fraud"), I'm not sure that we have to pretend that this is mysterious. One example of carelessness was already confirmed (non-blinded assessments) and there exist many opportunities for others which haven't been investigated. And it is clear that the p-values, which theoretically represent what might be expected due to chance when given one opportunity, really represent the taking of multiple opportunities to find these small differences.

But even if these differences are confirmed and are robust, it only allows you to say that they are unexpected due to chance. It doesn't tell you whether some other explanation, such as precognition, is more likely, since the experiments are not set up to tell us the probability that precognition is present.

Linda
fls is offline   Quote this post in a PM   Nominate this post for this month's language award Copy a direct link to this post Reply With Quote Back to Top
Old 23rd December 2010, 07:06 AM   #395
blobru
Philosopher
 
blobru's Avatar
 
Join Date: May 2007
Posts: 6,900
Originally Posted by fls View Post
Considering that we already know that 1 and 2 are present (for a loose consideration of "fraud"), I'm not sure that we have to pretend that this is mysterious. One example of carelessness was already confirmed (non-blinded assessments)...

That's my suspicion, too. With the online replication continuing to stagger about zero (latest tally: -0.47% after 568 participants -- even after I clocked +59.03%... though as a "low stimulus seeker"... sorry, Bem), one has to wonder why the statistical skew in the lab with non-blinded assessments, but, afawct, none online with blind [mechanical] assessments. It could be that unlike some other 'psi' effects this is one that only shows up in the lab; however, experimenter bias seems a likelier culprit.
__________________
"Say to them, 'I am Nobody!'" -- Ulysses to the Cyclops

"Never mind. I can't read." -- Hokulele to the Easter Bunny
blobru is offline   Quote this post in a PM   Nominate this post for this month's language award Copy a direct link to this post Reply With Quote Back to Top
Old 23rd December 2010, 08:23 AM   #396
GnaGnaMan
Graduate Poster
 
Join Date: Sep 2008
Posts: 1,707
Originally Posted by Robin View Post
On further investigation I find I am wrong about experiment 9 - the p value of 0.002 claimed is accurate.

There is a 0.02 probability a deviation of 4.41 or greater from the mean, but on a 1 tailed t test p is 0.002
0.02 using what test?

Quote:
I can think of only three possibilities:

1. Carelessness
2. Fraud
3. Precognition

Any comments?
I think there is a lot of grey area between 1 and 2.
GnaGnaMan is offline   Quote this post in a PM   Nominate this post for this month's language award Copy a direct link to this post Reply With Quote Back to Top
Old 23rd December 2010, 08:26 AM   #397
GnaGnaMan
Graduate Poster
 
Join Date: Sep 2008
Posts: 1,707
Originally Posted by fls View Post
One example of carelessness was already confirmed (non-blinded assessments) [...]
Is that really that bad? The assessment was after all not part of the experiment proper, so to say. It seems more analoguous to transcribing data by hand. I don't think that's usually done blinded, or is it?
GnaGnaMan is offline   Quote this post in a PM   Nominate this post for this month's language award Copy a direct link to this post Reply With Quote Back to Top
Old 23rd December 2010, 10:40 AM   #398
Beth
Philosopher
 
Beth's Avatar
 
Join Date: Dec 2004
Posts: 5,598
Originally Posted by GnaGnaMan View Post
Is that really that bad? The assessment was after all not part of the experiment proper, so to say. It seems more analoguous to transcribing data by hand. I don't think that's usually done blinded, or is it?
In this case, I think it could be. My understanding of the study was that the students doing the transcribing knew which words were practiced and which weren't. It would be easy to, completely unconsciously, favor the practiced words in scoring. However, it's a flaw that can be corrected by having the original data transcribed by someone (or preferably two) who is unaware of which words were practiced. That would provide a 'clean' set of data to be analyzed and the difference between them would be the bias that was introduced by allowing that flaw to creep in.
__________________
Beth
"You are not the stuff of which you are made."
Richard Dawkins, July 2005, 10:45

http://www.ted.com/talks/richard_daw..._universe.html
Beth is offline   Quote this post in a PM   Nominate this post for this month's language award Copy a direct link to this post Reply With Quote Back to Top
Old 23rd December 2010, 01:54 PM   #399
Robin
Penultimate Amazing
 
Robin's Avatar
 
Join Date: Apr 2004
Posts: 14,718
Originally Posted by GnaGnaMan View Post
0.02 using what test?
A two sample T test using the assumption of unequal sample size and variance.
__________________
We all hate poverty, war, and injustice
Unlike the rest of you squares.

Tom Lehrer - Folk Song Army
Robin is online now   Quote this post in a PM   Nominate this post for this month's language award Copy a direct link to this post Reply With Quote Back to Top
Old 24th December 2010, 05:46 AM   #400
fls
Penultimate Amazing
 
Join Date: Jan 2005
Posts: 10,226
Originally Posted by GnaGnaMan View Post
I agree that it is a minor case of misrepresentation and I would not have thought it relevant if it was in isolation. However, I do not think it is a legitimate reinterpretation.
Savva et al performed 3 experiments. Bem cites only 1 and fudges the numbers on that one. I don't see that as reinterpretation.

I would agree that this is minor. It can be said to be just putting a little spin on rather than really being dishonest. However, since there are other instances where the line, IMO, is clearly crossed it seemed wrong to omit this point. It seems to me that one should produce a complete account to the best of one's knowledge.
I agree that it goes to illustrations of whether he is trustworthy in his accounts. And that it's adding up to a much less than ideal picture.

Quote:
I have sent a (neutrally worded) request to Bem over a week ago. I do not regard it as suspicious that he has not replied but given that he has apparently not replied to requests for data made by other posters here much sooner I don't have much hope for a reply anytime soon.
What pushed me toward rewriting these issues and now spelling out the implications was really the reply to Alcock. I don't see how his denial there can be innocently interpreted.
Yeah, I'm having trouble with that as well.

Linda
fls is offline   Quote this post in a PM   Nominate this post for this month's language award Copy a direct link to this post Reply With Quote Back to Top
Reply

International Skeptics Forum » General Topics » General Skepticism and The Paranormal

Thread Tools

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Forum Jump


All times are GMT -7. The time now is 04:11 PM.
Powered by vBulletin. Copyright ©2000 - 2023, Jelsoft Enterprises Ltd.

This forum began as part of the James Randi Education Foundation (JREF). However, the forum now exists as
an independent entity with no affiliation with or endorsement by the JREF, including the section in reference to "JREF" topics.

Disclaimer: Messages posted in the Forum are solely the opinion of their authors.