|
Welcome to the International Skeptics Forum, where we discuss skepticism, critical thinking, the paranormal and science in a friendly but lively way. You are currently viewing the forum as a guest, which means you are missing out on discussing matters that are of interest to you. Please consider registering so you can gain full use of the forum features and interact with other Members. Registration is simple, fast and free! Click here to register today. |
![]() |
#1 |
Graduate Poster
Join Date: Jul 2009
Posts: 1,259
|
Near Death Experiences Explained... Debunked
(laces up running shoes to get a headstart)
Now, before everyone starts chasing me out of town waving virtual torches, y’all should know that what follows is not an argument that NDE’s represent proof for God and angels and the Second Coming of Jesus bringing us all back to heaven. But I can’t stand what the popular press does to scientific research, and to be perfectly honest, I was not impressed by this research anyway. Here’s why. Please read all the way to the end before commenting. I did a lot of original work on this, ALL BY MYSELF. ![]() Here's the actual real-live study yapped about in National Geographic: Right here. On careful analysis, the results are so far from anything that was reported in National Geographic (or anywhere else) that it’s simply embarrassing. The differences are so significant. But you really do have to do some digging. The claim was that:
Quote:
Quote:
STAT-TESTS-2SAMPTTEST. More about why this test was chosen later. Now enter in your data. Your p value is .049469,but this is NOT WHAT THE RESEARCHER SAY. They claim that the p value is .041. It is not. Get out your stats calculator and enter the numbers yourself. The alpha value they chose is the very common one of .05. This means that they've barely, BARELY inched over into showing an effect for that pCo2. This is just barely statistically significant, and it has no practical significance by any definition. If you round up at 2 decimal places rather than 3 (which you're not supposed to do, of course, but let's just say for the sake of argument...) you cannot reject the null hypothesis. The numbers get more robust for the petCo2 levels, because the p value is .0017. But there's a tremendous problem here. The multivariate analysis found that:
Quote:
Now, a stats expert I am not. But think about this: they said that they first used univariate analysis, considering each variable separately. By the time they got to the information in Table 2, what else could you use for those particular results besides an independent samples t-test for 2 means? What they actually said in the study was:
Quote:
The only obvious way to get this number that I can see would be to artificially conflate the sample sizes. For instance, if you say that you had 100 subjects in each group (n1 and n2 each = 100,) then you’ll end up with p < .0001. The effect would be very robust if you could just get a large enough sample size. But that is just about the worst methodology that anybody could ever even imagine coming up with. The sample only consisted of 52 people, 11 (yes, eleven) of which had NDE’s. This was from an original pool of 400 who had cardiac arrest, and only 76 survived! Ack. Seven of the 11 NDE’ers were atheists. I’m not sure if this was unusual or not, though, because I have no idea what the religious makeup of Slovenia is.) I don’t know. I’m really, really confused about this point, because I basically just do not see how they came up with a figure of .041 based on their data. Even so, a P-value of .041 is not the least bit impressive compared to an alpha of .05 either. Overall, higher petCo2 was not an independent predictor of NDE's, and higher PCo2 was but just barely, barely managed to cross over into even statistical significance. The only conclusion to really come to is that this study simply does not mean what it has been ballyhooed to mean. When all is said and done, of course, there’s also a fatal flaw that totally and completely invalidates the kind of conclusions that have been drawn from this study by the popular press, and it's that in order to draw any real conclusions from these stats methods, the sample sizes absolutely must be at least n = thirty for each group. The NDE group was n = 11. The researchers said clearly that this was a prospective observational study. That was the responsible thing to say because of its limitations, but particularly because of the size problem. The final lesson to take away from all of this, I think, is to just not believe the version you read in the popular press. Complex neurobiological phenomena do not have simplistic explanations. I’ll be sending all of this information to National Geographic, and I’ll let them know what I think of their fact-checking quality. All of this took me half an hour, and really, most of the people on this board could have done it too. |
__________________
Satan, aka Precious the Cat-- the Comic! Come and read at Stripgenerator! |
|
![]() ![]() |
![]() ![]() ![]() ![]() |
![]() |
#2 |
beer-swilling semiliterate
Join Date: Jul 2006
Location: Connecticut, or King Arthur's Court. Hard to tell sometimes.
Posts: 25,458
|
Impressive work, Maia. I couldn't have done it if you gave me 30 months. Two virtual thumbs up.
|
__________________
A møøse ønce bit my sister |
|
![]() ![]() |
![]() ![]() ![]() ![]() |
![]() |
#3 |
Philosopher
Join Date: Nov 2007
Location: Perth, Western Australia
Posts: 6,057
|
now, where did I put my virtual torch?
|
__________________
"Here we go again.... semantic and syntactic chicanery and sophistic sleight of tongue and pen.... the bedazzling magic of appearing to be saying something when in fact all that is happening is diverting attention from the attempts at shoving god through the trapdoor of illogic and wishful thinking." - Leumas |
|
![]() ![]() |
![]() ![]() ![]() ![]() |
![]() |
#4 |
Master Poster
Join Date: Jul 2008
Location: UK
Posts: 2,381
|
I've got mine.
![]() On the face of it, that seems impressive, Maia. However, I'm in no position to judge, not having had a science education and all that. I think there may be cause not to accept what you say at face value though, since you do have a history here of making suspect and even extraordinary claims. Take the study you enthused about in your A Rational Exploration of NDE-Related Research thread. Naturally, making dubious or mistaken claims about near death experience research in one thread isn't in itself serious cause to doubt your analysis in this one. It might be highly accurate. Still, it might be cause for anyone who actually understands what you wrote to cast an extra-critical eye over it. You said, in the other thread:
Quote:
![]()
Originally Posted by Maia
From an article called * Decoding The Mystery Of Near-Death Experiences
Quote:
Also, Beauregard used a sample size of ... Fifteen? In this thread, you criticise the study reported in National Geographic as being unlikely to be able to tell us anything significant because of its small sample size. But in this other NDE thread of yours, you're enthusing about research where the sample size isn't much different, it seems. The article I quoted goes on to say that skeptics have drawn completely opposite conclusions from the exact same study results.
Quote:
It turns out that Beauregard has written a book called The Spiritual Brain: A Neuroscientist's Case for the Existence of the Soul. Here's an excerpt:
Quote:
You then go on to state in the other thread:
Originally Posted by Maia
You then make the extraordinary claim in your next post in the thread:
Quote:
And for another thing, why haven't you taken into account the conventional wisdom that if you see relatives or peaceful beings of light in your NDE for whatever reason, you'll likely come back with a feeling of reassurance and optimism that is in fact highly likely to mitigate any trauma you might otherwise feel, and quite likely to entice one to meditate on the experience since it was so enjoyable. Also, if you would like to cause trauma survivors to have the same meditative state as the NDE experiencers in the study, and that's the reason these findings interest you so much, just why do you imagine this could ever soothe their trauma long-term, and how would you propose to get them to have the experiences? If there's no practical way of doing it other than half-killing them in the hope they have an NDE, what's the point of exploring this? Lastly, though you're of course right to be dubious of the interpretation the popular press puts on study findings, it seems you aren't cautious enough about findings documented in abstracts on places like PubMed. For instance, here's the Beauregard study on PubMed: Brain activity in near-death experiencers during a meditative state. It says nothing about the apparently small sample size, and the fact that the findings have been disputed. As I illustrated in my latest post of a couple of days ago in the What's the appeal of anti-psychiatry thread, relying too heavily on what you read in even reputable journals could possibly be tragically dangerous, or so it seems. The post makes several mentions of infamous trauma psychiatrist Colin Ross, million dollar challenge applicant and writer of 140 peer-reviewed papers, who incidentally is actually alleged to have carried out research into NDE's by deliberately nearly killing patients. Whether the allegations are true or not, I can't of course know. |
![]() ![]() |
![]() ![]() ![]() ![]() |
![]() |
#5 |
Graduate Poster
Join Date: Jul 2009
Posts: 1,259
|
Thanks jhunter!
![]() It's important to know that I'm not saying that this proves any other explanation for NDE's, because clearly it doesn't. It just means that on the basis of this evidence, this null hypothesis (that people who reported NDE's as corroborated by the Greyson scale did not actually have meaningful statistically or practically higher levels of pCo2 than those who did not report NDE's) cannot be rejected. Because only pC02 levels were supposedly independently correlated with both higher levels of reported NDE's and a higher NDE score, this means that the argument that higher Co2 levels in general cause NDE's or are solely responsible for the subjective and objective effects which correspond to NDE's cannot be supported by this study. Essentially, we need better studies. |
__________________
Satan, aka Precious the Cat-- the Comic! Come and read at Stripgenerator! |
|
![]() ![]() |
![]() ![]() ![]() ![]() |
![]() |
#6 |
beer-swilling semiliterate
Join Date: Jul 2006
Location: Connecticut, or King Arthur's Court. Hard to tell sometimes.
Posts: 25,458
|
This particular area of study has always interested me (near-death and out-of-body experiences). My feeling has always been that there is some area of the brain that, when stimulated, produces this type of experience. I don't subscribe to any paranormal explanation for the phenomenon (I know, that's no fun). Hopefully, there'll be a more definitive study at some point.
Again, excellent work ![]() |
__________________
A møøse ønce bit my sister |
|
![]() ![]() |
![]() ![]() ![]() ![]() |
![]() |
#7 |
RSL Groupie
Join Date: Jun 2007
Posts: 1,167
|
I must confess that I understood none of the OP, but thanks for your work, and for sharing it with us!
|
__________________
"If all those people knew what an insignificant, worthless, undeserving person I really am, they wouldn't ask me for advice on laundry detergent, let alone their lives and spirituality." - Sylvia Browne |
|
![]() ![]() |
![]() ![]() ![]() ![]() |
![]() |
#8 |
Chordate
Join Date: Apr 2003
Location: Cape Town. Still not mugged. Plenty of chameleons though, and stepped on a cobra.
Posts: 1,684
|
I don't think the issue of p=.041 vs .049 deserves the quibbling. The 5% p-value convention is a convention, nothing more; one false positive out of twenty versus one false positive out of twenty-one might matter for FRST funding applications, but not for hypothesis formulation. It shows a trend worth following up, with plenty of interpretational leeway in either direction.
However, I agree that the deciding factor here are the sample sizes. N=11 in that hell of a multi-variate system? This can only be purely exploratory. |
__________________
They had no god; they had no gods; they had no faith. What they appear to have had is a working metaphor. - Ursula K. Le Guin, "Always Coming Home" |
|
![]() ![]() |
![]() ![]() ![]() ![]() |
![]() |
#9 |
Banned
Join Date: Oct 2008
Posts: 7,345
|
I'm not sure what you're all worked up about. Your post reads somewhat frantic, and I'm not clear what conclusions you want me to draw.
You seem to be criticizing the 0.05 level of statistical significance, but I think it's appropriate it in this case. The correlation to CO2 levels does not seem to be an extraordinary claim. From what I read in your link, there already exists research that indicates it could be a factor:
Quote:
It's a matter of understanding statistics and how they apply to real life. Suppose we measure the friction and tackiness of a die. We find that a wet die has more friction and is more "sticky" on a wood table. We then lick one side of a die and roll it a bunch of times. We might find that our target number comes up more frequently, and the p-value is 0.049, which you call "just barely" statistically significant. I'm comfortable with saying licking the die influences the outcome of the roll. Contrast that against a study where (say) some psychic, let's call her RodneyVision, says she can make the die come up to whatever side she chooses every time, all through the sheer force of her mind. There's no plausible mechanism or evidence that she can physically influence the die under any conditions. We run the same trial and get the same p-value as in the prior test. Is that significant? No, not really, because there's no explanation for what happened except ordinary statistical variation. |
![]() ![]() |
![]() ![]() ![]() ![]() |
![]() |
#10 |
Illuminator
Join Date: Oct 2009
Posts: 3,701
|
|
![]() ![]() |
![]() ![]() ![]() ![]() |
![]() |
#11 |
Persnickety Insect
Join Date: Dec 2002
Location: Sunny Munuvia
Posts: 16,343
|
As Floyt says, the problem isn't so much the statistics as the sample size itself. It's a useful indicator to direct further research, but it's hardly conclusive.
And Maia, when you say
Quote:
|
__________________
Free blogs for skeptics... And everyone else. mee.nu What, in the Holy Name of Gzortch, are you people doing?!?!!? - TGHO |
|
![]() ![]() |
![]() ![]() ![]() ![]() |
![]() |
#12 |
Graduate Poster
Join Date: Jul 2009
Posts: 1,259
|
Now UY, really! Nobody is frantic or "all worked up." You KNOW that you don't have to descend to these kinds of comments. Just draw the conclusions that you decide to draw. This is a discussion.
Quote:
Quote:
Do you see what the central problem is here? If we pretend that the sample size is what it should have been for both groups, then it looks like there could be a very statistically significant correlation between pCo2 levels and NDE experiences. But there's not a lot of difference between that and saying, "well, if I'd only known all 6 of the Powerball numbers, then I could have won $100,000,000 on Thursday." If we start doing that, then we might just as well start arguing for licorice teapots orbiting Jupiter. This is exactly how statistics do indeed apply to real life. T-tests are not reliable when n doesn't equal 30 for both groups, and that's why. I don't know exactly what they did to come up with such different numbers (some kind of weird adjustment?) but if it wasn't a straight T-test, then in my opinion, it just wasn't honest. Running a straight comparision T-test when both n's don't equal 30 does show why conclusions can't honestly be drawn.
Quote:
![]() |
__________________
Satan, aka Precious the Cat-- the Comic! Come and read at Stripgenerator! |
|
![]() ![]() |
![]() ![]() ![]() ![]() |
![]() |
#13 |
Graduate Poster
Join Date: Jul 2009
Posts: 1,259
|
One more thing and then off to work:
With any rolling-the-dice experiment, another major difference is that we wouldn't do a T-test, we'd do a Z-test, because we already know the standard deviation of the entire population instead of the population of the sample. We know that certain rolls of the dice will come up a certain number of times, and that this will not change. With pCo2 levels, all we know is the standard deviation of the sample. So with the Z-test, our p value is going to be lower to begin with. We can be a whole lot more confident about how likely or unlikely an event is to occur. Because of that, I don't think the dice analogy is the most accurate one to use. If we could run the pCo2 levels test knowing what our population SD's were (which we clearly can't), for instance, we'd get a p value of .037, even with n=11 for one of the groups (which is what we have.) But we can't play games like that. We can speculate about what would happen if certain facts were known or if our n values were different, but we can only come to conclusions based on what actually exists and according to certain rules. For another example, I can't come to the conclusion that higher pCo2 levels are not associated with more NDE's or with a higher NDE score on the Greyson scale, only to the conclusion that based on this study, the null hypothesis (that they are not) cannot be rejected. |
__________________
Satan, aka Precious the Cat-- the Comic! Come and read at Stripgenerator! |
|
![]() ![]() |
![]() ![]() ![]() ![]() |
![]() |
#14 |
Banned
Join Date: Oct 2008
Posts: 7,345
|
I'm telling you how I read it, and I read it twice. To me it read like somebody breathless from an exciting event trying to relay a story in halting sentences assuming the other person knows what's going on. I'm telling you that I had a hard time following your points and due to the less than analytical way it was presented, question the numbers you put forth.
Quote:
Quote:
Quote:
|
![]() ![]() |
![]() ![]() ![]() ![]() |
![]() |
#15 |
Penultimate Amazing
Join Date: Jan 2005
Posts: 10,226
|
I would drop the criticism about the p-value. It's merely a convention anyway with no intrinsic meaning. And it gives the false impression that a 'significant' p-value makes a statement about whether a hypothesis might be true (a fallacy we should be trying to dispel).
Linda |
![]() ![]() |
![]() ![]() ![]() ![]() |
![]() |
#16 |
Graduate Poster
Join Date: Mar 2008
Posts: 1,586
|
I cannot contribute anything original to this discussion, but I just bumped into this analysis while going through my usual 'favourites':
http://www.nhs.uk/news/2010/04April/...s-and-CO2.aspx Maybe some of you will find it interesting, I really like the site: http://www.nhs.uk/News/Pages/NewsIndex.aspx |
__________________
Richard Dawkins: "We are all atheists about most of the gods that humanity has ever believed in. Some of us just go one god further." Pixie of key: "HOW IS YOU NOT UNDERSTANDING WHAT I AM GIVING LECTURES ON A PROBLEM." |
|
![]() ![]() |
![]() ![]() ![]() ![]() |
![]() |
#17 |
Graduate Poster
Join Date: Jul 2009
Posts: 1,259
|
The difference here, I think, is that anyone can flip a coin 600 times rather than 6. If someone does flip a coin 6 times rather than 600 and gets an inaccurate p-value, then it would be pretty bizarre to not flip it the additional 594 times. There's no way to do this with the extremely small sample of the population they had. We don't know what was different about the 374 people in the original population of 426 they started with. We do know what's different about all the unflipped coins: nothing. We know the population standard deviation of coin flipping; there's no way to know what the population SD of the 426 was in many, many ways. A big part of working with what you have is admitting the limitations of what you ended up with, not playing numbers games or whatever it was that actually happened with that sample of 52, n=11 in one group and n=41 in the other. I will be emailing those researchers at the addresses provided, because I do want to know what went on here. The media version is so different from the original study that I think there's a particular need for any inaccuracies to be corrected.
Kuko's linked story was a better summary than the National Geographic story for sure and provided more info (thanks Kuko!), but it still gives some very misleading impressions. Popular press, eh. Again, there was no indication of the fact that the link between pCo2 levels and reported NDE's does indeed just barely manage to inch over that statistical significance line in the study. Put it this way: if someone claimed that they demonstrated that they could affect a phenomenon in some kind of "paranormal" study with a p value of .049469 and an alpha of .05 in one prospective observational study where n=11 in one of the groups, how impressed would you be? Also, there was this statement:
Quote:
Here's the statement in the study itself:
Quote:
Anyway, the media will find something else to jump on next week and will be just as inaccurate about it. ETA: The potassium levels, if anything, seem more interesting to follow up on. It's hard to imagine what that could possibly mean, though. |
__________________
Satan, aka Precious the Cat-- the Comic! Come and read at Stripgenerator! |
|
![]() ![]() |
![]() ![]() ![]() ![]() |
![]() |
#18 |
Philosopher
Join Date: Dec 2004
Posts: 5,598
|
|
__________________
Beth "You are not the stuff of which you are made." Richard Dawkins, July 2005, 10:45 http://www.ted.com/talks/richard_daw..._universe.html |
|
![]() ![]() |
![]() ![]() ![]() ![]() |
![]() |
#19 |
Penultimate Amazing
Join Date: Jan 2005
Posts: 10,226
|
|
![]() ![]() |
![]() ![]() ![]() ![]() |
![]() |
#20 |
Join Date: Apr 2010
Posts: 536
|
Does it really require a NASA calculator to determine that people are making this crap up? [/cynicism]
|
![]() ![]() |
![]() ![]() ![]() ![]() |
![]() |
#21 |
Penultimate Amazing
Join Date: Jan 2005
Posts: 10,226
|
|
![]() ![]() |
![]() ![]() ![]() ![]() |
![]() |
#22 |
Philosopher
Join Date: Dec 2004
Posts: 5,598
|
|
__________________
Beth "You are not the stuff of which you are made." Richard Dawkins, July 2005, 10:45 http://www.ted.com/talks/richard_daw..._universe.html |
|
![]() ![]() |
![]() ![]() ![]() ![]() |
![]() |
#23 |
Graduate Poster
Join Date: Jul 2009
Posts: 1,259
|
There must be some reason why they ended up with such different numbers, but I really and honestly can't come up with what it is. I have contributed work to studies and run numbers before (no guarantees as to how good any of this was.) It's just bugging me that I can't see how they came up with these numbers. In any case, as others have pointed out, the much bigger problem is that it was such a very small sample that there was no way to get anything truly accurate out of the number of subjects they had. Mostly, though, they ended up with the subjects who seemed to be the healthiest to begin with (the younger ones whose hearts seemed to be in better shape.) The main conclusion I drew from the entire thing was that out of 426 resuscitation attempts, they had 76 survivors.
![]() |
__________________
Satan, aka Precious the Cat-- the Comic! Come and read at Stripgenerator! |
|
![]() ![]() |
![]() ![]() ![]() ![]() |
![]() |
#24 |
Master Poster
Join Date: Oct 2009
Posts: 2,791
|
Sorry for a very stupid question and possibly derailing the topic a bit, but wasn't NDE debunked ages ago?
I recall something vague about an old Penn and Teller show where they talked about how NDEs could be recreated with the NASA centrefuge or something like that. |
![]() ![]() |
![]() ![]() ![]() ![]() |
![]() |
#25 |
Masterblazer
Join Date: Aug 2005
Location: Montreal, Quebec
Posts: 6,843
|
|
__________________
Almo! My Music Blog "No society ever collapsed because the poor had too much." — LeftySergeant "It may be that there is no body really at rest, to which the places and motions of others may be referred." –Issac Newton in the Principia |
|
![]() ![]() |
![]() ![]() ![]() ![]() |
![]() |
#26 |
Philosopher
Join Date: Apr 2007
Posts: 6,859
|
|
![]() ![]() |
![]() ![]() ![]() ![]() |
![]() |
#27 |
Master Poster
Join Date: Jul 2008
Location: UK
Posts: 2,381
|
Originally Posted by Maia
I will quote a bit from the article to illustrate, with a couple of the relevant bits highlighted. This quote will also illustrate that this was not, as you appear to imagine for some reason, an article that merely promoted the study. It takes an objective look, quoting one or two people who are rather skeptical of it. It leaves me wondering why you're so outraged by the article:
Quote:
Also, it's almost immediately evident from the article that the sample size was tiny. It's only a short article, so your failure to pick those things up does give an impression of a rather over-hasty approach on your part, one that wasted a fair bit of your time, since it was totally unnecessary to go to the original study and do some in-depth calculations to discern the fact that this study had very little significance indeed. You've been debunking a red herring, one of your own making. |
![]() ![]() |
![]() ![]() ![]() ![]() |
![]() |
#28 |
Critical Thinker
Join Date: Aug 2008
Posts: 318
|
|
![]() ![]() |
![]() ![]() ![]() ![]() |
![]() |
Bookmarks |
Thread Tools | |
|
|