ISF Logo   IS Forum
Forum Index Register Members List Events Mark Forums Read Help

Go Back   International Skeptics Forum » General Topics » Science, Mathematics, Medicine, and Technology
 


Welcome to the International Skeptics Forum, where we discuss skepticism, critical thinking, the paranormal and science in a friendly but lively way. You are currently viewing the forum as a guest, which means you are missing out on discussing matters that are of interest to you. Please consider registering so you can gain full use of the forum features and interact with other Members. Registration is simple, fast and free! Click here to register today.
Tags probability , statistics

Reply
Old 8th September 2012, 06:23 AM   #1
Badly Shaved Monkey
Anti-homeopathy illuminati member
 
Badly Shaved Monkey's Avatar
 
Join Date: Feb 2004
Posts: 5,362
Anecdotal Evidence

I've just posted this at SBM, but I suspect the "hard science" members of JREF might be better placed to answer my question.

I wonder whether the question should be phrased like this: there is a prior probability that a medication works, do uncontrolled anecdotal accounts of positive benefit or nil benefit have the same effect on the post-anecdote probability of the medication having a benefit?

it's just a little kite-flying exercise, so please don't be mean about it, at least not immediately.
__________________
"i'm frankly surprised homeopathy does as well as placebo" Anonymous homeopath.
"Alas, to wear the mantle of Galileo it is not enough that you be persecuted by an unkind establishment; you must also be right." (Robert Park)
Is the pen is mightier than the sword? Its effectiveness as a weapon is certainly enhanced if it is sharpened properly and poked in the eye of your opponent.
Badly Shaved Monkey is offline   Quote this post in a PM   Nominate this post for this month's language award Copy a direct link to this post Reply With Quote Back to Top
Old 8th September 2012, 06:31 AM   #2
Badly Shaved Monkey
Anti-homeopathy illuminati member
 
Badly Shaved Monkey's Avatar
 
Join Date: Feb 2004
Posts: 5,362
Is it something like this?

It's a licensed product, so I expect it to work in, say, 80% of cases. If I find positive anecdotal success it simples lies with that majority, while being added to by the various positive confounders that are always present.

No effect is supposed to occur only 20% of the time, but all the positive biases would tend to reduce the proportion of these that actually get reported as nil benefit. Perhaps for a drug that is supposed to fail only 20% of the time, you'd actually expect to be given a report of nil benefit 5% of the time. If personal experience yields a lot of nil benefit reports, I think you could begin to have reasonable doubt that the true benefit of the drug ever was 80%, even though these anecdotes are uncontrolled observations.
__________________
"i'm frankly surprised homeopathy does as well as placebo" Anonymous homeopath.
"Alas, to wear the mantle of Galileo it is not enough that you be persecuted by an unkind establishment; you must also be right." (Robert Park)
Is the pen is mightier than the sword? Its effectiveness as a weapon is certainly enhanced if it is sharpened properly and poked in the eye of your opponent.

Last edited by Badly Shaved Monkey; 8th September 2012 at 06:34 AM.
Badly Shaved Monkey is offline   Quote this post in a PM   Nominate this post for this month's language award Copy a direct link to this post Reply With Quote Back to Top
Old 8th September 2012, 06:45 AM   #3
dlorde
Philosopher
 
dlorde's Avatar
 
Join Date: Apr 2007
Posts: 6,859
If it's a commercially marketed product, I guess you have to take into account the extent to which you believe the product claims - i.e. by how much are they being hyped?
__________________
Simple probability tells us that we should expect coincidences, and simple psychology tells us that we'll remember the ones we notice...

Last edited by dlorde; 8th September 2012 at 06:47 AM.
dlorde is offline   Quote this post in a PM   Nominate this post for this month's language award Copy a direct link to this post Reply With Quote Back to Top
Old 8th September 2012, 09:02 PM   #4
Andrew Wiggin
Master Poster
 
Andrew Wiggin's Avatar
 
Join Date: Aug 2009
Posts: 2,915
Also take into account the extent to which things like the manufacturer's reputation, fancy packaging, prior anecdotes of success, and how much you paid for it influence your expectations. These are all factors in a good placebo. As for how the anecdotes influence the likelyhood of an actual double-blinded study finding an actual effect, I'd think they're only loosely related.

I actually brought something similar up a couple of years ago, in a thread about ethnobotany. My hypothesis at the time was that when ethnobotanists gather stories about supposed medicinal plants used by primitive peoples, that those supposed medicinal plants would be more likely to actually be medicinal than a random sample of plant life. I still think that there's some potential validity to that, but when I stated this strongly I was just as strongly rebutted by folks who made some intelligent arguments.
__________________
"Everyone takes the limits of his own vision for the limits of the
world." - Arthur Schopenhauer

"New and stirring things are belittled because if they are not belittled,
the humiliating question arises, 'Why then are you not taking part in
them?' " - H. G. Wells
Andrew Wiggin is offline   Quote this post in a PM   Nominate this post for this month's language award Copy a direct link to this post Reply With Quote Back to Top
Old 8th September 2012, 11:33 PM   #5
Badly Shaved Monkey
Anti-homeopathy illuminati member
 
Badly Shaved Monkey's Avatar
 
Join Date: Feb 2004
Posts: 5,362
Thanks, for the replies but really it's the specific comparison with apparently confirming apparently discomfirming anecdotes that I was hope to explore.
__________________
"i'm frankly surprised homeopathy does as well as placebo" Anonymous homeopath.
"Alas, to wear the mantle of Galileo it is not enough that you be persecuted by an unkind establishment; you must also be right." (Robert Park)
Is the pen is mightier than the sword? Its effectiveness as a weapon is certainly enhanced if it is sharpened properly and poked in the eye of your opponent.

Last edited by Badly Shaved Monkey; 9th September 2012 at 01:05 AM.
Badly Shaved Monkey is offline   Quote this post in a PM   Nominate this post for this month's language award Copy a direct link to this post Reply With Quote Back to Top
Old 9th September 2012, 01:01 AM   #6
Badly Shaved Monkey
Anti-homeopathy illuminati member
 
Badly Shaved Monkey's Avatar
 
Join Date: Feb 2004
Posts: 5,362
Thinking it through a bit further, I wonder whether my scenario is a greyer variant of the 'Black Swan' problem.

In that, the prior probability ascribed to all swans being white would have been close to 100%. Endless observations of white swans would have little net effect on that near-unity probability. Positively biased observations of big white seagulls and geese that were identified as white swans would have added to the total of positive reports and been buried among them.

A single observation of a black swan, albeit only anecdotal, uncontrolled and subject to the potential fallibility of human senses and reporting systems, but made in defiance of all the biases that would militate against the observer believing the evidence of his eyes, has an overwhelmingly huge effect on the post-observation probability of the validity of the hypothesis that all swans are white.
__________________
"i'm frankly surprised homeopathy does as well as placebo" Anonymous homeopath.
"Alas, to wear the mantle of Galileo it is not enough that you be persecuted by an unkind establishment; you must also be right." (Robert Park)
Is the pen is mightier than the sword? Its effectiveness as a weapon is certainly enhanced if it is sharpened properly and poked in the eye of your opponent.
Badly Shaved Monkey is offline   Quote this post in a PM   Nominate this post for this month's language award Copy a direct link to this post Reply With Quote Back to Top
Old 16th September 2012, 10:58 AM   #7
r-j
Banned
 
Join Date: Nov 2008
Posts: 2,689
I don't understand that at all. But I have, various times in my long life, watched certain people try to argue in such a way that they discount all observations (anecdotal evidence) which backfires on them, because they lose all credibility in doing so.

It's like telling somebody who owns a few black swans that black swans do not exist.
r-j is offline   Quote this post in a PM   Nominate this post for this month's language award Copy a direct link to this post Reply With Quote Back to Top
Reply

International Skeptics Forum » General Topics » Science, Mathematics, Medicine, and Technology

Bookmarks

Thread Tools

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Forum Jump


All times are GMT -7. The time now is 10:54 PM.
Powered by vBulletin. Copyright ©2000 - 2020, Jelsoft Enterprises Ltd.

This forum began as part of the James Randi Education Foundation (JREF). However, the forum now exists as
an independent entity with no affiliation with or endorsement by the JREF, including the section in reference to "JREF" topics.

Disclaimer: Messages posted in the Forum are solely the opinion of their authors.