International Skeptics Forum - View Single Post - How to manipulate data, get results as good as PEAR, and be $1,000,000 richer

fls · 2nd March 2007, 11:57 AM

Originally Posted by Baron Samedi

Here we go. I tried your idea and ran a new simulation. I tested at 30, 386, and all of the primes inbetween, and only at these points if my observed Z > 3.1 do I stop and say it's a success with no need to continue. I also bumped it up to 1,000,000 simulated reps, just to make sure I get a good estimate.

Unrestricted (re-evaluate after each observation)
Success: 11,587
Failed: 988,413

Restricted (re-evaluate at primes)
Success: 9,216
Failed: 990,784

Doing that prime number restriction just dropped the result down to 0.92%. So with 400 subjects, each one testing between 30 to 386 guesses, I have a great chance of getting 3 "psychics", maybe more.

I'm sorry. Now I feel bad that you did all that work. I didn't give an adequate description of what I meant. That result is not at all surprising (except that I'm surprised it dropped as much as it did).

I meant that out of your 10,000 (or 1,000,000) trials, the only trials that you start testing for signficance after 30 guesses would be trials number 2, 3, 5, 7, 11, 13, 17, etc. And I chose prime numbers for an example, but I don't know whether or not the proportion of numbers that are prime numbers is equal to the proportion of trials subject to early review (I suspect using prime numbers is way too high - something like squares of whole numbers may be closer).

Linda

2nd March 2007, 11:57 AM	#20
fls Penultimate Amazing Join Date: Jan 2005 Posts: 10,226	Originally Posted by Baron Samedi Here we go. I tried your idea and ran a new simulation. I tested at 30, 386, and all of the primes inbetween, and only at these points if my observed Z > 3.1 do I stop and say it's a success with no need to continue. I also bumped it up to 1,000,000 simulated reps, just to make sure I get a good estimate. Unrestricted (re-evaluate after each observation) Success: 11,587 Failed: 988,413 Restricted (re-evaluate at primes) Success: 9,216 Failed: 990,784 Doing that prime number restriction just dropped the result down to 0.92%. So with 400 subjects, each one testing between 30 to 386 guesses, I have a great chance of getting 3 "psychics", maybe more. I'm sorry. Now I feel bad that you did all that work. I didn't give an adequate description of what I meant. That result is not at all surprising (except that I'm surprised it dropped as much as it did). I meant that out of your 10,000 (or 1,000,000) trials, the only trials that you start testing for signficance after 30 guesses would be trials number 2, 3, 5, 7, 11, 13, 17, etc. And I chose prime numbers for an example, but I don't know whether or not the proportion of numbers that are prime numbers is equal to the proportion of trials subject to early review (I suspect using prime numbers is way too high - something like squares of whole numbers may be closer). Linda