determining what constitutes a success

Joined
Aug 4, 2006
Messages
926
This seems like a question that would've been posed here a lot but I don't see it on the first page here. Can anyone who's more familiar with the challenge comment on how it's determined that an event that a claimant causes is unlikely enough due to chance to constitute a success?

For example if someone claims to be able to predict outcomes of coin flips obviously it wouldn't be considered a success if they successfully predicted 5 coin flips, but how many flips would have to be accumulated and what would the final probablity of the guesses/outcomes have to be (after all the flips) to be considered a success?

And if the claimant one time successfully guesses a 10 digit number I'm assuming that wouldn't be considered a success either because it still could be a fluke, but how many times would he/she have to do that for it to be verified as a success?

Thanks to anyone with a better sense of this who answers.
 
Depends on the odds.
But for coinflips it would probably be something like getting 80-90%+ of some 20 coin tosses correct.

10± a bit would be average just on chance. with the ± a bit guessing 12-13 correct isn't good enough, but go to 18 out of 20 then it is closer.


NOTE: i'm just some guy, and i'm sure Darat or someone else will correct me later. :D
 
The number of accurately predicted coinflips necessary to constitute a succes would be predetermined and mutually agreed upon by the JREF and the claimant.

The claimant will have to state his expected succesrate, then JREF will calculate the number of tries and successes necessary to meet the required odds. For example, it takes far fewer tries to demonstrate/disprove a 100% succesrate than a 70% succesrate.
 
This seems like a question that would've been posed here a lot but I don't see it on the first page here. Can anyone who's more familiar with the challenge comment on how it's determined that an event that a claimant causes is unlikely enough due to chance to constitute a success?

For example if someone claims to be able to predict outcomes of coin flips obviously it wouldn't be considered a success if they successfully predicted 5 coin flips, but how many flips would have to be accumulated and what would the final probablity of the guesses/outcomes have to be (after all the flips) to be considered a success?

And if the claimant one time successfully guesses a 10 digit number I'm assuming that wouldn't be considered a success either because it still could be a fluke, but how many times would he/she have to do that for it to be verified as a success?

Thanks to anyone with a better sense of this who answers.

Welcome to the Forum, latent aaaack.

The success rate depends on the claim - and the cockyness of the applicant. Tobias already gave you the ballpark.

Perhaps you will find threads that spark your interest on other pages int he Forum. Enjoy the search function. Take your time.

For an overview and an insight into what success rate the JREF asks from applicants, check out the Challenge Application Forum. Look for the threads with several posts, as the negotiations between the JREF and the applicant have made at least some progress and therefore hold useful information for you.

The last two tests took place with applicants named Achau Nguyen and Angela Patel.

Enjoy the Forum.
 
This seems like a question that would've been posed here a lot but I don't see it on the first page here. Can anyone who's more familiar with the challenge comment on how it's determined that an event that a claimant causes is unlikely enough due to chance to constitute a success?

For example if someone claims to be able to predict outcomes of coin flips obviously it wouldn't be considered a success if they successfully predicted 5 coin flips, but how many flips would have to be accumulated and what would the final probablity of the guesses/outcomes have to be (after all the flips) to be considered a success?

And if the claimant one time successfully guesses a 10 digit number I'm assuming that wouldn't be considered a success either because it still could be a fluke, but how many times would he/she have to do that for it to be verified as a success?

Thanks to anyone with a better sense of this who answers.
Here's a laws of chance table.

And welcome to the boards.
 
The claimant will have to state his expected succesrate,

I'm more interested in what the claimant can do, rather than what they believe they can do. What if they are incorrect about their own assesment. For example, they state their expected success rate of 80%, but they can really perform at 60%.

Then if they get 70%, they are really performing acceptably, but you would say they'd fail.
 
I'm more interested in what the claimant can do, rather than what they believe they can do. What if they are incorrect about their own assesment. For example, they state their expected success rate of 80%, but they can really perform at 60%.

Then if they get 70%, they are really performing acceptably, but you would say they'd fail.

Tai:
I claim I can view skullplates using xray vision,
but in fact can do no such thing.
(yes, in this example my belief is contrary to reality, same as all woos)
I state my expected success rate as 93.276%.
I can really perform at 0%.
When tested I have a success rate of around 10%.
I am performing acceptably!!!
Whould YOU say that I passed or failed?
 
I'm more interested in what the claimant can do, rather than what they believe they can do. What if they are incorrect about their own assesment. For example, they state their expected success rate of 80%, but they can really perform at 60%.

Then if they get 70%, they are really performing acceptably, but you would say they'd fail.

You are assuming that just by chance they anyone would perform at around 50% or less?

I think a proper protocol could easily set a low enough margin, even though I think I have seen instances where this was doubtful.

But it really don't matter one way or another: They will not perform any better than chance, and if they do, they can still re-apply another year.

(Funny, someone is supposedly able to bend and break the known laws of the physical universe, but they mess up when it comes to simply calculating a lousy percentage ...)
 
You are assuming that just by chance they anyone would perform at around 50% or less?

No, I did not assume that.

They will not perform any better than chance, and if they do, they can still re-apply another year.

"They will not" and "if they do" are contradictory.
 
I'm more interested in what the claimant can do, rather than what they believe they can do. What if they are incorrect about their own assesment. For example, they state their expected success rate of 80%, but they can really perform at 60%.

Then if they get 70%, they are really performing acceptably, but you would say they'd fail.
To the best of your knowledge, has any claimant performed significantly better than random chance? You decide how to define significantly, just state what your definition is.
 
I'm more interested in what the claimant can do, rather than what they believe they can do. What if they are incorrect about their own assesment. For example, they state their expected success rate of 80%, but they can really perform at 60%.

Then if they get 70%, they are really performing acceptably, but you would say they'd fail.

In recent protocol negotiations, 70% accuracy was considered an acceptable performance for a preliminary test. KRAMER himself brought it up during correspondence with applicants.
 
Well, back on the original topic (and ignoring the troll and it's baiters :)), I believe it's been stated that the odds of winiing by chance are adjusted to be less than 1 in 1000 for the preliminary and 1 in 1,000,000 for the final test. You can use something like the aforementioned laws of chance table to determine what it would take for that.

If I claim to be able to predict coinflips at 100% accuracy, then I'd have to be able to correctly predict 10 out of 10 flips for the preliminary, and 17 out of 17 for the final. Now, the JREF will tend to vary this so you can miss a few, but I'm not sure how to calculate "X out of Y" hits chances, but that should give you an estimate.
 
Who knows, datasets on performance apparently aren't released.
Or apparently they are, the problem is just that very few people actually show up for the preliminary test. Here's from a few cases where the applicants decided to show up:


Result of the Hans-Peter Borer test
This is to inform you about the test of the Swiss candidate Hans-Peter Borer, which took place last Saturday. He claimed to find a (switched-on) mobile phone with the help of a pendulum.

We gave him our standard test, consisting of 13 trials which in turn consist of a 1:10 set up each.

His result: 2 hits.
http://www.internationalskeptics.com/forums/showthread.php?t=29864



Result of the Achau Nguyen test
To tabulate the results, we brought out a white scoreboard which displayed columns for “Word Sent,” “Word Received,” and “Running Score”. The sequence “sent,” “received,” “score” was read for all 20 words. No “Sent” words were even close to the “Received” words. (e.g. The first word he sent was “ovary”, though E. received “shopping mall.”) The running score became a column of zeros.
http://www.internationalskeptics.com/forums/showthread.php?t=28936
http://www.csicop.org/specialarticles/nguyen.html



Result of the Angela Patel test
When the time then came to reveal the results there will be no surprises when I say that Angela scored zero out of three. She was keen to see if the real addresses were close to what she had discovered, but she remarked with disappointment, "they weren't even on the same page!"
http://www.internationalskeptics.com/forums/showthread.php?t=34609



Look, I know you're just a troll, but we have to take turns in exposing your lies sometimes.

ETA: Ooh, Claus was there before me I see now, who would have known ;)
 
Last edited:
I'm more interested in what the claimant can do, rather than what they believe they can do. What if they are incorrect about their own assesment. For example, they state their expected success rate of 80%, but they can really perform at 60%.

Then if they get 70%, they are really performing acceptably, but you would say they'd fail.
Claiming ignorance about basic statistics and having a homepage named statisticool? Cute...
 
I think the answer depends on what the claim is and what the chance of it happening by chance is. If you claim to be able to dowse water with a W% success rate, then the test will be constructed with X trials and Y containers, one of which contains water, and you will have to succeed in Z of the trials. The exact values for W, X, Y and Z would be probably be determined by JREF's statistical expert(s).

I think the chances mentioned above are correct, the chance of passing the preliminary test by chance is 1 in 1 000, and the chance of passing the challenge by chance would be 1 in 1 000 000.

Even if you fail you could "win". Say you claim to be able to predict coin flips 100% of the time but only succeed 80% of the time, that would be enough to get some scientists interested in studying your ability. You could also re-apply in a year, claiming a 70% chance of success, and you'd win the challenge then.
 

ISF - Join now!

Every member here is approved by hand. No bots, no spam, just people who care about evidence and honest debate.

Membership is free!

Create your free account

Back
Top Bottom