From: Damien Broderick (thespike@satx.rr.com)
Date: Sun Jan 01 2006 - 14:28:33 MST
At 03:39 PM 1/1/2006 -0500, John K Clark wrote:
>I don't think experimenter fraud
>counts for more than 20%, test subject fraud is about 50%, and juggling
>statistics to fool oneself about 30%.
Here's a fairly recent summary by a physics Nobel laureate and a
professional statistician:
http://anson.ucdavis.edu/~utts/azpsi.html
< Strong statistical results are of course meaningless if experiments are
not properly conducted. Debunkers of parapsychology are fond of showcasing
the very few experiments that have been found to have serious problems. But
that ignores the fact that the vast majority of experiments were done using
excellent protocols, paying close attention to potential subtle cues, using
well-tested randomisation devices and so on. For the past decade the U.S.
government experiments were overseen by a very high-level scientific
committee, consisting of respected academics from a variety of disciplines,
all of whom were required to critique and approve the protocols in advance.
There have been no explanations forthcoming that allow an honest observer
to dismiss the growing collection of consistent results. >
>When paraphysiologists test a
>professional physic they ALWAYS get a much stronger psi response when
>they adopt a carefree attitude toward experimental controls. Gee, I
>wonder why?
Untrue. The key finding from recent work is that the more scrupulous and
rigorous the controls, the more robust the results (within the limits of
what is feasible for a low-power phenomenon). John can you please provide
references to support your assertion?
To emphasize my earlier remarks about small effect sizes and implications
for replication, here's a segment from the same paper:
< But, as anyone with a training in statistics knows, even where an
influence exists, an isolated experiment with an insufficient number of
trials may not demonstrate a statistically significant effect. Accordingly,
without a more sophisticated analysis, "failure to reproduce an effect"
does not demonstrate its absence. Suppose, for example, psychic abilities,
in line with the results already described, increase the chances of a
successful match from 1/4 to 1/3. Then (according to the accepted
statistical theories), an experiment with 30 trials, which has been typical
of these experiments, would have less than a 17% chance of achieving a
result of statistical significance. The more recent larger experiments
still utilise only about 100 trials, and have only about a 57% chance of
achieving statistical significance.
Detailed analysis of the complete collection of experiments on this type of
phenomenon shows that what holds, despite changes in equipment,
experimenter, subjects, judges, targets and laboratories, is far greater
consistency with the 1 in 3 success rate already mentioned than with the 1
in 4 chance expectation rate. Such consistency is the hallmark of a genuine
effect, and this, together with the very low probability of the overall
success rate observed occurring by chance, argues strongly for the
phenomena being real and not artifactual.
Reexamination of other types of psychical investigations reveals that they
too achieved replicable effects, which went largely unappreciated because
of a poor understanding of statistics. For instance, an analysis of
experiments in precognitive card guessing and related "forced-choice"
experiments, published by Honorton and Ferrari in the Journal of
Parapsychology, found that gifted subjects were able to achieve
consistently about a 27% success rate when 25% was expected by chance.
Similar U.S. government experiments have been revealed to have achieved the
same 27% success rate over thousands of trials. If chance alone were the
explanation for these results, it would be truly remarkable to achieve a
27% success rate over thousands of trials, and it would be even more
remarkable to see identical results in the government work. For further
details about the recent evidence, including both a favourable and a
skeptical assessment of the U.S. government experiments, consult the
Journal of Scientific Exploration, Vol. 10(1), or
<http://www-stat.ucdavis.edu/users/utts/>http://www-stat.ucdavis.edu/users/utts/
on the Internet. >
What's going on here, I think, is revealed in John's caustic but mis-aimed
comments about
"a waitress with an eight?s grade education in Duluth Minnesota ... my
ghostly spirit guide Mohammad Dun told me spoke to me about it in a dream."
and so on. This is absolutely irrelevant to the claims of scientists
working on parapsychology. Yes, such folk beliefs surely created the
initial background interest, a century ago, in apparently paranormal
phenomena, and is still used by confidence tricksters to bilk idiots. So
what? Forget about them. Talk about the real experiments. Utts and others
cite plenty of published papers. Jeering at someone else entirely do not
advance your argument one whit; in fact, it makes it look as if you're
evading the issue.
Damien Broderick
This archive was generated by hypermail 2.1.5 : Wed Jul 17 2013 - 04:00:54 MDT