From: Rolf Nelson (rolf.h.d.nelson@gmail.com)
Date: Wed Oct 24 2007 - 20:32:57 MDT
On 10/23/07, William Pearson <wil.pearson@gmail.com> wrote:
>
> It is really a shame there isn't (too my knowledge) a database of
> inventors and their creations and failures, I could run some stats on
> and look for common themes and percentage success rate, create
> hypotheses and crunch some data.
In the ideal world, yes. But, in the real world, as you know, you won't find
structured, unbiased, relevant data to "feed into a Bayesian classifier" on
this subject. (If I'm wrong let me know, I'm sure we'd all love to use the
dataset as well!)
I try to view it as a spectrum: at the bottom of the low end would be
wishful thinking and the abandonment of reason; at the top of the high end
would be inferring the existence of quantum mechanics based on a Universal
Prior and your direct observations, and then inferring chemistry from
quantum mechanics, and so on until you figure out there's a 30.1872% chance
that Tom Hanks will win an Oscar. We don't have the cognitive abilities to
reach the top of the high end, but instead of despairing, we should try to
do the best we can with the data we have.
Bayesian reasoning involves determining a prior probability "before you've
considered the data", and then getting a final posterior probability by
shifting based on the data you've collected. It can be applied if you have
2000 data points, or if you have 2 data points, or even if you have 0 data
points (in which case the posterior probability remains trivially equal to
the prior probability.)
If we have 2000 good independent data points, great, we can use statistical
techniques and don't even have to worry much about what the prior is; but if
we have only 2 good data points, and can't find more, then we just have to
infer a prior, shift a little bit on the minimal data we have, and make our
decisions despite the uncertainty.
-Rolf
This archive was generated by hypermail 2.1.5 : Wed Jul 17 2013 - 04:00:58 MDT