From: Ben Goertzel (ben@goertzel.org)
Date: Fri Feb 24 2006 - 22:15:56 MST
Hi,
> You're trying to solve a decision problem: whether to try for FAI or
> rush for the Singularity.
Actually I am not trying to solve exactly that decision problem; I am
working on AGI and FAI at the same time and trying to understand the
relation between them....
At some point my colleagues and I may need to try hard to solve that
decision problem in a more rigid sense -- if I have a powerful AGI on
hand and I have to decide whether to set a switch that will let it
start self-modifying and hence potentially move toward hard takeoff.
I am not at that point now....
I have however decided to spend more of my time working on AGI design
and engineering than on FAI theory...
> In order to solve a decision problem, you need
> to apply a decision theory.
This is not necessarily true, actually...
>The most powerful and general decision
> theory we have is based on Bayesian Probability Theory.
Well, probability theory is very valuable when one has enough data to
feed into it. In the case of the particular decision you cite, I
don't think there is enough data that current probability-theory
methods are very helpful.
> Maybe you think that BPT-based decision theory is imperfect, but your
> Indifference-based decision theory is weaker and less powerful than
> Bayesian decision theory.
I am not basing any of my own personal or professional decisions on an
"indifference-based decision theory," at least not consciously or
explicitly.
I cited the indifference principle very explicitly as part of a
(possibly misguided) attempt to clarify SOMEONE ELSE'S ARGUMENT made
on this list, not as part of a presentation of my own point of view.
I'm sorry that this led to misunderstanding. I believe I was quite
explicit in my wording, though.
> If you're not confident enough in Bayesian
> decision theory to solve this problem, then you need to develop a
> stronger decision theory, not a weaker one!
In fact I have spent a fair bit of effort working on ways to augment
probability theory with heuristics that are effective in cases when
there is not enough data to apply current probabilistic methods. (In
principle I consider this a weak way out, because I have a feeling
that "purer" probabilistic methods probably do suffice for all cases
of pragmatic reasoning in the classical-physics domain: but
probability math is far from fully developed, and in practice there
are a lot of practical decision problems where there is not enough
data to make judgments using known probability methods.)
> > As a side point: After all, the general applicability of classical
> > probability theory is *already* in question even within the human
> > world, via Youssef's quantum probability theory...
>
> When I talk about Probability Theory, I mean in the Bayesian sense. If
> there's a conflict between QM and Bayesian Probability Theory, you'll
> have to point it out to me.
The precise nature of the conflict depends on the QM formalism that you choose.
You can choose quantum logic + classical probability theory, or you
can choose Boolean logic + complex probability theory.
See the papers here
http://physics.bu.edu/~youssef/quantum/quantum_refs.html
for information on the latter option
Either approach formalizes the way in which the quantum world violates
standard probability theory.
-- Ben
This archive was generated by hypermail 2.1.5 : Wed Jul 17 2013 - 04:00:56 MDT