From: Ben Goertzel (ben@goertzel.org)
Date: Wed Jun 02 2004 - 08:30:11 MDT
Hi,
> > I suspect there is some kind of INTELLIGENCE INDETERMINACY
> PRINCIPLE
> > (IIP) of the rough form.
>
> This is a neat summary of the basic pathology behind the
> probabilistic approach to goal system design and
> self-modification. If this principle existed as stated, the
> SIAI project would be in serious trouble.
Yes, in my view, the SIAI project is in serious trouble
> Fortunately
> intelligence is the power to remove uncertainty, not increase it.
On the contrary, intelligence can do either one, depending on context...
In purely analytic mode, intelligence may remove uncertainty; but in
creative mode, it will often increase it...
> > [FAI] doesn't require that the self-modification of an AI system be
> > tractably deterministic... it merely requires that certain
> > probabilities about the future evolution of the system be estimable
> > with a high degree of confidence... it argues for the
> shutting-down of
> > some dead-end paths toward Friendly AI, such as the path
> that seeks an
> > AI whose overall behavior is in some sense predictable.
>
> I don't understand what you're saying here.
You are quoting Eliezer or somebody else there, not me.
> Probabilities of
> the system exhibiting given behaviour classes being estimable
> seems like 'overall behaviour that is in some sense
> predictable' to me. Without a resolution of that
> contradiction I can't dispose of the rest of your argument.
The contradiction is between my quote and somebody else's.
> Clearly the self-modification has to be tractable and
> deterministic for the AI to do it in the first place, but we
> want to know what will happen without running a takeoff.
> There is no way to do this probabilisticaly; fortunately it
> is possible to design architectures and goal contents that
> will provably stay in some useful area of utility-function-space.
OK, well, I admit it's possible you're right, but I strongly doubt it.
Certainly I have been wrong before.
I look forward to seeing your architecture/goal-contents and your formal
proof.
-- Ben
This archive was generated by hypermail 2.1.5 : Tue Feb 21 2006 - 04:22:38 MST