From: Ben Goertzel (ben@goertzel.org)
Date: Tue Feb 22 2005 - 22:30:37 MST
> Corrolary : Regardless of the speed at which the intelligence of the
> AGI grows, the options available to the AGI increase with the
> intelligence of AGI. Incremental increases to IQ do not result in
> local minima, because the horizon of the AGI is pushed wider with each
> improvement.
This doesn't seem to follow mathematically. For instance, the series
1/2, 3/4, 7/8, ... etc.
keeps increasing at each step but it approaches an upper boundary rather
than increasing indefinitely.
Your argument doesn't show that something like this can't occur...
But also, even if this objection were overcome, the "local minima" argument
isn't the only complaint. There may be a lot of ways to increase the number
of available options toward infinity -- and some of them may be more
interesting than others -- and the safe incremental approach may shut off
the more interesting ones...
But still, the BIGGEST objection to the ITSSIM approach is that it prevents
future AI's from mitigating existential risks if doing so would decrease its
own safe-ness.
I have a variant version that doesn't display this problem so severely,
which I'll post something about shortly.
-- Ben
This archive was generated by hypermail 2.1.5 : Wed Jul 17 2013 - 04:00:50 MDT