From: Randall Randall (randall@randallsquared.com)
Date: Thu Jun 03 2004 - 18:13:45 MDT
On Jun 3, 2004, at 7:38 PM, Ben Goertzel wrote:
>> But, in any case, building a very clever system to reach
>> a goal (Friendliness) seems to me to be more in line with
>> what Eliezer is doing than building a generalized,
>> humanlike person. Since it seems easier to build that
>> than a humanlike person, it would be reasonable to worry
>> about the attractors that other projects might fall into.
>
> I'm not sure why you think it's easier to build this kind of
> single-goaled, super-powerful optimization process, than to build a
> human-level self-improving general intelligence.
You're reading "super-powerful" into what I said. I'm saying
merely that given building a goal-directed system seems less
complex than building a goal-directed system *and* a system
which can differentiate goals *and* a system that prevents
goals from becoming the only goal, which would revert to the
earlier case.
This is analogous to the governor on a boiler. It's easier to
build a boiler without a governor, but unsafe. To build a
governor for the boiler, though, you have to understand all
the failure modes you are preventing.
I, personally, have seen no reason to believe that Friendliness
is possible, so I'm hoping that none of you succeed before
molecular manufacturing provides an escape route for us doubters.
> One important point is that we have an example of a human-level general
> intelligence -- billions of examples, in point of fact. But we have no
> examples of the kind of optimization process Eliezer's proposing now,
> so
> to construct one, we must proceed entirely based on theory and
> experimentation. And IMO current mathematical and computing theory
> does
> not bring us very far toward knowing how to create this kind of
> optimization process within reasonable computational space and time
> constraints.
Aren't single-goal optimization processes what general optimization
processes (e.g., people) built from? If so, one would seem to have
to understand the optimization process before one adds on the goal
determination and decision process, since that's a special case of
the optimization process.
-- Randall Randall <randall@randallsquared.com> 'I say we put up a huge sign next to the Sun that says "You must be at least this big (insert huge red line) to ride this ride".' -- tghdrdeath@hotmail.com
This archive was generated by hypermail 2.1.5 : Tue Feb 21 2006 - 04:22:38 MST