Re: On the dangers of AI (Phase 2)

From: Randall Randall (
Date: Wed Aug 17 2005 - 15:59:08 MDT

On Aug 17, 2005, at 3:49 PM, Richard Loosemore wrote:
> Now, I know I *could* in some future world, flip a switch in such a
> way that I would enjoy being gay, or being a woman. I know that after
> I flipped the switch, I would enjoy it deeply, even though the thought
> makes my skin crawl right now.
> Would I flip the switch? I would be happy to do it.
> Now please explain which motivation system allowed me to make that
> choice?

When faced with a choice (to flip or not to flip such a switch, in this
case), how do you decide the outcome of the choice? "Pure thought" is
only useful as a tool to examine outcomes against goals. In order to
a choice, you have to have some method for measuring "better" outcomes
internally. Whatever direction or scale you use to measure "better" is
what other people here are calling your "goal". It may be that you have
more than one goal, or that you have a (large) set of conflicting goals
held by various subsystems of you, but each decision you make, each time
you choose between "better" and "not as good", the measurement is a
reflection of the goal or goals involved in the decision.

It's not enough to demand that someone show you which particular motive
or goal produced a given decision; the fact that you decided presupposes
a reason to have done so.

Randall Randall <>
Property law should use #'EQ , not #'EQUAL .

This archive was generated by hypermail 2.1.5 : Wed Jul 17 2013 - 04:00:51 MDT