From: Philip Sutton (Philip.Sutton@green-innovations.asn.au)
Date: Thu May 27 2004 - 08:09:31 MDT
Ben/Eliezer,
I've probably failed to understand the concepts here, but I'll keep going
anyway!
Eliezer said:
> External reference semantics says, "Your supergoal content is an
> approximation to [this function which is expensive to compute] or [the
> contents of this box in external reality] or [this box in external
> reality which contains a description of a function which is expensive
> to compute]."
Then Ben said:
> One good thing about this idea, conceptually, is that it has the
> potential to make the AI's goal system fundamentally social rather than
> solipsistic. If the external referent for Friendliness is "my friends'
> conception of Friendliness" then the society of the AI's friends is
> brought into the dynamics of self-modification.
>
> i.e.: What is the external referent? (cut) If it's
> something informal like "my friends' understanding of Friendliness" then
> there is too much potential for the AI to influence the external referent,
> thus making it no longer "external."
An 'external referent' to "*my* friends conception of Friendliness"
cannot be fully external because they are *my* friends. What if you
take out the self-referential bit and make the external referent as close
to fully universal as you can get ie. "Your supergoal content is an
approximation to [ friendliness as would be maximally acceptable to all
sentients everywhere ]
This is a bit like a super goal with external referent: "Do unto others as
they would have you do unto them (while not turning yourself into a
slave!)".
Cheers, Philip
This archive was generated by hypermail 2.1.5 : Wed Jul 17 2013 - 04:00:47 MDT