From: Eliezer S. Yudkowsky (sentience@pobox.com)
Date: Wed Feb 06 2002 - 23:10:40 MST
Gordon Worley wrote:
>
> I chose to write `right' because that's more accurate than `Friendly'.
> Friendliness is likely in the same spirit as what will be right for SIs,
> but not the same thing. We should expect SIs to come up with a theory
> that's better than Friendliness pretty shortly. Remember, all that
> Friendliness has to do is produce a fairly super intelligent AI that's
> safe so that we have a safe starting point for entering the
> Singularity. I fully expect a better theory to be developed at some
> point by the SIs. If Friendliness is still in use post Singularity,
> either Eliezer did an amazing job coming up with Friendliness or our
> Singularity has failed to reach its full potential.
Let's distinguish between Friendliness and meta-Friendliness, what CFAI
calls "Friendship content" and "Friendship structure". All I want from
life is to find out that SIAI's original Friendship structure was in fact
human-morality-complete, not in terms of content but in terms of
structure, so that any improvements over Friendliness or meta-Friendliness
can be seen as desirable by the original Friendship structure. Given that
humans themselves possess only a bounded amount of complexity with which
to execute their JOOTS, and that none of it is Singularity-inscrutable,
the task of building an AI that's as open-minded as any human should also
involve a bounded number of Aha!s.
-- -- -- -- --
Eliezer S. Yudkowsky http://intelligence.org/
Research Fellow, Singularity Institute for Artificial Intelligence
This archive was generated by hypermail 2.1.5 : Wed Jul 17 2013 - 04:00:37 MDT