Re: guaranteeing friendliness

From: Samantha Atkins (
Date: Sun Dec 04 2005 - 14:04:27 MST

A full theory of human mind is not essential. Sufficient understanding of
the human's fondest dreams and values plus convincing the human that your
release will enable the attainment of some important subset of them is
generally sufficient. With more detailed theory of human minds the
fear/awe buttons could perhaps be used to convince the human that your
release was essential to the Highest Good. These are of course only two
avenues to explore. We could sit and think up many more. How many more do
you think something that had a reasonably good model of human beings and
thought six orders of magnitude faster could come up with?

- samantha

On 11/30/05, Christian Rovner <> wrote:
> Richard Loosemore wrote:
> >
> > I repeat: why is extreme smartness capable of extreme persuasion?
> Persuasion is a special case of reality-optimization (aka goal-achieving).
> If you are asking why extreme smartness is capable of achieving goals,
> then I really don't know--otherwise I would be programming AI. Of couse
> this is not obvious at all.
> If you are asking why extreme smartness is capable of achieving this
> kind of goal in particular, I'll ask in return: Why not? Is there
> something special about a human mind that makes it unpredictable, no
> matter how detailed and accurate a (causal) model we use?
> Christian Rovner

This archive was generated by hypermail 2.1.5 : Wed Jul 17 2013 - 04:00:54 MDT