Re: Defining Right and Wrong

From: Gordon Worley (
Date: Tue Nov 26 2002 - 18:55:03 MST

On Tuesday, November 26, 2002, at 01:18 PM, Michael Roy Ames wrote:

> intelligence as a player. Now the question may be asked (I asked it
> ;) do
> we give FAI our current HEM and nothing else?

Irrelevant. If the Theory of FAI works, then even if you try to give
it Nazi morality it will find the correct morality (assuming that Nazi
morality is not correct, which is doubtful).

> Will this lead to the optimum
> results?

Unless we are very lucky (or unlucky, depending on how you look at it),
we might be able to estimate correct Friendliness content and maybe
help it a little bit (give it some priors it won't have to change).
The FAI will still need to verify everything, so the advantage would be
minimal. The whole point is that the programmers can't make the FAI
into a UFAI.

> Will this give good results at all? Sure, HEM may be a good
> start, but what comes after? And how do we know its better?

We'll have a hard time telling. That's why the theory has to work, or
I'm breaking out the fruit preserves.

Gordon Worley                          "Man will become better when                 you show him what he is like."                                --Anton Chekhov
PGP:  0xBBD3B003

This archive was generated by hypermail 2.1.5 : Wed Jul 17 2013 - 04:00:41 MDT