Re: [sl4] Universal versus 'local' Friendliness

From: Amon Zero (
Date: Mon Mar 07 2011 - 10:03:39 MST

Hi All - Replies to multiple people in this post:

On 6 March 2011 19:18, Monica Anderson <> wrote:

> Have you read

Hadn't seen that, it's interesting, thanks Monica!

Tim Freeman said:
>Thus the "locally Friendly" problem isn't easier to solve than the general
FAI problem.
>Thus identifying the in-group for the AI isn't going to help it win much.

Thanks Tim. Yes, both very good points I think. The first has come up on
other lists where I also posted my question. Aswell as making me realise
that I imagined a universally Friendly AI to be some kind of Buddhist
(Friendly to everything, erring on the side of caution - which sounds
crippling to me) has lead to some people discussing a refined version of the
question: Would a narrow-circle definition of "Friendlies" be easier to
define & implement than a wide one? Why / why not? (Alternatively, *at what
point* does a circle of defined-Friendlies become wide enough to be
problematic? Is there such a point?).

Your second point is a new one to me, and hasn't come up elsewhere in this
discussion. Obviously I don't have any answers, but I appreciate the food
for thought!

On 6 March 2011 20:24, Mark Waser <> wrote:

> Hi,
> The list is certainly dormant although I believe that it still sends
> any messages to any number of people that you'd like to get the attention
> of.
> I'm cross-posting my answer to my blog (
> as well in hopes of a greater chance
> of more discussion (though I'll answer any replies wherever they appear ;-).

Hi Mark -

Thanks very much for that! I'll have to chew on this a bit and get back to
you. As I mentioned I've posted this elsewhere (Exi & ExtroBritannia), and
will definitely check out your blog for any insight. I'm hoping to pull
together anything learned from this (optimistic, I know!) and write up a
summary, so will let you know if/when I manage to make that happen.

All the Best,

This archive was generated by hypermail 2.1.5 : Wed Jul 17 2013 - 04:01:05 MDT