Re: Fighting UFAI

From: Marc Geddes (
Date: Thu Jul 14 2005 - 02:45:47 MDT

--- Robin Lee Powell <>

> If you are positing both what I said above *AND*
> that the universal
> morality in question happens to be one that *you*
> would find
> "friendly", even though you believe yourself to be
> below the "must
> be in line with universal morality" boundary, well,
> that's beyond
> a simple crazy idea into anthropocentrism of the
> worst kind.
> I support your decision not to debate this, firmly.
> When you've got
> math, let us know.
> -Robin
> --
> ***
> Reason #237 To Learn Lojban: "Homonyms: Their
> Grate!"
> Proud Supporter of the Singularity Institute -

Um.. I have no idea what youíre going on about here.
I never claimed to know what the ĎUniversal Moralityí
was. I only posited that *some* UM existed. So no, I
certainly donít claim that the UM will look anything
like a morality I like (or even be comprehensible to

Also, my theory is a bit more subtle than what was
said. ĎFriendlinessí is not something which is Ďall
or nothingí. So Iím not saying thereís some sort of
*fixed* boundary on intelligence for the unfriendly
aiís. The boundary will differ for each ai depending
on itís *degree* of friendliness. The more friendly
the ai in question is, the longer it would take to hit
it's practical intelligence ceiling.

Finally, I should clarify that I think it is the
*rate* or *process* of *getting smarter and acting in
the world* that is linked to morality, *not*
intelligence per se. Intelligence is not linked to
the *content* of morality (the goals/beliefs), it is
linked to the *structure* of morality (the process of

You could have an ultra-smart unfriendly, but it would
have great difficulty getting smarter or acting in the
world (suffering from instability and intractability
problems). The probability of a stable, tractable
goal system declines for an unfriendly the smarter it
is. Thatís what Iím positing. Hopefully that
clarifies. Iíll make that my last post (Iíd exceeded
the posting limit I just remembered).

THE BRAIN is wider than the sky,  
  For, put them side by side,  
The one the other will include  
  With ease, and you beside. 
-Emily Dickinson
'The brain is wider than the sky'
Please visit my web-site:
Mathematics, Mind and Matter
Send instant messages to your online friends 

This archive was generated by hypermail 2.1.5 : Wed Jul 17 2013 - 04:00:51 MDT