From: Mitch Howe (mitch@iconfound.com)
Date: Sat Oct 19 2002 - 13:20:41 MDT
Gordon Worley wrote:
> Friendliness allows an FAI the ability to find morality. And not just
> any morality, but the correct morality. And if there is no correct
> morality, then it will figure that out, too. Most of your post
> discusses volition-based morality, not volition-based Friendliness (I'm
> not even clear on what volition-based Friendliness is supposed to
> be--your own version of Friendliness?
Well, I didn't just make up the term "volition-based Friendliness", nor did
I work from a completely made up definition. Quoting CFAI, section 1.3:
"Thus, Creating Friendly AI uses "volition-based Friendliness" as the
assumed model for Friendliness content. Volition-based Friendliness has
both a negative aspect - don't cause involuntary pain, death, alteration, et
cetera; try to do something about those things if you see them happening -
and a positive aspect: to try and fulfill the requests of sentient
entities."
> Volition-based morality is a theory that a human named Eliezer S.
> Yudkowsky has proposed as being closer to correctness than, say,
> Christian morality. Humans, though, have too limited a mental capacity
> to check the correctness of a moral theory, so an FAI's initial moral
> theory (if one is given to the AI) is irrelevant because all known
> theories are equally likely to be wrong.
There's no guarantee that an SI can check the correctness of a moral theory
either, since we don't know yet what that will take. I think it's all well
and good to recognize that, whatever it is we want to figure out (morality,
etc.), a superintelligence will be more capable of succeeding. But you are
correct that we do have to work in the meantime based with what we know.
And yes, focusing on volition is one of the better systems around to do
this. But I'm not ready to say that volition has only a 0.73 percent chance
of correctness as opposed to Christianity's 0.0001 chance. I think the fact
that other systems so frequently reduce to a low denomenator of volition
putts the odds of its correctness *much* higher, though I am, of course,
just guessing.
> There is nothing wrong with discussing morality; you need some system
> by which to decide what is right and wrong. Otherwise you think that
> everything is right or, less often seen, everything is wrong.
> Discussing the morality of an SIFAI is silliness, though. Beyond the
> level of human level AI, all you can do is make guesses. Some guesses
> may turn out to be better than others, but they're still all just
> guesses.
Of course. This silliness is what I was trying to show. I obviously don't
know what things are going to look like post-Singularity -- there had
certainly better be more going on that speeding trucks. But I do think it
is silly to apply a transhuman characteristic to a Friendly SI (such as
total understanding of human cognition) and then worry about it being
applied in a mundanely human way ("manipulation"). To have an exhaustive
understanding of how the mind works is to be able to completely avoid
whatever the SI or the human might define as manipulation. It is also the
ability to anticipate a human's desire to avoid conditions like wireheading.
In short, greater intelligence and understanding are the keyn to accurately
mapping out the boundaries of volition.
--Mitch Howe
This archive was generated by hypermail 2.1.5 : Wed Jul 17 2013 - 04:00:41 MDT