From: Tennessee Leeuwenburg (tennessee@tennessee.id.au)
Date: Tue Feb 22 2005 - 17:15:22 MST
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
| Perhaps we were talking past each other then, because that's
| exactly what I'm advocating; "circumventing evolution" is a term
| I'd use to describe this, but if you want to use a different term
| then I don't have a problem with that.
I think there is something we disagree about. You seem to be worried
that Friendliness will *evolve into* a paperclip machine (or
whatever). Firstly, I think anything smart enough to deserve the name
will also have the intelligence to interpret any goal it recieves in a
non-naive way. Secondly, I don't think we need to try to hard-wire in
some goal that cannot be removed - we just need to look at the system
dynamics in such a way that evolutionary pressure will *encourage*
Friendliness rather than a paperclip machine.
|> It's not about 'escaping' evolutionary pressure - that is like
|> saying that everything would be easier if the laws of the
|> universe were different. Survival of the fittest /will/ happen.
|> We need to ensure that something interesting is the fittest
|> thing.
|
| Okay, how do you propose to do this?
Well, I think designing in a reasonable sense of self-preservation of
identity into the AGI and using a reproductive model will mostly take
care of things. No individual intelligence that values its own
existence is going to wilfully construct an offspring which will
guarantee its own demise, unless perhaps it regards its own demise as
inevitable, and can fulfil some other important goals by doing so. It
is a short step from self-preservation to morality.
|> Oh, and I read some of the mail archive, and ended up on a page
|> by Ben Goetzel talking about "Solomonoff Induction". Is anyone
|> interested in my pointing out some implied assumptions I found in
|> there, or has this already been done to death?
|
|
| I'd be interested in hearing them.
I'll get round to them... real life keeps getting in the way. Watch
this space...
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.2.4 (GNU/Linux)
Comment: Using GnuPG with Thunderbird - http://enigmail.mozdev.org
iD8DBQFCG8sZFp/Peux6TnIRAvO4AJ0eAfYeXxczxtz7hxBclWDk4riztACfdb/O
7PXArhzptRinE1Eu3ICTtW4=
=dUh9
-----END PGP SIGNATURE-----
This archive was generated by hypermail 2.1.5 : Wed Jul 17 2013 - 04:00:50 MDT