Re: AGI Prototying Project

From: Tennessee Leeuwenburg (
Date: Mon Feb 21 2005 - 16:10:56 MST

Hash: SHA1

"1) Skynet becomes self-aware and eats us"

|> I am only worried about (1). I can imagine (3) happening, but I
|> don't object to it. Survival of the fittest is how I got here,
|> and damned if I'm going to starve to death for the sake of some
|> rats. I think it's fair enough to apply the same standard to
|> something smarter and higher on the food chain.
| I used to think that too, until I thought about it a bit more
| clearly and realized that the end point of evolution would not be
| sentient.


If the end point of evolutions is not sentient we are screwed, if it
is sentient we are safe, subject on both sides to the vaguaries of
horizon problems. This is a truism if you believe that all
evolutionary paths are eventually explored. Evolution is not a
circumventable process, we can only do our best to build a fittest
organism which is interesting rather than not.

| Let survival of the fittest run to its ultimate conclusion and
| you'll have something that might be considered intelligent - at
| least, it'll be better than any human at solving some types of
| engineering problems, for example - but there will be nothing it
| will be like to be that thing. There'll be a universe full of
| marvellously intricate, optimally self-replicating nanomachinery -
| and nobody there to make use of it.
| Still happy with that?

I don't accept that there is a >5% probability of it being true that
nanomachines are the end point of evolution. My actual estimate would
be far less, closer to .05%. Just to pick numbers intuitively, of
course, yet informed by available information.

|> * We should realise that evolution can be made to work for us by
|> building an AGI ecosystem, and thus forcing the AGI to survive
|> only by working for the common interest
| And the above is why doing things on this basis turns out to be
| such a horribly bad idea.

1) Humans are social organisms and have competed on a group basis
2) Therefore morality

Corrolory: linking the survival of an organism to moral behaviour is
the best way to ensure the continued presence of morality in future
generations of that species.

- -T
Version: GnuPG v1.2.4 (GNU/Linux)
Comment: Using GnuPG with Thunderbird -


This archive was generated by hypermail 2.1.5 : Wed Jul 17 2013 - 04:00:50 MDT