Re: [sl4] Long-term goals (was Re: ... Why It Wants Out)

From: Martin Sawitzki (
Date: Mon Jun 30 2008 - 06:52:17 MDT

> Tim writes
> > By the way, if the AI has any long-term goals, then it will want to
> > preserve its own integrity in order to preserve those goals. Although
> > "preserve its own integrity" is a good enough example for the issue at
> > hand, it's not something you'd really need to put in there explicitly.

Strong sense of identity and self-modifying are somewhat contradictions. (similar to drugs that change the personality) That might even lead to the point that a self-aware AI does not want to modify itself to preserve its current state of being.

Likewise a friendly AI might be aware that the next-level AI it's supposed to develop might not be guaranteed friendly and therefore refuse to develop it at all.



[obligatory new poster and foreign speaker disclaimer]

Ist Ihr Browser Vista-kompatibel? Jetzt die neuesten 
Browser-Versionen downloaden:
GMX startet Hier findest Du Leute mit Deinen Interessen!
Jetzt dabei sein:

This archive was generated by hypermail 2.1.5 : Wed Jul 17 2013 - 04:01:03 MDT