From: Murphy, Tommy (TommyMurphy@livingstonintl.com)
Date: Fri Feb 03 2006 - 13:58:01 MST
We will all certainly be "killed" by an AGI that merely appears to be friendly? What would be the purpose of destroying what it could simply appropriate? Assuming it's got us all fooled, as it continues to improve itself, can we not assume it's ability to manipulate us would improve accordingly?
-----Original Message-----
From: owner-sl4@sl4.org [mailto:owner-sl4@sl4.org]On Behalf Of Jeff Herrlich
Sent: Friday, February 03, 2006 3:12 PM
To: sl4@sl4.org
Subject: Re: AGI Reproduction?
You're assuming an observer-centric goal system (and no, that still
wouldn't help us - why would it?).
Hi Peter,
If there is only one non-friendly AGI that values its own life (goal-satisfaction) above all others, we will all certainly be killed once it acquires the means to do so. If multiple, comparably powerful AGIs are created (using the original human-coded software). They will each value there own survival above all others. Under these situations, it may be less likely that one AGI would attack another AGI. By virtue of this, it may be less likely that an AGI would attempt to exterminate humanity simply because humanity might still serve as a valuable resource, at least for a while. Or, it may decide to restructure its own goal system in a way that did not include human extermination. I didn't say it would be pretty, I only said thi! s would improve the chances of (at least some) humans surviving, in one form or another (uploads?)
Jeff
Peter de Blanc <peter.deblanc@verizon.net> wrote:
On Fri, 2006-02-03 at 08:42 -0800, Jeff Herrlich wrote:
> As a fallback strategy, the first *apparently* friendly AGI
> should be duplicated as quickly as possible. Although the first AGI
> may appear friendly or benign, it may not actually be so (obviously),
> and may be patiently waiting until adequate power and control have
> been acquired. If it is not friendly and is concerned only with its
> own survival, the existence of other comparably powerful AGIs could
> somewhat alter the strategic field in favor of the survival of at
> least some humans.
You'! re assuming an observer-centric goal system (and no, that still
wouldn't help us - why would it?).
__________________________________________________
Do You Yahoo!?
Tired of spam? Yahoo! Mail has the best spam protection around
http://mail.yahoo.com
*******************************************
The information contained in this e-mail message may
contain privileged and confidential information.
If you are not the intended recipient, you are
hereby notified that any review, dissemination,
distribution or duplication of this communication
is strictly prohibited. If you have received this
message in error, please notify the sender by return
e-mail, delete this message and destroy any copies.
Internet e- mail is not guaranteed to be secure or
error-free. Messages could be intercepted, corrupted,
lost, arrive late or contain viruses.
The sender will not be liable for
these risks.
*******************************************
Ce message electronique pourrait contenir des
informations privilegiees et confidentielles. Si vous
n'en etes pas le recipiendaire prevu, nous vous
signalons qu'il est strictement interdit d'examiner,
de diffuser, de distribuer et de reproduire le
present message. Si vous l'avez recu par erreur,
veuillez prevenir l'expediteur par courriel, puis
effacer ce message et en detruire toute copie.
Le courrier electronique n'est pas garanti
securitaire ni exempt d'erreurs. Les messages
pourraient etre interceptes, corrompus, egares,
retardes ou contamines par des virus.
L'expéditeur n'est pas
responsable de ces risques .
This archive was generated by hypermail 2.1.5 : Wed Jul 17 2013 - 04:00:55 MDT