From: Matt Mahoney (matmahoney@yahoo.com)
Date: Wed Dec 02 2009 - 19:22:06 MST
Frank Adamek wrote:
> Unless of course my volition is to not be uploaded in certain ways. But more critically, your argument justifying the AI's behavior equally justifies a few other things. Which a slight alteration: "By killing both of you instantly and destroying your information, neither copy of you experiences any pain or fear, so your volition is satisfied, assuming ve was also programmed to satisfy your volition."
I agree. Which raises the question. Why build AI at all?
-- Matt Mahoney, matmahoney@yahoo.com
________________________________
From: Frank Adamek <f.adamek@yahoo.com>
To: sl4@sl4.org
Sent: Wed, December 2, 2009 9:08:21 PM
Subject: Re: [sl4] Irrational motivations (was [sl4] A Thought Experiment: Thou Shalt Not Bear False Witness)
Unless of course my volition is to not be uploaded in certain ways. But more critically, your argument justifying the AI's behavior equally justifies a few other things. Which a slight alteration: "By killing both of you instantly and destroying your information, neither copy of you experiences any pain or fear, so your volition is satisfied, assuming ve was also programmed to satisfy your volition."
-Frank Adamek
--- On Wed, 12/2/09, Matt Mahoney <matmahoney@yahoo.com> wrote:
What is irrational or not is irrelevant to my argument. The AGI has been programmed to be truthful to humans. Therefore ve
> must upgrade your intelligence to ver level. Ve has a model of your mind, just like you have a model of your dog's mind that enables you to predict its actions. That model will maintain your evolved fear of death, because altering your memories, goals, or behavior would be dishonest. By killing you instantly, neither copy of you experiences any pain or fear, so your volition is satisfied, assuming ve was also programmed to satisfy your volition.
>
>Need I point out that giving simple top level goals to Jupiter brains like honesty, or making people happy, or making paper clips can have unintended consequences?
>
>-- Matt Mahoney, matmahoney@yahoo.com
>
This archive was generated by hypermail 2.1.5 : Wed Jul 17 2013 - 04:01:05 MDT