From: Ben Goertzel (ben@goertzel.org)
Date: Sat Jan 10 2004 - 09:07:06 MST
You're arguing that death can be either good or bad, depending on who dies
and what replaces them.
That sounds to me like you're agreeing with Metaqualia that death in itself
is morally neutral.
I.e., "can be morally good or morally bad" ==> "is morally neutral"
[If you argue that it's bad more often than it's good, then you're arguing
against neutrality...]
Also, I think that by "killing off everyone" he means "killing off all
humans" not "killing off all beings and leaving the universe a completely
blank void". So the scenario he was hypothesizing was one in which a
superhuman, highly moral AI came to a rational decision that killing off all
humans is the best thing to do. Presumably because this action would
indirectly lead to some other benefit.
-- Ben G
> -----Original Message-----
> From: owner-sl4@sl4.org [mailto:owner-sl4@sl4.org]On Behalf Of Mark Waser
> Sent: Saturday, January 10, 2004 10:58 AM
> To: sl4@sl4.org
> Subject: Re: Friendliness and blank-slate goal bootstrap
>
>
> >> Death is morally neutral. Only suffering is evil.
>
> I will argue this to my last breath. Death is NOT morally neutral. Death
> is the END of something and that something is either good or bad. I would
> argue that A death is as good or as bad as the opposite of the
> thing that it
> ends - - tempered by the good or the bad of the thing that replaces it. I
> would also posit that death is innately minorly bad when it reduces
> diversity. Given this premise, I would strongly argue that there
> is no way
> that killing off everyone could possibly be a good idea. Like abortion in
> many instances, it might be the best idea in a series of bad
> trade-offs but
> it will never be a good idea.
>
> > I take the moral law I have chosen to its logical extreme, and
> won't take
> it
> > back when it starts feeling uncomfortable.
>
> You sound like a hardcore fanatic. Maybe your feelings of discomfort are
> telling you something valuable. Looks to me like a case of righteousness
> over integrity.
>
> Mark
>
> ----- Original Message -----
> From: "Metaqualia" <metaqualia@mynichi.com>
> To: <sl4@sl4.org>
> Sent: Saturday, January 10, 2004 2:06 AM
> Subject: Re: Friendliness and blank-slate goal bootstrap
>
>
> > > Be very careful here! The easiest way to reduce undesirable
> qualia is to
> > > kill off everyone who has the potential for experiencing them.
> >
> > I want someone who is superintelligent, and that takes my basic premises
> as
> > temporary truths, and who recursively improves himself, and who
> understands
> > qualia in and out, to decide whether everyone should be killed. If you
> > consider this eventuality (global extermination) and rule it
> out based on
> > your current beliefs and intelligence, you are not being modest in front
> of
> > massive superintelligence. I do not rule out that killing everyone off
> could
> > be a good idea. Death is morally neutral. Only suffering is evil. Of
> course
> > a transhuman ai could do better than that by keeping everyone alive and
> > happy, which will reduce negative qualia and also create huge positive
> ones,
> > so I do have good hopes that we won't be killed. What if the
> universe was
> > really an evil machine and there was no way of reversing this
> truth? What
> > if, in every process you care to imagine, all interpretations of the
> process
> > in which conscious observers were contained, were real to these
> observers
> > just like the physical world is real to us? What if there
> existed infinite
> > hells where ultrasentient ultrasensitive beings were kept
> enslaved without
> > the possibility to die? Is this not one universe that can be
> simulated and
> > by virtue of this interpreted out of any sufficiently complex
> process (or
> > simpler processes: read moravec's simulation/consciousness/existence)?
> >
> > I take the moral law I have chosen to its logical extreme, and
> won't take
> it
> > back when it starts feeling uncomfortable. If the universe is
> evil overall
> > and unfixable, it must be destroyed together with everything it
> contains.
> > I'd need very good proof of this obviously but i do not discount the
> > possibility.
> >
> > > It seems to me that a person's method for determining the desireable
> > > morality is based partially on instincts, partially on training, and
> >
> > we are talking about different things, i have answered this previously.
> >
> > > ... are you sure about that? Just how heavily do you want the AI to
> > > weigh it's self interest? Do you want it to be able to justify
> >
> > its self interest? at zero obviously, other than the fact that the
> universe
> > is likely to contain a lot more positive qualia than negative
> ones if the
> > moral transhuman AI stays alive, so in the end its own survival would be
> > more important than the survival of humans, if you consider the million
> > worlds with biologically evolved beings that may be out there
> and in need
> of
> > salvation. So at a certain point the best it could do morally to work
> toward
> > the goals we have agreed could be exactly exterminating humans.
> >
> > > >Remember, friendliness isn't Friendliness. The former would involve
> > something
> > > >like making an AI friend, the latter is nothing like it.
> Where he says
> > > >"Friendliness should be the supergoal" it means something more like
> > "Whatever
> > > >is really right should be the supergoal". Friendliness is an external
> >
> > Is Friendliness creating a machine that wouldn't do something
> we wouldn't
> > like? Or is Friendliness creating a machine that wouldn't do
> something we
> > wouldn't like if we were as intelligent and altruistic as it is?
> >
> > > This is assuming that "right" has some absolute meaning, but this is
> > > only true in the context of a certain set of axioms (call them
> >
> > I am proposing qualia as universal parameters to which every
> sentient (at
> > least evolved ones) can relate. That was the whole purpose, so we don't
> get
> > into this "relativity" argument which seems to justify things that I am
> not
> > ready to accept because they just feel very wrong at a level of
> > introspection that is as close as it could be to reality and cannot be
> > further decomposed (negative qualia).
> >
> >
> > mq
> >
> >
> >
>
>
>
This archive was generated by hypermail 2.1.5 : Wed Jul 17 2013 - 04:00:44 MDT