From: Russell Wallace (russell.wallace@gmail.com)
Date: Sat Jul 23 2005 - 15:16:26 MDT
On 7/23/05, Chris Capel <pdf23ds@gmail.com> wrote:
> On 7/23/05, Russell Wallace <russell.wallace@gmail.com> wrote:
> > The deadliest flaw in CV (actually it's far worse than merely deadly,
> > as will be seen) is that it's still chasing the ghost of
> > universe-dictated morality, that in simpler form was the source of
> > Eliezer's and my first subjunctive planet kills.
>
> What were those?
Gung-ho plans to develop AI without realizing the need for Friendliness.
> "Guiding light" here being the overall convergence of volition? So
> you're claiming that volition wouldn't really converge?
I'm using "guiding light" as a metaphor for morality - to put things
in colder terms, volition will converge, but on a state with zero or
negative rather than positive utility.
> I really don't understand this. What axiom system, and how is it
> K-strategist? Grasping at your analogy, I guess that you mean to say
> that human society has tended toward power-grasping, power
> conglomeration, and that CV would extrapolate this tendency and
> cause--who?--to be placed in absolute power over everyone else.
Probably not one individual, but the Collective. True Communism at
last. (Watch 'Zardoz', it gets the atmosphere nicely. "We want to die,
what's the trick?")
> Doesn't this idea seem to go against the idea of CV giving each
> sentience equal weight (or perhaps weight corresponding to their level
> of awareness) in determining what the CV of humanity is? Or maybe you
> mean to say that each individual human, when they gain knowledge and
> awareness, tend to lust for power? But I think that most power-hungry
> people are abberant in this sense. In any case, there's a fundamental
> incoherence in trying to find the convergence in two people's desire
> to hold power over each other. Because each's idea is that they
> themself are the one in power, and so the intersection of those two
> desires is very small.
You would think so from calculating the abstract logic, but look at
what people empirically vote for even when they themselves are at the
bottom of the heap.
> Then again, the effects of gravity on a huge
> number of individual particles gives rise to planets and suns. Each
> particle in a planet attracts each other particle, yet the effect
> doesn't cancel out completely and lead to no gravity. In sum, it
> causes large amounts of pressure to be exerted on the core of the
> planet. Perhaps this is a good metaphor for the CV dystopia you have
> in mind?
Maybe so.
> I'm less trying to criticize your post than to give you food for
> thought, so you might perhaps be able to make clearer the exact nature
> of the danger you see.
Thanks! You make some good points in the rest of your post - I'm going
to have to reread them a few times to really absorb them and form an
opinion, so I'll postpone further comment for the moment.
- Russell
This archive was generated by hypermail 2.1.5 : Wed Jul 17 2013 - 04:00:51 MDT