From: Mike (mikew12345@cox.net)
Date: Fri Jun 04 2004 - 15:47:41 MDT
Eli,
Before you turn this superAI loose on humanity, how about letting it run
just *your* life for maybe a few months, and see how it looks then.
Mike W.
> -----Original Message-----
> From: owner-sl4@sl4.org [mailto:owner-sl4@sl4.org] On Behalf
> Of Eliezer Yudkowsky
> Sent: Friday, June 04, 2004 12:51 PM
> To: sl4@sl4.org
> Subject: Volition versus decision
>
>
> Ben Goertzel wrote:
> >>
> >> But what if you are horrified by the consequences of this
> choice in
> >> 30 years? How much thought did you put into this before
> deciding to
> >> make it the eternal law of the human species? Are you so
> confident
> >> in the power of your moral reasoning, oh modest Ben?
> >
> > The basic principle of democracy (which is based on choice, not
> > extrapolated volition) has been around a long time, I don't
> claim to
> > have invented it. Modern democracies work, pretty much, by letting
> > people do what they want except when their wants collide,
> in which case
> > a hodge-podge of moral principles and pragmatic
> considerations are used
> > for conflict resolution. I don't always like the results
> but I find
> > them sorta acceptable, preferable to what I see coming out of other
> > governmental systems on Earth today or historically.
> >
> > Extrapolated volition based FAI is as much like fascism as
> it is like
> > democracy --- fascism being a political philosophy in which
> the will of
> > the individual is, in principle, subordinate to the will of
> the state
> > which is understood to represent the greater good. In fact
> the will of
> > the state, in fascism, is supposed to represent what people
> SHOULD want,
> > and in some fascist theorists' writings, what the WOULD
> want if they
> > were freed of the damaging illusions plaguing their minds.
> >
> > I am confident that the majority of Earthlings would vote for
> > democracy
> > over extrapolated-volition-based AI-control. But of
> course, this is a
> > silly statements, since "voting for democracy" presumes
> that democracy
> > already exists ;-) You could argue that, even if people
> would vote for
> > democracy now, their future selves wouldn't want them to...
> etc. etc.
>
> But do you, Ben, let alone the majority of modern-day
> Earthlings, have the
> ability to correctly weigh the risks and benefits of direct democracy
> controlling a superintelligence (EEEK!) versus collective
> volition? I
> don't. I could be wrong, and the collective volition could
> turn over the
> power to the UN General Assembly (EEEEEK!).
>
> It's people's lack of moral caution or fear of screwing up, their
> horrifying readiness to trust themselves with power, that
> causes me to turn
> the choice of meta-dynamic over to a collective volition
> (which is a form
> of humane transhuman intelligence, and one that structurally
> cares very
> strongly about our expected first-person perspectives on the
> issue). I
> don't trust myself. Nor do I trust other humans. From my studies of
> cognitive psychology I know in too much detail how little
> grasp the human
> mind has upon reality, how ready people are to make decisions
> on the basis
> of what sounds like a good idea at the time and how unrelated
> that is to
> their actual future experience and reactions. Just the
> prospect of humans
> deciding on the basis of their half-a-paragraph *description* of the
> outcomes, let alone their description of a human *prediction* of the
> outcomes, makes me want to flee screaming into the night.
>
> --
> Eliezer S. Yudkowsky http://intelligence.org/
> Research Fellow, Singularity Institute for Artificial Intelligence
>
This archive was generated by hypermail 2.1.5 : Wed Jul 17 2013 - 04:00:47 MDT