RE: SIAI's flawed friendliness analysis

From: Ben Goertzel (ben@goertzel.org)
Date: Thu May 29 2003 - 19:30:34 MDT


Volition is a component of "deep, real human happiness" but I don't think
it's the only component.

One could construct extreme cases of human minds that were strongly
self-determined yet were morally and aesthetically repugnant to all of us...

ben g

> Ben Goertzel wrote:
> > Ok, I concede -- if that is really Bill's definition of
> happiness, then of
> > course a superintelligent AI that is rigorously goal-driven and is given
> > this as a goal will create something like euphoride (cf "The
> Humanoids") or
> > millions of micromachined mannequins.
> >
> > Detailed specification of a richer definition of "human happiness", as
> > hinted at in The Humanoids and its sequel novels, is an interesting and
> > nontrivial problem...
>
> The richer definition is, I think, volition; maximize self-determination,
> minimize unexpected regret.
>
> --
> Eliezer S. Yudkowsky http://intelligence.org/
> Research Fellow, Singularity Institute for Artificial Intelligence
>



This archive was generated by hypermail 2.1.5 : Wed Jul 17 2013 - 04:00:42 MDT