From: Bill Hibbard (test@demedici.ssec.wisc.edu)
Date: Thu Jun 08 2006 - 11:58:29 MDT
On Thu, 8 Jun 2006, Robin Lee Powell wrote:
> On Thu, Jun 08, 2006 at 03:50:34AM -0500, Bill Hibbard wrote:
> > On Wed, 7 Jun 2006, Robin Lee Powell wrote:
> >
> > > On Wed, Jun 07, 2006 at 12:24:55PM -0500, Bill Hibbard wrote:
> > > > If you think RL can succeed at intelligence but must fail at
> > > > friendliness, but just want to demonstrate it for a specific
> > > > example, then use a scenario in which:
> > > >
> > > > 1. The SI recognizes humans and their emotions as accurately
> > > > as any human, and continually relearns that recognition as
> > > > humans evolve (for example, to become SIs themselves).
> > > >
> > > > 2. The SI values people after death at the maximally unhappy
> > > > value, in order to avoid motivating the SI to kill unhappy
> > > > people.
> > > >
> > > > 3. The SI combines the happiness of many people in a way
> > > > (such as by averaging) that does not motivate a simple
> > > > numerical increase (or decrease) in the number of people.
> > > >
> > > > 4. The SI weights unhappiness stronger than happiness, so
> > > > that it focuses it efforts on helping unhappy people.
> > > >
> > > > 5. The SI develops models of all humans and what produces
> > > > long-term happiness in each of them.
> > > >
> > > > 6. The SI develops models of the interactions among humans
> > > > and how these interactions affect the happiness of each.
> > >
> > > Have you read The Metamorphosis Of Prime Intellect?
> > >
> > > The scenario above immediately and obviously falls to the "I've
> > > figured out where human's pleasure centers are; I'll just leave
> > > them on" failure.
> >
> > I address this issue in my 2005 on-line paper:
> >
> > The Ethics and Politics of Super-Intelligent Machines
> > http://www.ssec.wisc.edu/~billh/g/SI_ethics_politics.doc
> >
> > There exists a form of happiness that is not drug-induced ecstasy.
>
> I read all of the paragraphs with the word "happiness" in them. I
> see nothing that addresses this issue even in the slightest.
My paper discusses the difference between hedonic and
eudiamonic from the reference:
Ryan, R.M. and Deci, E.L. 2001. On happiness and human potentials:
A review of research on hedonic and eudiamonic well-being. Annual
Review of Psychology 52, 141-166.
and makes the point that the SI should use "expression of
long-term life satisfaction rather than immediate pleasure."
Here's a way to think about it. From your post you clearly
would not do anything to permanently turn on human pleasure
centers. This is based on your recogition of human
expressions of happiness and your internal model of human
mental procoesses and what makes them happy. Given that the
SI will have as accurate recognition of expressions of
happiness as you (my point 1) and as good an internal model
of what makes humans happy as you (my points 5 and 6), then
why would the SI do something to humans that you can clearly
see they would not want?
This archive was generated by hypermail 2.1.5 : Wed Jul 17 2013 - 04:00:56 MDT