From: Marc Geddes (marc_geddes@yahoo.co.nz)
Date: Fri Jun 04 2004 - 01:36:18 MDT
--- Eliezer Yudkowsky <sentience@pobox.com> wrote: >
Norm Wilson wrote:
> > Marc Geddes wrote:
> >
> >
> >> Collective Volition is drawing on the
> extrapolation of the volitions
> >> of all sentient persons. But if the FAI
> establishes that it qualifies
> >> as a 'Sentient person', then the FAI has to
> include it's own volition
> >> in it's calculations.
> >
> > My understanding of Eliezer's paper was that the
> FAI would extrapolate
> > the collective volition of "humans" specifically,
> as opposed to the
> > volition of sentient beings in general, with the
> underlying premises
> > being:
> >
> > (1) Human behavior is affected by an abstract
> concept of morality, AND
> >
> > (2) Humans who know more, are smarter, etc.,
> behave in manners that are
> > more consistent with that abstract concept.
> >
> > To base collective volition on intelligence alone
> would ignore the
> > significance of the first premise.
>
> Correct, with the notation that this is the *initial
> dynamic*, which would
> presumably be intended to come up with a better
> definition of sentience -
> perhaps the successor dynamic will include
> chimpanzees.
>
> --
> Eliezer S. Yudkowsky
> http://intelligence.org/
> Research Fellow, Singularity Institute for
> Artificial Intelligence
My point was that there is a fuzziness in the initial
defintion of 'sentience' which could balloon into a
fatal infinite regress if the FAI itself becomes
sentient.
Why should the cognitive proccesses of the FAI itself
be except from the abstract conception of 'morality'?
It might be plausible to exclude the FAI if the FAI
itself is not sentient, but if the FAI is sentient,
then you have a problem.
=====
"Live Free or Die, Death is not the Worst of Evils."
- Gen. John Stark
"The Universe...or nothing!"
-H.G.Wells
Please visit my web-sites.
Science-Fiction and Fantasy: http://www.prometheuscrack.com
Science, A.I, Maths : http://www.riemannai.org
http://personals.yahoo.com.au - Yahoo! Personals
New people, new possibilities. FREE for a limited time.
This archive was generated by hypermail 2.1.5 : Tue Feb 21 2006 - 04:22:38 MST