guaranteeing friendliness (was Re: 180 IQ idiot)

From: Chris Capel (
Date: Fri Nov 25 2005 - 19:48:02 MST

On 11/24/05, T <> wrote:
> Phillip Huggan wrote:
> > Weather is not climate. Climate temperature changes are non-linear
> > but not chaotic, even if we don't know 100% of the factors involved.
> > When CO2 levels have historically risen, so have global temps. This
> > line of reasoning applied in the context of a singularity would
> > suggest it is okay to turn on an AGI of uncertain friendliness.

> I don't understand why this has anything to do with the Singularity. Are
> you suggesting that there is a similar correlation between Intelligence
> and Friendliness that you can identify for us?

My understanding of where Phillip was going is that the techniques
used to provide some measure of certainty about the direction the
world climate is moving, if developed, would have a lot of
resemblances to the sort of guarantee that Eliezer is trying to
develop with friendliness. It would have to take a very complicated
system, with lots of unknowns, and try to derive statistically certain
conclusions about the long-term direction of those systems that we can
be justified in believing are valid and immune to unintentional
manipulation by the modeler and verifiable in some formal capacity.

Chris Capel

"What is it like to be a bat? What is it like to bat a bee? What is it
like to be a bee being batted? What is it like to be a batted bee?"
-- The Mind's I (Hofstadter, Dennet)

This archive was generated by hypermail 2.1.5 : Wed Jul 17 2013 - 04:00:53 MDT