Re: FAI (aka 'Reality Hacking') - A list of all my proposed guesses (aka 'hacks')

From: Marc Geddes (
Date: Sat Jan 29 2005 - 00:06:07 MST

 --- Russell Wallace <>

> > (10) General intelligence will turn out to
> subsume
> > the Friendliness problem. So morality will turn
> out
> > to be inseparable from general intelligence after
> all.
> I hope nobody who's actually working on AI believes
> this. If anyone
> here is and does, speak up and I'll try to talk you
> out of it ^.^

>From the point of safety it would actually be much
better for us all if (10) was true, because then the
difficulty of FAI would be no greater than the
difficulty of AGI.

As I understand it, Eliezer originally believed (10)
but then he changed his mind and now thinks that the
Friendliness problem is different to the problem of
general intelligence. I believed him for a while.

However my reason for finally going back to (10) is
related to my ideas for Objective Morality and the
abolition of the facts/value distinction. The idea
that Friendliness is seperate from general
intelligence just screwed everything up.

I just don't buy the idea that there's there's this
wonderful theory of general intelligence on the one
hand and then on the other we're stuck with trying to
choose some mysterious 'invariant' which has to be
choosen by us to be 'nice' and plugged in by hand. I
say: Baloney!

Better to try to change the model of general
intelligence to fit in Friendliness. And here's the
beginnings of my idea for that:

I simply looked at the 4 levels of intelligence
described in LOGI (Modalities, Concepts, Thoughts,
Deliberations) and generalized by analogy to obtain 4
new levels of intelligence (Snap-Shots, Wisdoms,
Relations and Expansions) which fully encompass

The Snap-Shot level is the intelligence looking at
it's own internal states (representing it's volition).
 The Wisdoms level is stringing together the internal
states of an intelligence (i.e the intelligence is
trying to extrapolate it's own volition). The
Relations level is looking at the state of the
intelligence relative to the states of various things
in the external world. And The Expansions level is
stringing together the various relationships between
states to maximize utility.


Find local movie times and trailers on Yahoo! Movies.

This archive was generated by hypermail 2.1.5 : Wed Jul 17 2013 - 04:00:50 MDT