The Geddes conjecture

From: Marc Geddes (
Date: Thu Jul 14 2005 - 22:59:18 MDT

--- Ben Goertzel <> wrote:

> Marc Geddes wrote:
> > Iím positing that an unfriendly AI cannot
> self-improve
> > past a certain point (i.e itís intelligence level
> will
> > be limited by itís degree of unfriendliness). I
> posit
> > that only a friendly ai can undergo unlimited
> > recursive self-improvement.
> I find this to be a most improbable statement...

You find it 'improbable', Robin finds it 'lunatic' and
M.Wilson thinks I'm 'merely engaging in wishful
thinking' This is great! :D It shows that my theory
is actually crazy enough to be true.

Niels Bohr Quote:

> I am curious, however, how you are construing the
> sense of "Friendly" here.

I define a 'friendly' AI as one that respects volition
(seeks harmonious relationships with other sentients)
and is growth oriented (always seeks to better itself
subject to the volition-respecting restraint).

> For instance, what about a superhuman AI whose goal
> is to advance science,
> math and technology as far as possible?
> What fundamental limits on self-improvement do you
> think such an AI is going
> to encounter?
> Can you please outline your argument, using this
> science-focused AI as an
> example?

I'll try. Let me first use a bit of maths notation to
loosely define what I was actually suggesting.

Let's call this: 'The Geddes Conjecture'

The Geddes conjecture in maths terms:
Function Intelligence (Predictions) 
= Big Oh [ Function Friendliness (Utilities) ]
Intelligence I loosey define as the ability to make
accurate prediction  and friendliness I loosely
defined above as utilities which tend to promote
volition and growth (self-betterment).
A graph of the intelligence function would have IQ
along one axis, and a set of all predictions made
(mapped to a single point) on the other axis (the
graph gives the implied IQ for a given set of
A graph of the friendliness function would have some
suitable measure of volition/growth along one axis,
and the set of utilities of the goal system (mapped to
a point) along the other axis (the graph gives the
implied level of friendliness for a given set of
The Geddes conjecture then states that I think the
intelligence level of any sentient is bounded (in the
limit) by the level of friendliness of that sentient. 
Now in the case of the example you gave,  an AI whose
goal is to advance science, math and technology as far
as possible, it seems to me that such an AI might
actually become friendly in the long-run (but it
wouldn't be friendly to start with!)  Because
advancing science requires an ever increasing ability
to make accurate predictions, and according to the
Geddes conjecture, this would require adding utilities
for respecting volition.
How could unfriendly utilities be limiting the
predictive ability?  The answer I think, is that
growth is somehow connected to respecting volition. 
The process of interacting with other sentients in a
harmonious way actually helps us to grow (become
better people ourselves).  So I think growth is a
*moving towards* altruism.  This sounds vaguely
plausible.  As an observed human fact it does seem
like learning to harmoniously interact with others in
some sense makes us 'more than we are' (expands our
circle of being).
Let me try to give another reason for a link between
friendly utilities and intelligence.  What *use* is
morality?  If it doesn't have some sort of innate use
to us, it's not clear why we couldn't simply dispense
with the concept altogether.  So surely we should
assume that being moral is connected with our own
well-being somehow.  That being the case, a link
between friendly utilities and getting smarter sounds
vaguely plausible.
Note that 'harmonious relationships' (gettiing along
with others) is analogous to 'health' (which consists
of internal mental and physical states of a sentient
functioing harmoniously).  There's an analogy between
the social and personal sphere.  Could there be
something more here than just analogy?  Perhaps
hurting others adversely affects mental health in
general?  Note that as an observed fact, evil people
do tend to suffer from mental instability more than
decent folks.  
This are admittedly all rather weak sounding
arguments, buyt they are suggestive none the less. 
They do I think, move my conjecture from being
'ludicrous' to being 'vaguely plausible'  
> I might agree that there are limits to the
> intelligence of an AI embodying a
> classic human notion of "evil", if only because once
> a system becomes smart
> enough, it may inevitably understand how idiotic
> this human notion of "evil"
> is.  The emotional complexes underlying human evil
> and destructiveness may
> well be tied to limitations in intelligence.
> However, as Eli has ably pointed out over the years
> (following in the
> footsteps of many prior futurists and sf authors),
> there are many ways for a
> superhuman AI's goal system to threaten human life,
> even if that AI has no
> "evil" in it.
> -- Ben G
I don't think 'evil' is an active agency like
'friendliness' is.  I define evil simply to be an
absence of friendliness in a sentient.  If a
superhuman AI threatens human life, it's evil by my
THE BRAIN is wider than the sky,  
  For, put them side by side,  
The one the other will include  
  With ease, and you beside. 
-Emily Dickinson
'The brain is wider than the sky'
Please visit my web-site:
Mathematics, Mind and Matter
Send instant messages to your online friends 

This archive was generated by hypermail 2.1.5 : Wed Jul 17 2013 - 04:00:51 MDT