Re: Fighting UFAI

From: Michael Wilson (
Date: Thu Jul 21 2005 - 01:20:18 MDT

Tennessee Leeuwenburg wrote:
> One day she lands on Earth at the end of her mission. Upon opening the
> hatch, she casts her eyes first on an enormous bunch of red roses
> which have been given to her.
> "Oh", she says, "so that's what it's like".
> Has she learnt anything new about colour?

That's actually a good example of how the seemingly mysterious problem
of 'qualia' is the result of relatively poor human introspection
capability. Imagine an AGI with full self-examination and
self-modification capability performing the same experiment. The AGI
could work out exactly what bit streams its visual sensors would
produce upon seeing something red, would know exactly how its sensory
processing code would react and could in effect imagine a red rose
in its 'mind's eye' without ever needing to see something red first.
In principle, the human could use a detailed map of her brain wiring
to work out where to stick some electrodes to generate the mental
image of a red rose before she actually lays eyes on it, but a
reflectively transparent AGI can do this kind of thing with fairly
trivial effort and much greater precision and reliability.

> If you accept that she has, then qualia must be real, because she
> already knew everything that science could inform her about the
> world and about colour.

Human knowledge can't be applied to all cognitive processing in an
orthogonal sense. This should be obvious; memorising detailed
instructions on how to ride a bicycle does not immediately grant
you the ability to ride a bike competently, because you cannot
deliberatively modify you neural circuitry with an act of will.
Humans have several ways of storing information which have varying
degrees of accessibility for different tasks and cannot always be
easily translated between. By contrast a fully reflective AGI could
learn to ride a bike simply by reading the instructions (and doing
some physical modelling, and generating the appropriate code) and
can in principle apply any of its knowledge base to any cognitive
task (though in practise this may incur significant computational
efficiency penalties). It is unfortunate that the human brain's
complicated many-tiered information storage and encoding strategy
(the result of much messy evolutionary design against tight real
time performance requirements) looks sufficiently confusing from
the inside to spawn endless debates about qualia and the nature
of knowing.

 * Michael Wilson

How much free photo storage do you get? Store your holiday
snaps for FREE with Yahoo! Photos

This archive was generated by hypermail 2.1.5 : Wed Jul 17 2013 - 04:00:51 MDT