From: Philip Sutton (Philip.Sutton@green-innovations.asn.au)
Date: Sun May 23 2004 - 18:23:47 MDT
The issue of "messing with source code" arises with genetic
engineering.
The problem I think stems from failing to distinguish consciously
enough between directly-determined versus emergent behaviours.
If a system has a module with a stand alone function and the source
code for that module is isolatable then we should be able to upgrade
the module without mucking up the rest of the system. But if the
property we want to change is the result of subtle interplay between
many systems and may have emerged in the first place as a bit of
evolutionary serendipity, *and* the system we are dealing with is
immensely complicated, then it will be most likely be exceptionally
difficult to determine the causation of the emergent behaviour and it
might be near enough to impossible to change the behaviour of interest
without stuffing up other needed behaviours.
If people want to upload and then enhance themselves more than
trivially then the science that allows uploading might *first* have to work
out how to *emulate* humans with a code structure that is more
amenable to upgrading than the coding method developed via historical
biological evolution.
So the first uploading technology might well be one where self-
modification is effectively *not* possible (due to the massive risk of
serious failure). The option for personal self-modification might have to
await the capacity to reconfigure the code for an uploaded human using
a code system designed specifically to maximally facilitate successful
self-modification.
My guess if that the development of a coding system to facilitate
successful self-modification will be cracked first for AGIs.
As an aside, I think that anyone who thinks they can get humanity into
the drivers seat of the Singularity before AGIs get there will be sorely
disappointed. So working out how ordinary biological humans can live
comfortably with powerful (most likely sefl-modifiying) AGI is the first
issue to crack before worrying about human uploading.
Cheers, Philip
This archive was generated by hypermail 2.1.5 : Tue Feb 21 2006 - 04:22:36 MST