Re: guaranteeing friendliness

From: Jef Allbright (
Date: Sat Nov 26 2005 - 11:17:48 MST

On 11/26/05, Ben Goertzel <> wrote:
> > Let me rephrase: I can imagine that over the next ten thousand
> > subjective years, it would be possible, desirable, and necessary that
> > you should, one change at a time, grow into a mind of which it was
> > possible to prove that future self-modifications obeyed some invariant
> > or other.
> But this rephrasing does not explicitly address the question of
> whether this "one change at a time" modification can be done in such a
> way as to preserve subjective "continuity of consciousness."
> I don't think this is a criticall important question, but I find it an
> interesting one.

Ben, you're framing the question as if there's an objective basis for
subjective "continuity of consciousness."

Consider the case of a human who has suffered brain damage such that
their friends notice a distinct change in personality. Whether the
individual reports feeling a gap depends on his/her capabilities and
available information for comparing the two states, and the feeling of
a gap will only occur when some kind of comparison is being performed
by the mind-system, not in some way as a direct result of "lack of
continuity of consciousness."

Consider the case of an extremely (objectively) aware being, able to
discern differences in its own state from moment to moment, again as a
result of its capabilities and available information. In this case,
increased awareness of objective state information would make it
increasingly clear that there is no essential continuity of
consciousness, but it would still be reasonable to talk about
continuity of identity and all that entails.

- Jef

This archive was generated by hypermail 2.1.5 : Wed Jul 17 2013 - 04:00:53 MDT