Re: Dangers of human self-modification

From: Samantha Atkins (
Date: Sat May 22 2004 - 01:19:43 MDT

On May 21, 2004, at 3:35 AM, Eliezer Yudkowsky wrote:
> Some examples of possible consequences, off the top of my head:
> You've got memories of enjoying cheeseburgers. What happens to the
> memories when the sensory substrate of recollection shifts? Are you
> going to keep the old hardware around for recollection? Will you add
> in a complex system to maintain empathy with your old self?

Surely this is rather trivializing of the entire subject? If we ever
expect to upload we have to deal with such an issue but that issue is
quite a ways out along the path of self-augmentation. The immediate
first goal is become considerably brighter, fully communicative with
one another and more emphatic/compassionate. We aren't talking a
single leap onto another substrate as a first step.

> Your old sense of taste was fine-tuned and integrated into your sense
> of pleasure and pain, happiness and disgust, by natural selection.
> Natural selection also designed everything else keyed into those
> systems. If you pick new senses, do they make sense? Does the
> pattern subtly clash with the pattern of systems already present?

This reminds me of the talk we heard at the Foresight Gathering.
Everything now is so finely balanced and wonderful how dare we think
of disruptive changes that might lose what we have. Well sorry but
the current state sucks and will lead to our deaths and likely the
death of the species. Disruptive change is already upon us and we
better learn to dance as nimbly as we can.

> Will your new sense of taste be more or less complex than your old
> sense of taste? More intense or less intense? If more intense, does
> the new sense of taste balance with a mental system that is known to
> stay sane only under ancestral conditions of environment and
> neurology? Consider the effects on humans of non-ancestral Pringles
> and chocolate cake, loads of sugar and salt and fat not present in any
> ancestral foods. Adopting a more intense taste system can have the
> same effect, if the rest of the mind isn't upgraded accordingly to
> balance with the increased intensity of sensation.

I am not interested in changing sense of taste at the moment. Whether
we stay sane in the face of real changes versus trivial made up
strawmen is a good question. But it is not a question that should make
us assume that we can afford not to change. Yes we must exercise
caution but that is true along any path leading forward.

> Maybe you would prefer to gradually grow into new tastes? What does
> the sharp discontinuity of direct self-alteration do to your sense of
> personal continuity?

That old chestnut! Wait and see.

> If the new taste sensation is more intense, do you become addicted to
> the act of self-modification for more intense sensations?

Not if I am sufficiently self-aware and bright enough to augment for
strengthened self-control.

> Can you really appreciate the long-term consequences of altering your
> mind this way? Does the new design you decided upon make any sense
> with respect to those criteria that you would use if you thought about
> the problem long enough?

I can appreciate it somewhat today and more so if my intelligence
increases in a balanced enough way. There are limits as to what a
given level of intellect can achieve in any available amount of time.
The level of intellect must change to accomplish more. The amount of
time available is largely not of our choosing. I think we have
relatively little time. The only way to make more efficient use of
it is to make ourselves as capable as we can within the limits of
staying sane enough. We don't have forever to think about how to do it
most perfectly.

> What is the long-term effect of adopting the general policy of
> eliminating old complexity that inconveniences you, and inscribing new
> complexity that seems like a good idea at the time?

I don't know what you are talking about. It is you who came up with
this (not very useful imho) example implying such a pattern. I think
it more likely we will become more complex rather than less. We may
choose to focus on something other than taste for an enhancement
candidate for a while. It is not about mere "inconvenience". It is
about overcoming the limits on our ability to do what we can to produce
a positive outcome.

> Other humans share your current taste sensations. Think of your
> awkward refusal of foods at dinner, the mainstream artistry of cooking
> you'll no longer be able to appreciate. Are you distancing yourself
> from the rest of humanity? Lest someone chime in that diversity is
> automatically good, let me add that this is one hell of a nontrivial
> decision.

Sigh. This is surely a very minor awkwardness and not exactly a high
consideration when we are talking about our very survival. Again, why
play this game around something as mundane as taste? Why pretend I
intend things I do not? If there is a subtle layer beneath this
approach I am missing then please speak more plainly. On the face of
it this is bordering on the absurd.

- samantha

This archive was generated by hypermail 2.1.5 : Wed Jul 17 2013 - 04:00:47 MDT