Re: Friendliness and blank-slate goal bootstrap

From: Metaqualia (
Date: Mon Oct 06 2003 - 11:38:27 MDT

> individuals. For example a machine might not have the concept of pain

my point exactly. qualia are an additional piece of the puzzle, modern
physics has holes.

> To even complicate it further, humans at times can enjoy pain and fear,

I think that this confusion arises because we use the same words to mean
different things.

> roller coasters, horror movies, S&M. The AI would have to undertake a

Compare horror movies and being followed by a mob of 25 people trying to
beat you up (I tried both).
We call the horror movies scary and the other experience scary, but this is
a mistake.

They do not feel the same. What I feel mainly when I watch a horror movie is
curiosity. It is our curiosity for the dangerous, that is satisfied when
watching a horror movie. Now that I am not curious anymore, I no longer
watch them. People trying to break your legs, that is terror, you wouldn't
want to watch a movie that really terrorized you.

And S&M. I cannot speak for the whole community since everyone has their
particular kink or comfort zone or interpretation of it, but "pain" is not
what it is about. There is a whole range of unique mental states involved in
consensual power exchange, and if they weren't extremely pleasurable, nobody
would engage in such an activity.

Roller coasters: this I really don't understand. Very rapid acceleration and
deceleration have physiological effects that cannot be controlled even
though you know that the ride is safe. Roller coasters scare the hell out of
me and I hate them. I suppose people who enjoy coasters are not terrified
but enjoy it, so this does not count as real fear.

Tentative morale of the story: real end-of-the-chain qualia-grade pain is
never pleasurable. Somewhere between the sensory input [situation commonly
perceived as painful] and the triggering of the appropriate emotion,
something can go wrong, for whatever reason, and things that should be
painful end up feeling pretty good, thank your parents and move on :) Or you
have a mix of both but the pleasure is intense enough to overcome the pain.

One more in favour of the "figure out qualia" subgoal : we want an AI to
look at beings and say, this guy is in pain, this one is having a ball. Etc.


This archive was generated by hypermail 2.1.5 : Wed Jul 17 2013 - 04:00:42 MDT