From: Keith Henson (hkhenson@rogers.com)
Date: Sun May 16 2004 - 08:18:22 MDT
At 03:39 AM 14/05/04 -0400, Randall Randall wrote:
>On May 14, 2004, at 1:24 AM, Pablo Stafforini wrote:
>
>>Michael,
>>
>>>I agree that what-is-in-a-person's-interests cannot always be
>>>equated with what-the-person-presently-wants. But I think
>>>that you omitted mentioning the cases where a person
>>>understands that their future wants will be different, and
>>>factors that into their decision-making process. I do this
>>>all the time, and I'm sure you do also. Eg: Though I desire
>>>an espresso now, I know that if I drink one I'll have trouble
>>>falling asleep later, so I make myself a warm milk instead.
>>
>>OK, but then we shouldn't describe the action as a case of allowing the
>>agent to do what she wants to do. Suppose I know I will be probably
>>ruined if I continue gambling, but I decide to do it anyway. I'm then
>>doing what is not in my best interest to do. I'm then acting
>>irrationally. Eliezer's maxim, then, becomes inapplicable. To assess the
>>agent's behavior we must look for an alternative rationale.
>
>Either you are using the term "best interest" for something I would
>not use that term for, or you are making the mistake of assuming
>that a single objective "best interest" exists which can be determined
>by an outside observer.
There are multiple viewpoints for "best interest" that are sometimes in
conflict. So the problem may not be solvable at all.
>In order to determine a person's best interest, you would have to
>weigh their options against their goal system (not yours!) and
>choose the best option which is consistent with that goal system.
Ultimately it is the "goal" of a person's genes to get themselves into the
next generation and beyond. To do so the genes build brains that have
psychological traits that--in the past--worked most of the time to get the
genes of those bodies and brains down the time stream. Of course, genes
are more remote from day to day behavior than the board of directors of a
major company is from a shipping clerk. Thus brains develop a viewpoint
which is sometimes in conflict with the "goal" of a person's genes.
As an example, consider the celibate priest. It is possible for genes for
celibate behavior to evolve as they did in bees and ants, but there is no
evidence that the close relatives of celibate priests do enough better at
reproducing to compensate for the loss of the gene copies in the
priests. Besides, priests are post agriculture which is to say way too
recent for evolution to have shaped the psychological traits behind such
behavior.
(There should be a reader who can explain the psychological trait behind
celibate priests.)
There is a statement out of evolutionary theory which (paraphrased) says
that features and behavior of a species are either the result of direct
selection or side effects (spandrels) of features or behaviors that were
selected. Sometimes, such as the behavior of taking addictive drugs, it is
clear that the trait is a side effect. In the case of drug addiction, a
side effect of a brain reward system mediated by chemicals. Some plants
make chemicals with a shape close enough to mimic the effects of the
natural reward chemicals.
>Unless you are intelligent enough to closely simulate that person,
>however (and no human currently is), you are unlikely to be able
>to make such a determination, so you must accept the person's own
>decisions as the closest approximation to their "best interest"
>that you can find.
Ten years ago I would have agreed with you. There is a strong libertarian
outlook in me that was shaped by decades of Heinlein's influence.
But over the past ten years I have come to see "the person" as something
less than a unified whole, burdened by evolved psychological traits that
may be way out of step with reality. Gambling, drug addiction and cult
involvement, i.e., infection with a parasitic meme, are pathological states
where intervention may be justified--not simply because of the damage to
the person, but the dangers to the larger community.
We are *social* primates. Rational behavior (sanity if you will) is to a
considerable extent maintained by interactions with those "in your
tribe." That being the case, we have self-interest in keeping our
neighbors sane, and they in keeping us sane--especially since we have
mechanisms that switch on irrational behavior in response to environmental
conditions (though "irrational behavior" may be rational from a gene's
viewpoint in a hunter gatherer environment--key phrase in Google
"xenophobic memes").
There really isn't a simple answer.
To relate this back to the subjects of main interest here, understanding
these matters might be essential to providing the environment in which
friendly AI can be developed.
Keith Henson
PS. If you have never read it, this article is an excellent overview for
"all you zombies."
http://www.vivaconsulting.com/education/hijacking.html
This archive was generated by hypermail 2.1.5 : Tue Feb 21 2006 - 04:22:36 MST