From: Byrne Hobart (bhobart@gmail.com)
Date: Tue Apr 29 2008 - 08:39:53 MDT
On Tue, Apr 29, 2008 at 7:05 AM, Stathis Papaioannou <stathisp@gmail.com>
wrote:
> That's for a start, and if I find that my new
> programming leads me to behaviour that my higher order mind tells me
> is undesirable, I will adjust it accordingly. Thus there is no reason
> why my highest ideals should ever come into conflict with base sensual
> pleasure.
I suspect that there will be a danger of 'zombifying' oneself with actions
like this: if you automatically rewrite your preferences to maximize
pleasure and productivity, you have to wonder if this will mean obliterating
your ability to perceive such things so you don't have some kind of
unhappiness-generating philosophical crisis over your new set of
preferences. The ability to engage in speculation over whether or not one
should participate in the rat-race can lead to arbitrary levels of
unhappiness, and since you can rewrite your utility curve you will have to
end up either a) depressed despite feeling pleasure at doing good, or b)
unable to introspect enough to reconsider your initial decision.
One possible safety mechanism would be to simulate your responses under
various utility-schemes, observed by various utility-schemes. So, for
example, the drug addict can observe how he'd behave without the addiction,
how he'd behave if he thought being a hard-working family-raising tax-paying
law-abider was euphoric, etc., but could *also* observe his current
drug-addicted perspective from the meta-perspective of a non-addict, a
eudaimonic view, etc. This introduces a problem of indefinite abstraction,
though: what does the drug-addict personality think of the responses of the
non-addict personality to the eudaimonic package of perceptions. Besides
just declaring artificial boundaries, I am not sure how to resolve this.
This archive was generated by hypermail 2.1.5 : Wed Jul 17 2013 - 04:01:02 MDT