From: Mike & Donna Deering (deering9@mchsi.com)
Date: Sun Jul 07 2002 - 08:22:37 MDT
Eliezer writes: "To understand the relative probability of FAI and UAI, you need to look
at the various a priori arguments for and against these possibilities in
minds-and-general, the arguments that are dependent on specific features
of the mind, and so on. Then you include and exclude controllable
features of the initial conditions such that the feature set maximizes
the probability of FAI as you understand it. Seems straightforward, but
it wouldn't be possible if two options always equaled 50/50."
Okay, lets look at the a priori arguments. In minds in general UAI could result from competition for resources, fear of actions of others, or conflicts of goals with others. On the other hand, FAI could result from an environment of unlimited resources, the ability to perfectly simulate the thoughts of other sentients and predict their actions, and perfect coordination of goals with all other sentients, or the result of a purposeful design sophisticated enough to address the conflicts with the priors of UAI.
Without Friendliness design the situation is pretty much hopeless. Now we have to evaluate two other factors. Both the honesty of the person designing the Friendliness design and the effectiveness of the friendliness design. Unless we have high confidence in both of these simultaneously we do not have high confidence in the final result. Considering the history and propensity of humans for selfish motivations I don't see how we can assign a high confidence to the first. And considering the history and propensity of humans for overconfidence and hubris I don't see how we can assign a high confidence to the second.
Mike.
This archive was generated by hypermail 2.1.5 : Wed Jul 17 2013 - 04:00:40 MDT