From: Rafal Smigrodzki (rafal@smigrodzki.org)
Date: Wed May 21 2003 - 17:37:07 MDT
Eliezer wrote:
> Rafal Smigrodzki wrote:
>>
>> ### Here we definitely agree - a huge government-funded AI program
>> would be a great idea. Since the FAI may be interpreted as the
>> ultimate public good, a boon for all, yet profit for nobody, a good
>> case can be made for public funding of this endeavor, just like the
>> basic science that gave us modern medicine. This program, if open to
>> all competent researchers, with results directly available to all
>> humans, could vastly accelerate the building of the FAI.
>
> How can you get FAI by throwing money at the problem?
> *Unfortunately* I can see how throwing sufficiently vast amounts of
> money at AI might result in advances on AI. But how do you get
> advances in Friendliness? It seems to me that the optimal political
> scenario for ensuring FAI over UFAI calls for all researchers to have
> *exactly* equal resources and funding, so that the smartest
> researchers have an advantage. Why is this the optimal political
> scenario? Because there is absolutely no way in heaven or Earth that
> the political process is capable of distinguishing competent
> Friendliness researchers from noncompetent ones. Any non-maxentropy
> *political* resource distribution will probably be pragmatically
> worse than an even distribution. Furthermore, you don't want absurd
> quantities of resources, either, as otherwise you may push research
> into the territory where brute-forcing AI becomes possible.
>
> The more you look into the problem, the more you realize how hard it
> is to find forces that genuinely *improve* our prospects, rather than
> making things even worse.
### This is a very interesting point of view. I think you might overestimate
the amount of correlation between AI-applicable smartness and
Friendliness-competence. There is clearly a positive correlation between
high IQ and lack of meanness, measured by criminality, marital fidelity,
etc. but if the correlation is not high enough, having the smartest
programmers might not always be sufficient to minimize the risk of UnFAI, in
the absence of additional character analysis.
The optimal scenario would be for all Friendliness-competent and -willing
researchers to have ridiculously good funding, with nothing left for the
UnFriendly ones. Since we do not have a validated measure of
Friendliness-competence, you are using smartness as a substitute. This would
be reasonable, if Friendliness theory predicted that it would be impossible
for a programmer to build an AI which would be stably Friendly to him alone,
because then the only rational behavior for both the cynical egoist and the
altruist, would be to work on true FAI, and the smartest programmers would
be most likely to see it. However, if there was an objectively existing
chance to build an AI with a special love for its maker alone, then some of
the smartest programmers might be tempted to try for personal godhood.
As long as the political funder is not trying to micromanage the situation,
support would be offered to people with the ideas judged most promising by
their peers who had shown some results before. This is not likely to put the
best FAI programmers at a systemic disadvantage. The correlation between
being a good FAI programmer and getting funded probably won't be very high -
but it will be better than zero and is unlikely to be negative (unless of
course the peer reviewers are in their majority totally wrong).
But the condition for would is definitely an absence of direct control by
non-experts, such as politicians and activists of all sorts. I share your
skepticism as to the likelihood of this condition being satisfied, but one
can always try to give good advice to the government guys. Maybe they'll
listen.
Rafal
This archive was generated by hypermail 2.1.5 : Wed Jul 17 2013 - 04:00:42 MDT