From: Wei Dai (weidai@weidai.com)
Date: Sat Nov 10 2007 - 17:37:31 MST
Jef Allbright wrote:
> I expect that you would be disappointed and frustrated as the higher
> intelligence would appear to respond in riddles. Pressed further, it
> might respond [cf. Eliezer] that there are no mysterious answers, but
> only mysterious questions, and that when you have composed a proper
> question you will have the answer you seek.
I definitely expect better results than that. At least some of the questions
are clearly not "mysterious". For example we already have a completely
non-mysterious (but imperfect) answer to "How does induction work?" in the
form of Solomonoff Induction. If human beings can discover Solomonoff
Induction, shouldn't an SI be able to do better?
> Note that we can expect a more advanced intelligence to demonstrate
> objectively superior results (relative to a given goal) but this does
> not imply that it could effectively explain it to a lesser
> intelligence.
Take Solomonoff Induction again. Sure you need a certain level of
intelligence to understand it, but not SI-like.
> It could provide the **appearance** of a satisfactory answer.
I realize this is a possibility, and stated it myself as "search for the
string that maximizes intellectual satisfaction for the questioner". However
this is an outcome that I would really like to avoid.
The rest of alternatives that you suggest either represent more mundane
failures of Friendliness or are covered by my response above.
This archive was generated by hypermail 2.1.5 : Wed Jul 17 2013 - 04:01:00 MDT