From: Mitch Howe (mitch_howe@yahoo.com)
Date: Mon Feb 04 2002 - 15:13:02 MST
Eliezer Yudkowsky wrote:
> Even if your target audience is core
> Singularitarians, I still don't think this is good enough. Knowledge that
> doesn't come with an explanation is not real knowledge. I'm not saying
> that you have to repeat everything that's said in "Created Friendly AI",
> but something being presented as a FAQ should at least contain the
> outlines of the arguments for each position.
> At the very least, if you want to say that the Sysop does not have an
anthropomorphic power-corrupts
> module, you have to spend a few sentences arguing it and include a link to
> a more detailed reference like CFAI 2.2.whatever. You can't just say:
> "Don't worry! The Sysop is totally immune to the tendency to be corrupted
> by power!" This does not reassure people.
Maybe we disagree on what the purpose of an FAQ should be. If my FAQ was
supposed to be taken as an attempt to convince everyone that a Sysop is a
wonderful idea, I have failed miserably.
My goal was to have the reader go away thinking, "When all is said and done,
the Sysop Scenario is just an interesting thought experiment. But I do see
that there could be some pretty great things on the horizon. Now I wonder
what the future is *really* going to be like. . ."
My goal was not thave the reader go away thinking, "Eliezer and the
Singularitarians think they know exactly what is best for humanity and are
building a supercomputer that will force their opinion on everyone and
everything." Unless every effort is made to dispel this myth, this will be
the default opinion of most people who catch wind of the Sysop Scenario.
(Please reassure me that this is not a myth!)
I agree that it wouldn't hurt to beef up some of the answers by providing
more metaphors and links. I intend to do this. But I think it would be a
mistake to project the idea that CFAI proves a Sysop to be the very
embodiment of Friendliness. I didn't think this was your position; If it
is, I confess to having missed the whole point of CFAI.
--Mitch Howe
_________________________________________________________
Do You Yahoo!?
Get your free @yahoo.com address at http://mail.yahoo.com
This archive was generated by hypermail 2.1.5 : Wed Jul 17 2013 - 04:00:37 MDT