From: Heartland (velvethum@hotmail.com)
Date: Fri Jun 17 2005 - 23:16:50 MDT
> What if we build a key?( or a bridge based on nanotech) I have a few
> ideas and would love to see what we could collaboratively think of.
>
> digger
This would be even mildly funny if you didn't miss the point of the message.
The point was that I *didn't* think that preventing grey goo required
building FAI, yet I thought this was exactly what Eliezer meant by his
resolve to finish building FAI before arrival of nanotechnology. I couldn't
understand why he would offer such a poor argument to justify building FAI.
It turns out that I initially misinterpreted what he meant by, "we need to
finish FAI before nanotech," as, "we need to finish FAI before grey goo,"
rather than, "we need to finish FAI before nanotech-era computers."
(Actually, my opinion has always been that only necessity of preventing UAI
threat warrants an attempt to create FAI, considering huge amount of
existential risk associated with building recursively self-improving AI.)
Slawek
> Heartland <velvethum@hotmail.com> wrote:
> p3 wrote:
>> I don't understand why the development of molecular
>> nanotechnology will mean the inevitable destruction of
>> all things everywhere (on earth, at least), or why the
>> development of smarter-than-human intelligence will
>> somehow avoid this disaster.
>
>
> Eliezer replied: "Because by far the simplest and most commercially
> attractive application of
> molecular nanotechnology is computers so ridiculously powerful that not
> even
> AI researchers could fail to create AI upon them. Brute-forced AI is not
> likely to be Friendly AI. Hence the end of the world."
>
> Oh, so that's what that
> we-need-to-build-Friendly-AI-before-nanotech-or-we'll-die thing was about.
> For the last 3 years I've been under impression that SIAI was building
> Friendly AI to avert the end of the world caused by inevitable grey goo.
> This didn't seem to me as a good enough reason to take an even greater
> existential risk and attempt to create FAI. Now, this justification is
> much
> more logical. I suggest you make this explanation more visible in SIAI
> introductory material to avoid misinterpretation of the main reason why
> SIAI
> is trying to finish building FAI before the arrival of nanotechnology.
>
> Slawek
This archive was generated by hypermail 2.1.5 : Wed Jul 17 2013 - 04:00:51 MDT