From: Giu1i0 Pri5c0 (pgptag@gmail.com)
Date: Fri May 21 2004 - 10:05:33 MDT
Hi this is Giu1i0 resubscribed with a new address.
Well you can say the same of electricity. Once I cut a powered wire
with a knife (!!!), fortunately the house safety system kicked in in
time and the only damage was a scare and a big dent in the knife. Of
course I imagined only the easily mentally accessible consequences of
the act (cutting the wire and getting to the next thing), not the
actual consequences.
So the danger of doom is always there with all tech even low tech. Of
course when we talk of self modification we have to realize that the
technology is very complex and not yet fully understood (so the risk
is very high) and the potential consequence of failure are very
catastrophic. So the technology should be developed and deployed
carefully, step by step, always trying to stay on the safe side... but
this is nothing new, just sound engineering.
On Fri, 21 May 2004 06:35:43 -0400, Eliezer Yudkowsky
<sentience@pobox.com> wrote:
>
> Eliezer Yudkowsky wrote:
> ...And finally, what about all the consequences, and categories of
> consequences, that you haven't foreseen? When you imagine the act of
> self-modification, you will imagine only the easily mentally accessible
> consequences of the act, not the actual consequences. Just because you
> can't see the doom, doesn't mean the doom isn't there.
This archive was generated by hypermail 2.1.5 : Wed Jul 17 2013 - 04:00:47 MDT