From: Ben Goertzel (ben@goertzel.org)
Date: Wed Feb 06 2002 - 06:24:26 MST
I'll say it again: It may be fun to talk about, but this sort of
post-Singularity stuff is *way* beyond our capability to project or
comprehend with any reasonable degree of confidence
Enjoy the speculation, but don't fool yourselves that there's anything but a
Leap of Faith involved here...
ben g
> -----Original Message-----
> From: owner-sl4@sysopmind.com [mailto:owner-sl4@sysopmind.com]On Behalf
> Of Eliezer S. Yudkowsky
> Sent: Wednesday, February 06, 2002 12:04 AM
> To: sl4@sysopmind.com
> Subject: Re: Sysop hacking
>
>
> > Mitch Howe wrote:
> >
> > I'm in the process of beefing up the Sysop Scenario FAQ, but I'm stuck
> > on the question "Could someone hack into a Sysop?"
> >
> > The only answer I have is "No, because if its design were so vulnerable
> > a Friendly SI would never implement it." I believe that this is a valid
> > answer, but such ontological responses are not very satisfying to the
> > reader or the writer.
>
> The response I would give is this:
>
> Any hack requires a vulnerability. Human code often contains
> vulnerabilities because humans have relatively short attention spans and
> can't linger for hours over each line of code. Also, humans often write
> lousy code because we are basically hunter-gatherers and programming is
> not one of the tasks we are evolved to accomplish. It's quite possible
> that code written by a sufficiently smart AI simply would not contain
> code-level flaws, such as buffer overruns and the other flaws that modern
> hackers exploit. To put it another way, the "game" of hacking doesn't
> necessarily continue between superintelligent programmers and
> superintelligent attackers. It could, but it's also possible that given a
> superintelligent programmer there are just no mistakes left to exploit,
> even if the attacker is a hundred times more superintelligent.
>
> -- -- -- -- --
> Eliezer S. Yudkowsky http://intelligence.org/
> Research Fellow, Singularity Institute for Artificial Intelligence
>
This archive was generated by hypermail 2.1.5 : Wed Jul 17 2013 - 04:00:37 MDT