From: Brian Atkins (brian@posthuman.com)
Date: Fri Apr 06 2001 - 01:07:16 MDT
James Higgins wrote:
>
> At 01:39 PM 4/5/2001 -0400, Brian Atkins wrote:
> >Samantha Atkins wrote:
>
> You two related or something?
No
>
> > > I don't see how this follows. If you upload a human who quickly
> > > self-improves ver capabilities and becomes an SI and if
> > > super-intelligence brings with it expanded moral/ethical understanding
> > > then I see no reason this combination is less trustworthy than starting
> > > from scratch and only putting in what you believe should be there in the
> > > beginning. Yes a lot of evolved complicated behavior and conditioning
> > > is not present in the AI. But some of that complicated behavior and
> > > conditioning is also the bed of universal compassion and utter
> > > Friendliness.
> > >
> >
> >Lot of ifs there...
>
> Well, everything we talk about would need to be qualified a hundred
> different ways if we got down to the nitty gritty. At this point, this is
> more SF than a science. That will change, however, over the next few years.
>
> >What it seems to come down to is you are either relying on objective
> >morality (in which an AI should do better since it has less evolved
> >crap to deal with), or a natural convergence to the "Friendly zone". In
> >which case we also argue that a properly designed AI should easily
> >outperform a human attempting to upgrade him/herself. The reason I think
> >is easy to see: you can't really predict in advance which particular
> >human will become utterly Friendly vs. which particular human will become
> >the next Hitler when presented with the total power uploading/becoming a
> >SI would give them. History has shown a tendency for power to corrupt
> >humans. At least with an AI we can sharply reduce the risks by a) designing
> >it right b) testing testing testing
>
> BS. Testing means jack when you are talking about something that will
> complete reinvent itself a thousand times. And I have a feeling that the
> design will have little long-term impact also. Certainly, the design will
> influence how well the AI starts off, and how well it upgrades itself the
> first few times or so. But beyond the first dozen or less redesigns, it
> will have little, if any, resemblance to the original. So excuse me if I
> don't have much faith.
Feel free to debate the technical points with Eliezer, he'd love to hear
some real criticism of the details...
>
> >You want to talk about who designs the first AI? Well who decides who gets
> >to be the first upload?
>
> Hey, that is easy. I'll be first. ;)
Well James we don't see didley squat in terms of new ideas from you, so I
doubt you would be the one the world would choose (if they were picking), and
I doubt you will be the one to invent uploading technology. I asked you for
your ideal scenario and got nothing except further knee jerk reactions to
ours. I point out the flaws in uploading humans, and instead of coming up
with some argument against that you choose to make seat of the pants judgements
about our AI plans without (apparently) even reading our online documentation.
-- Brian Atkins Director, Singularity Institute for Artificial Intelligence http://www.intelligence.org/
This archive was generated by hypermail 2.1.5 : Wed Jul 17 2013 - 04:00:36 MDT