From: Stuart Armstrong (dragondreaming@googlemail.com)
Date: Wed May 07 2008 - 03:35:18 MDT
> What's this entity called "you" in the rules above? Humans have
> brains in their heads. They can think only with their brains and
> cannot copy that process onto another computational substrate. This
> makes it possible to identify humans by name and count them
> essentially by labelling the skulls. The process doesn't work for
> entities that don't share those anatomical restrictions, so it's not
> obvious how to identify the entities you're talking about any more.
> However, identifying the entities is an essential requirement for the
> laws you're proposing there to have meaning, so the whole thing seems
> unworkable.
That's avery good point; I was still thinking in a very human-centric
way (what ethical codes should a language translating program have?
not to mention self replicating viral programs, etc...)
> Using words to confine an entity that's going to be a better lawyer
> than you seems pretty dicey.
That's not the problem, though. The rules have to be integrated in
such a way that the AI wants to implement them.
> I don't see how to win with the slow-fast scheme. Once the dominant
> force in the world is a disorganized bunch of AI's competing with each
> other, and those AI's are able to do engineering, then neither human
> cognition nor human evolution are driving things any more. A
> collection of AI's isn't an individual AI, so no one program written
> by humans is in control any more either. With no traceable ongoing
> cause-and-effect bewteen humanity and anything identifiable that
> matters, I'd say we lost. I don't see hope of getting back on this
> horse after being thrown off.
I'm more optimistic about purely slow development; it seems possible
to "muddle through" that. It's the slow-fast that seems unbeleivably
worrying... Hope for diminishing returns on intelligence, then?
> J. Storrs Hall believes a slow takeoff is survivable, judging by his
> recent Singularity Summit talk. I didn't read his "Beyond AI" book
> yet so I don't know how he's able to believe that.
I'll have to read that, once I get some of that "free-time" everyone
seems to be talking about...
Stuart
This archive was generated by hypermail 2.1.5 : Wed Jul 17 2013 - 04:01:02 MDT