From: Dale Johnstone (DaleJohnstone@email.com)
Date: Mon Mar 26 2001 - 17:36:45 MST
Christian L. wrote:
>When I first started subscribing to this mailing list, I thought that the
>goal of SingInst was to build a transhuman AI. I was wrong. The goal is
>obviously to build a Utopia where Evil as defined by the members of the list
>will be banished. The AI would be a means to that end, a Santa-machine who
>uses his intelligence to serve mankind.
I can't speak for SingInst but as someone who's working towards the same goal, I'm in it because I see a possible way to do away with death and misery once and for all. Although I'm obsessed with AI, I'd switch to collecting detergent coupons if that would do any good. Unfortunately it doesn't. Building a transhuman AI though does.
List members do *not* get to define what is evil and what is banished. Basically, the idea is to build a really smart mind, and help it to *understand* us and our common desire for a better world in the same way that we do - not by some rigid laws cast in stone, but by thoroughly understanding the subtleties and details. If we've done our job correctly, it will eventually understand this even better that we do. Think of it as raising a child if you prefer.
Okay, next step is to have it create an even better version of itself - actually it will want to do this all by itself because it's such a good idea. It's up to it to choose how to do this, but because it's Friendly and smarter than we are; it'll make its successor (or a modified version of itself) Friendly too. The mark2 version will also do the same, only better. This is what I refer to as the Friendliness attractor. Each successive generation is better able to understand us and better able to help us.
Okay, now for the SysOp idea. This is what Eliezer and many on the list think the AI will come up with in order to be as Friendly and as fair to us as possible. We can't say for sure, but it's the current favourite. We don't get to decide this, the AI does. If the AI eventually thinks of something better then that's what'll happen.
>If you organize yourself in a "pack" and follow the rules set up there, you
>can get personal protection and greater means of achieving your goals (they
>normally coincide with those of the pack). When you interact with another
>pack-member, you can be pretty sure that he/she will not break the rules and
>risk exclusion from the pack. This can be called trust. The rules that the
>pack sets up can be called ethics.
This is all well and fine in the jungle when about the worst I could do was hit you with a stick. However, pretty soon nanotechnology is going to become readily available and it's practically impossible to defend yourself against it. If any single individual has the ability to turn the crust into bubbling slag - you can bet your life some crazy nut will do it, either deliberately or by accident.
The world by and large hasn't woke up to the facts yet. It's clear that things aren't going to get any better by themselves. I hope you can now understand the urgency in our desire to apply a little transhuman intelligence to the problem.
Regards,
Dale Johnstone.
This archive was generated by hypermail 2.1.5 : Wed Jul 17 2013 - 04:00:36 MDT