Re: Limiting scope of first attempt at seed AI

From: Russell Wallace (
Date: Fri Jul 15 2005 - 20:26:43 MDT

On 7/15/05, Chris Capel <> wrote:
> I think this means that domains (as an alternative to CV, if that
> makes sense) are out of the scope of discussion--we don't want our
> *first* iteration transhuman divvying up the universe first thing
> after takeoff

Just to clarify my intent regarding domains:

1) I think Domain Protection might be regarded as either an
alternative or a precursor to Collective Volition (i.e. if enough
people believe in CV, I certainly wouldn't object to them trying it,
provided they do it in a domain such that if it turns out badly - as I
think likely - it won't destroy the universe).

2) I definitely do _not_ advocate a first iteration transhuman-level
AI divvying up the universe off its own bat (or engaging in any other
activity that implies world conquest) first thing after takeoff!
Instead, I advocate that once Friendly AI is developed, humanity
should agree to use it as a tool to divvy up the universe into domains
in accordance with human volition (with a small "v").

It is important to remember that while the political issue of
Friendliness _content_ is easier to discuss on mailing lists, the
technical issue of Friendliness _architecture_ is the problem that
needs to be solved first. (I would say it's the hard problem, but
that's a technologist speaking... for all I know, persuading the
United Nations that something along the lines of domain protection is
necessary will make Friendliness architecture look like memorizing the
seven times tables! Either way, though, I think it's important not to
confuse the two.)

- Russell

This archive was generated by hypermail 2.1.5 : Wed Jul 17 2013 - 04:00:51 MDT