Re: State of the SI's AI and FAI research

From: Michael Wilson (
Date: Tue Feb 15 2005 - 20:09:17 MST

> I'm disappointed in myself and others that this or other special
> purpose list still hasn't morphed into mainly a theory-extending
> forum for discussing technical topics directly related to SeedAI
> and Friendliness.

I'm disappointed about that too. But seed AI is just too dangerous
to start handing out blueprints to people who don't have a clue
about friendliness, refuse to acknowledge the depth of the problem
and would cheerfully destroy the world if they got the chance. While
the majority of list participants do not meet this description,
there are plenty who do and it only takes one reckless half-brilliant
project to build an Unfriendly AI for it to be Game Over.

> It's just that the knowledge requirements to contribute anything
> relevant to the existing theory appear so high that there's
> hardly anyone who can meet them.

Thus the low expected utility of a public discussion; it probably
wouldn't be worth the time even if the planetkill risk wasn't there.

> By taking account of what you know vs. everything you need to
> know to write code. If you're at the stage where you still don't
> know what you don't know then, yes, your reaction is appropriate.

This is seed AI. There will be things we didn't know we didn't know
no matter how much theoretical effort we put in prior to the first
successful AGI project. The best we can hope to accomplish is being
reasonably sure that none of the things we don't know we don't know
are going to prove fatal.

 * Michael Wilson

ALL-NEW Yahoo! Messenger - all new features - even more fun!

This archive was generated by hypermail 2.1.5 : Wed Jul 17 2013 - 04:00:50 MDT