RE: Fighting UFAI

From: pdugan (
Date: Thu Jul 14 2005 - 15:07:10 MDT

>===== Original Message From H C <> =====
>Intelligence is an optimization process, wisdom is intelligent (highly
>optimized, perhaps?) information.

Thats the best definition I've read on this list, it does wonders to frame the
whole "AI with mind vs. mindless AI" issue as well as the difference between
Friendly and Unfriendly AI. The reason the paperclip example is useful is
because the task of tiling the whole light cone with a consistent physical
pattern requires plenty of intelligence, but hardly any imagination or wisdom.
To make an analogy to human pattern-reasoning, it takes intuitive intelligence
for a cave-man to throw a rock with precision, but it takes wisdom to apply
meta-patterns to that general optimization process of throwing and decide on
the proper context (target, position, escape route if target is bigger than
you and survives ect.) If dinosaurs got into genetic engineering they would
have been wise to not breed a man who could simply learn to throw, but who
could learn to reason about his throws. Of course these hypothetical dinosaurs
should also be aware that the extra work involved in ensuring wise throws from
their grown-up creation would not ensure that well-reasoned rocks throws
wouldn't attempt to harm them.

    By analogy, their are many more UFAI designs than FAI designs, but their
are also many more blind optimization process UFAI designs than their are
blind optimization process FAI designs. Meta-information makes better

   Patrick Dugan

This archive was generated by hypermail 2.1.5 : Wed Jul 17 2013 - 04:00:51 MDT