From: Matt Mahoney (matmahoney@yahoo.com)
Date: Thu May 08 2008 - 14:50:41 MDT
--- Stuart Armstrong <dragondreaming@googlemail.com> wrote:
> > There is no need to talk to the AI. It has an exact model of your
> > brain. It knows exactly what you are thinking. If it has anything
> > to say to you, it will just reprogram your beliefs.
>
> I'd claim this as a giant cheesecake - just because the AI could,
> doesn't mean it would. If we don't want it to, then we should be
> setting the whole thing up so that it doesn't do so.
Humans will no more design AI than the bacteria from which you evolved
designed humans. The fact that we would not be here in our DNA-based form
without our bacterial ancestors does not make us a godlike slave to them.
How should bacteria communicate with us?
A godlike AI should know how to make us happy. It should also know what
is in our best interest as a species, such as to fear things that could
kill us and then die, rather than be happy all the time. It should know
what is in the best interest of life in general, which might be to
exterminate some species. It might have goals we can't comprehend,
perhaps to exterminate all life in favor of something else. How do you
propose to influence its thinking?
-- Matt Mahoney, matmahoney@yahoo.com
This archive was generated by hypermail 2.1.5 : Wed Jul 17 2013 - 04:01:02 MDT