From: Matt Mahoney (matmahoney@yahoo.com)
Date: Tue Oct 30 2007 - 15:35:47 MDT
So we want is AI that is as smart as us, but no smarter?
My real concern about CEV is that it only defines the problem of friendliness.
It is not a solution.
A smarter than human AI could predict what we would want in the long run
better than we could ourselves. But the step of actually motivating it to do
so is unsolved. The analogous situation is that we know how to care for our
pets better than they do themselves, but that is no guarantee that we or our
descendants will actually do so.
If that is the case, then we are either destined for godlike intelligence or
doomed, depending on your beliefs about consciousness. (Do you become the
godlike intelligence that replaces the human race if it imperfectly copies
your memory?) The first AI that is as smart as us will start improving
itself, friendly or not.
-- Matt Mahoney, matmahoney@yahoo.com
This archive was generated by hypermail 2.1.5 : Wed Jul 17 2013 - 04:00:58 MDT