From: Tim Freeman (tim@fungible.com)
Date: Wed Apr 30 2008 - 14:42:34 MDT
From: Matt Mahoney <matmahoney@yahoo.com>
>I realize you can describe a simple guiding principle for Friendliness
>such as "increase total human utility". But it is up to humans to
>write the code that describes our utility function.
This assumes that humans can't write fairly compact code that
estimates the human utility function, given human behavior as input.
I have a specification of that code, and it's fairly simple. You
could execute it if you had a Python interpreter running on a
more-than-astronomically fast computer. See
http://www.fungible.com/respect/paper.html.
If you wish to maintain your conclusion, you could argue that my spec
is wrong, or that an implementation of it that really works on
buildable hardware would be incomprehensibly difficult. I'd really
like to see a good argument that my spec is wrong, so please go that
way if you have a choice.
>You cannot have any help from AI because that would mean the AI is
>helping to reprogram its own goals.
You're splitting things into two pieces when you don't need to, and
then arguing that each piece must precede the other so it can't be
done. The two pieces are the AI writing code and the AI determining
what its goals are. It is possible to solve both problems at once by
putting the code generation into the utility function.
-- Tim Freeman http://www.fungible.com tim@fungible.com
This archive was generated by hypermail 2.1.5 : Wed Jul 17 2013 - 04:01:02 MDT