From: Stathis Papaioannou (stathisp@gmail.com)
Date: Fri Jun 27 2008 - 20:05:59 MDT
2008/6/28 Tim Freeman <tim@fungible.com>:
> A definition I could work with is that a "restriction" is an
> undesirable state. So if the AI is supposed to preserve its own
> integrity while ensuring that no humans are hurt, then we assign a
> positive utility to whatever fits "preserve its own integrity", and a
> negative utility to "humans being hurt", and then we scale them
> somehow and add them up, and that's our utility function.
We could restate the goals as "preserve the integrity of humans" with
the restriction "ensure that it is not hurt". At some point, the two
goals could come into conflict, and there is no a priori way of
deciding which is more important. It's just something that has to be
arbitrarily specified, or deduced from other goals which are
themselves arbitrarily specified.
> By the way, if the AI has any long-term goals, then it will want to
> preserve its own integrity in order to preserve those goals. Although
> "preserve its own integrity" is a good enough example for the issue at
> hand, it's not something you'd really need to put in there explicitly.
Yes, but for humans survival isn't just a means to an end, it is an
end in itself.
-- Stathis Papaioannou
This archive was generated by hypermail 2.1.5 : Wed Jul 17 2013 - 04:01:03 MDT