From: Matt Mahoney (matmahoney@yahoo.com)
Date: Fri Jul 18 2008 - 20:54:26 MDT
--- On Thu, 7/17/08, CyTG <cytg.net@gmail.com> wrote:
> If the intelligence of an selfimproving AI is on the rise,
> exponentially, why would it ever wanna 'merge' with another AI?
We don't know that self improvement is possible except as an evolutionary process. As discussed earlier, we lack even a mathematical model of deliberate self improvement, as well as a means for an agent to test another agent of greater intelligence. Even in simplified environments, we know of no classes of problems which are provably hard to solve but easy to verify.
Also, "merge" is not well defined. If by "merge" you mean that the agents cooperate to the greatest extent possible, then this is still a form of trade. If their utility functions differ, then some negotiation is required to reach an agreement on their combination. Even if the agents are identical, then their utility functions will differ if both agents are motivated by self interest, for different selves.
It is to the advantage of both agents to share and redistribute information to avoid redundancy and allow specialization. I expect cooperation to happen because cooperating groups will have a selective advantage over non-cooperators. But is this trading or merging? Do the employees of a company trade or merge? Do the cells in your body trade or merge, or something in between?
-- Matt Mahoney, matmahoney@yahoo.com
This archive was generated by hypermail 2.1.5 : Wed Jul 17 2013 - 04:01:03 MDT