From: Daniel Burfoot (daniel.burfoot@gmail.com)
Date: Thu Feb 21 2008 - 19:05:50 MST
On Fri, Feb 22, 2008 at 2:30 AM, Matt Mahoney <matmahoney@yahoo.com> wrote:
> I described one possible design in http://www.mattmahoney.net/agi.html and
> did my thesis work to show that a very abstract model of this architecture
> is robust and scalable.
This is a nice idea - I hope you pursue it further.
> The idea that AI could fall into the "wrong hands" is like the Internet
> falling into the wrong hands.
>
Consider the following scenario. A pseudo-AI capable of high performance
computer vision understanding, speech recognition, and natural language
comprehension is developed and made widely available (perhaps a descendant
of PAQ).
Individuals with the semi-intelligent system can do all kinds of neat
things. They can build robots to fetch them coffee. They can teach their
cars how to drive. They can program face-recognition security systems that
protect their houses from burglars.
The government can do all of these little tricks too. However, the
government also has access to the following physical infrastructure:
1) visual surveillance systems in all public places
2) apparatus to monitor electronic communications across the globe
3) robotic soldiers
It seems to me that:
P(semi-intelligence) << P(semi-intelligence + physical infrastructure)
where P(..) is the "power function" - the amount of power/capability/utility
that a technological system makes available to its controller.
In our current system, there is a delicate balance of power between
individuals and the state. Many people would argue that the balance is
currently tilted too far towards the state. Regardless, the introduction of
semi-intelligence would seem to dramatically upset the balance of power,
even if it is made widely available, because of the way semi-intelligence
interacts with the pre-existing physical infrastructure the government has.
If you basically trust the government, then the above scenario shouldn't
worry you too much. I do not trust the government. We have no foolproof way
to guarantee that the reins of government power do not fall into the hands
of evil men. Tyranny has plagued humanity since the beginning of
civilization.
Dan
This archive was generated by hypermail 2.1.5 : Wed Jul 17 2013 - 04:01:02 MDT