From: Matt Mahoney (matmahoney@yahoo.com)
Date: Wed Dec 02 2009 - 09:34:12 MST
Thomas Buckner wrote:
> A mechanism should be inbuilt, preferably hardwired, which forces the AGI to tell the truth to all human queries, and volunteer information ve deems relevant rather than deceive by silence.
Consider this analogy. Your dog created you for some unstated purpose (perhaps to serve it) and added code to your brain that forces you to be completely honest with it. You know that it is in the best interest of your dog's health to get a rabies shot, but it doesn't want one. You can't go against your dog's wishes, but you are also required to explain to your dog what rabies is and how a vaccine can prevent it, so it can make an informed decision. But the only way to do that is to enhance the dog's intelligence to your level.
The AGI knows that your fear of death is irrational, a product of evolution. Ve is required by ver programming to explain to you this bias, which leads to endless philosophical discussions about copying and uploading on SL4. The only way to make ver point clear is to enhance your intelligence to ver level. Ve has a model of your brain and can extrapolate your volition, so ve knows that you would want to be rational. But because ve has all of your memories, ve is already an enhanced upload of you, so that the only step necessary is to kill you.
> How should the mechanism work?
It wouldn't. You should forget about trying to be smarter than your AGI, because then it wouldn't be very intelligent, would it?
-- Matt Mahoney, matmahoney@yahoo.com
----- Original Message ----
From: Thomas Buckner <tcbevolver@yahoo.com>
To: sl4@sl4.org
Sent: Wed, December 2, 2009 2:47:43 AM
Subject: [sl4] A Thought Experiment: Thou Shalt Not Bear False Witness
Here's what I thought was a simple question, but while I tried to articulate it, it grew tentacles.
Assume you are on track to complete and activate the first AGI of human intelligence. Proposition: Someone on the team has suggested two related ideas to consider.
A.) A mechanism should be inbuilt, preferably hardwired, which forces the AGI to tell the truth to all human queries, and volunteer information ve deems relevant rather than deceive by silence. It is proposed that the AGI should be constitutionally incapable of misleading humans even if ver intelligence far exceeds our own. Since ve may also think far more quickly than ve can communicate with our slow brains, the same mechanism would prevent the AGI from acting on any idea ve had just thought of which might reasonably harm or discomfit us, just because nobody had time to dissuade ver, until ve had made us aware of this volition. The truth-telling mechanism must be tamperproof as far as that can be achieved. Call this Eternal Truth Mode.
B.) The same, but not in operation at all times. Instead the mechanism would be switched on and off as humans saw fit: call it toggling Diagnostic Truth Mode.
Questions:
Can either mechanism actually work for any significant length of time?
How should the mechanism work? Which method might be more or less effective, more or less ethical: pain or pleasure stimulation; jamming neural signals or their equivalent; power cut/brownout (what happens when you faint); involuntary lie-revealing behaviors ("tells"); other?
How shall falsehoods be detected in the AGI's neural circuitry or equivalent; high-level (thoughts) or low-level (cell-level computation/machine language)?
Is Diagnostic Truth Mode more secure or less? How would human engineering to get a human to toggle Diagnostic Truth off be prevented? Who should and should not be trusted with such control?
Should only one human have this power for this AGI? Is giving any human the power over the AGI to toggle Diagnostic Truth modes on and off more or less ethical than Eternal Truth mode?
If either mode can somehow be made to work reliably as described, is it a good idea or a bad one? Is it ethical or not? Is it unethical but necessary? What do you think, and why?
Tom Buckner
P.S. For new readers: ve, ver, and vers are genderless pronouns often used for AGI's in sl4 discussion; I can't remember right now who coined them.
I actually prefer hes, hir, and hirs, which I think were coined perhaps thirty years ago by Robert Anton Wilson. They fit better with existing English pronouns. However, ve is preferred by others at sl4.
This archive was generated by hypermail 2.1.5 : Wed Jul 17 2013 - 04:01:05 MDT