From: Eliezer S. Yudkowsky (sentience@pobox.com)
Date: Wed Apr 04 2001 - 23:12:02 MDT
James Higgins wrote:
>
> I'm not personally convinced that uploading a human first isn't the correct
> answer. I'm rather inclined to trust a person more than an AI to map out
> our destiny.
The problem with uploading is that, chronologically, it arrives
considerably after both artificial intelligence and military-grade
nanotechnology. So unless you use a superintelligent AI as assistant,
uploading can't plausibly come first. But if you can trust an AI to play
assistant, then you can trust it to be Transition Guide (i.e., essentially
the same order of risk in both cases).
I'm also more inclined to trust an AI more than a person... maybe even
more than I'd trust myself, since I'm not designed for recursive
self-improvement. Actually, I should amend that: After I've been around
an AI and had a chance to chat with ver, then I expect to wind up
justifiably trusting that AI's seed morality around as much as I'd trust a
human seed morality, and I can also foresee the possibility of standing in
the AI's presence and just being overawed by vis seed morality. Either
way, I also expect to wind up trusting that AI's transcendence protocol to
preserve morality significantly more than I'd trust a human-based
transcendence protocol to preserve morality.
-- -- -- -- --
Eliezer S. Yudkowsky http://intelligence.org/
Research Fellow, Singularity Institute for Artificial Intelligence
This archive was generated by hypermail 2.1.5 : Wed Jul 17 2013 - 04:00:36 MDT