From: Ben Goertzel (ben@goertzel.org)
Date: Sun Sep 15 2002 - 19:38:17 MDT
> For example, as I understand AI, I
> don't see how
> you could possibly "force-feed an AI with false knowledge" unless you'd
> already brainwashed it into operating the belief-formation system under
> your direction.
I'm guessing that with sufficiently advanced tech it will also be possible
to "force-feed" *humans* with false memories and knowledge.
So I was not assuming a symbolic or semi-symbolic knowledge structure in my
statement.
>
> It seems to me that a seed AI, even without deliberately obfuscating
> itself in any way that would interfere with Friendly AI, can
> easily create
> so many simultaneous dependencies within the system that no human
> programmer is capable of modifying it without AI assistance. Moreover,
> that no human programmer is capable of ripping apart the AI and putting
> the pieces back together in any kind of working order. In fact, I would
> tend to suspect that this would happen naturally and without any
> deliberate planning being required on the AI's part.
I think it's *possible* this will be the case.
I also think it's possible that reprogramming (brainwashing) the mind of a
near-human-level general AI may be possible for a sufficiently clever team
of experts...
To my mind, it's too early to say...
> And there's no
> reason why an AI *couldn't* plan ahead for the contingency of being
> stolen. Things that enable an AI to resist unfriendly external
> modification are not necessarily things that interfere with Friendship
> programmers seeing inside the AI and *consensually* modifying it.
True. And if the AI is just near-human rather than transhuman, there's no
reason why it couldn't plan ahead for this contingency, but be outsmarted by
humans who see through its planning...
-- Ben G
This archive was generated by hypermail 2.1.5 : Wed Jul 17 2013 - 04:00:41 MDT