From: Ben Goertzel (ben@goertzel.org)
Date: Sun Mar 10 2002 - 10:30:36 MST
Hi,
> Actually, as described, this is pretty much what I would consider
> "human-equivalent AI", for which the term "transhumanity" is not really
> appropriate. I don't think I'm halfway to transhumanity, so an
> AI twice as
> many sigma from the mean is not all the way there. Maybe you should say
> that Novamente-the-project is striving for human-equivalence; either that,
> or define what you think a *really* transhuman Novamente would be like...
Well, not surprisingly I disagree with you here. I do not think that either
of your alternatives:
a) make the (false) claim that our project is striving for human equivalent
AI, or
b) put significant effort into making a detailed description of a
profoundly-transhuman Novamente
is the best course for us at the present time.
We are striving first for human-equivalent, then slightly-transhuman, then
profoundly-transhuman AI.
I do not think it is important for us to articulate in detail, at this
point, what we believe a profoundly-transhuman AI evolved from the Novamente
system will be like.
Because, I think it is probably NOT POSSIBLE for us to envision in detail
what a profoundly-transhuman AI evolved from the Novamente system will be
like.
I think the farthest out we can plausibly extrapolate, right now, is the
slightly-transhuman level. The next level beyond that will be co-created by
ourselves and the slightly-transhuman AI itself.
I choose to focus my efforts on how to get from here to the
slightly-transhuman level. Of course, the future beyond that enters into
our thinking on system design occasionally. But I don't think that focusing
more on the profoundly-transhuman level, at this point, would help us get to
that level any faster or any better.
> Incidentally, this thing of having the project and the AI having the same
> name is darned inconvenient. Over on this end, the group is SIAI, the
> architecture is GISAI, and the AI itself will be named Ai-something.
Just refer to "The Novamente project" versus "The Novamente system."
I am not so sure this nomenclature is inferior to inventing a panoply of
acronyms ;)
> Claims are meant to be tested. So far the claims have been tested on
> several occasions; you (and, when I was at Webmind, a few other
> folk) named
> various things that you didn't believe could possibly fit into
> the CFAI goal
> system, such as "curiosity", and I explained how curiosity indeed fit into
> the architecture. That was an example of passing a test for the
> first claim
> and third claim. If you have anything else that you believe cannot be
> implemented under CFAI, you can name it and thereby test the claim again.
I understand how, in principle, any other goal can be "fit into" the CFAI
goal architecture.
Similarly, in principle, any other goal can be "fit into" the more flexible
Novamente goal architecture.
This is sort of like how there are many models of universal computing, each
of which can emulate any kind of computing.
It's also sort of like how, in principle, any piece of human common sense
knowledge can be expressed in predicate logic. However, this "in principle"
expression doesn't necessarily work out well in practice.
Making a hand-waving argument that curiosity (or X or Y or Z) can be
represented in your goal system, does not demonstrate that this
representation can be pragmatically useful in the course of a real AI
system's experience. Just as making a 10-page 90%-accurate representation
of the concept of "curiosity" in terms of predicate logic does not imply
that this representation is of any use.
I am not drawing a technical parallel between your goal architecture and
predicate logic, but merely using the predicate logic expression of common
knowledge as an example of an "in principle" expression that sounds good at
first, but doesn't pan out so well in practice. Your reduction of other
goal systems to parts of your goal hierarchy may be a similar case.
> Similarly, I've explained how a goal system based on predicted-to versus
> associated-with seeks out a deeper class of regularities in
> reality, roughly
> the "useful" regularities rather than the "surface" regularities. Two
> specific examples of this are: (1) CFAI will distinguish between genuine
> causal relations and implication-plus-temporal-precedence, since only the
> former can be used to manipulate reality; Judea Pearl would call this
> "severance", while I would call this "testing for hidden third causes". I
> don't know if Novamente is doing this now, but AFAICT Webmind's
> documentation on the causality/goal architecture didn't show any way to
> distinguish between the two.
This is an error on your part, which I tried but failed to correct at the
time.
In fact, both the Webmind AI Engine and Novamente distinguish between
"temporal implication" and "causality", including the former as one
component of the latter. I sent you a long and detailed paper by Jeff
Pressing (of the Webmind Inc. R&D group) which discussed many leading
approaches to causal inference in the research literature, including
probabilistic approaches (though not Pearl's specifically), and which very
clearly distinguished "temporal implication" from causality.
Furthermore, the rough-draft paper on "varieties of self modification" that
I recently posted to this list, mentioned causality, and clearly indicated
two key ingredients to causality:
a) temporal implication
b) the existence of a plausible causal mechanism
In Novamente the notion of a "plausible causal mechanism" has a particular
implementation in terms of other Novamente concepts such as schema and
patterns, which I can't go into here due to lack of time...
> (2) CFAI will distinguish contextual
> information that affects whether something is desirable; specifically,
> because of the prediction formalism, it will seek out factors that tend to
> interfere with or enable A leading to B, where B is the goal
> state. I again
> don't know about Novamente, but most of the (varied) systems that were
> described to me as being Webmind might be capable of
> distinguishing degrees
> of association, but would not specialize on degrees of useful association.
This is a misunderstanding on your part, again.
Of course, in reasoning about the fulfillment of goals represented by
GoalNodes, both Webmind and Novamente can reason about degrees of useful
association, and degrees of useful implication, useful causation, etc etc.
This kind of goal-oriented reasoning was implemented in Webmind, it is not
yet implemented in Novamente, but it is certainly in the design.
> To give a concrete example, Webmind (as I understood it) would, on seeing
> that rooster calls preceded sunrise, where sunrise is desirable,
> would begin
> to like rooster calls and would start playing them over the
> speakers.
Eliezer, this is a complete misunderstanding of the Webmind system's
approach to causal inference.
Of course, we did not fall into such an elementary error as this. In fact
this rooster/sunrise example is specifically discussed in the Novamente
documentation.
I do not blame you for not understanding the Webmind AI Engine, or the
Novamente system, since you have never been given any kind of systematic or
comprehensive documentation for either of them.
However, you seem to have formed a rather incorrect view of both
architectures, based on conversations with myself and other Webmind Inc.
staff.
Interestingly, all of your misconceptions seem to have a common pattern: In
each case, you mistakenly assert the belief that the Webmind/Novamente
architecture embodies some basic conceptual mistake, which in fact it does
not embody.
When you finally see the full Novamente design, you will find that it does
not embody any such elementary conceptual mistakes.
If the Novamente design fails, it will not be for such simple and obvious
reasons. It will rather be, because our approach to some component of
intelligence is insufficiently computationally efficient.
> CFAI
> would try playing a rooster call, notice that it didn't work, and
> hypothesize that there was a common third cause for sunrise and rooster
> calls which temporally preceded both (this is actually correct; dawn, I
> assume, is the cause of rooster calls, and Earth's rotation is the common
> cause of dawn and sunrise); after this rooster calls would cease to be
> desirable since they were no longer predicted to lead to sunrise. Maybe
> Webmind can be hacked to do this the right way; given the social process
> that developed Webmind, it certainly wouldn't be surprising to
> find that at
> least one of the researchers thought up this particular trick.
This is not a "trick", this is a standard idea from the causal inference
literature which is at least dozens of years old, and which was considered
at the very start of the design process for the Webmind AI Engine causal
inference module. It is covered in the review paper on causal inference by
Jeff Pressing which I sent you some years ago.
Perhaps you failed to grasp the Webmind causal inference module because the
language used to discuss it was different than the language you tend to use
to discuss causality. Or perhaps because the design was overcomplicated;
the Novamente design is a bit simpler but accomplishes the same thing.
> My point is
> that in CFAI the Right Thing is directly, naturally, and
> elegantly emergent,
> where it goes along with other things as well, such as the Right Thing for
> positive and negative reinforcement, as summarized in "Features
> of Friendly
> AI".
>
> So that's what I would offer as a demonstration of the second claim.
I have not seen you produce anything resembling a demonstration that you
have a design for an AI system capable of correctly doing causal inference.
I have not provided you with such a demonstration either, regarding my own
AI systems.
One difference between our claims, however is:
-- I am only claiming that what I have (and have not fully revealed to you)
is a *probably workable* approach to seed AI
-- You are claiming that what you have (and have not fully revealed to me)
is in some sense the *uniquely best* approach to seed AI
Your claim is much stronger and would thus seem to require much stronger
justification.
> When I was at Webmind I could
> generally convince someone of how cleanly causal, Friendliness-topped goal
> systems would work, as long as I could interact with them in person.
Well, Eliezer, after you left Webmind Inc. following your brief and
enjoyable visit, no one seemed to be jumping up and down to revise the goal
architecture in accordance with your ideas.
I honestly don't think that anyone who talked to you during your visit to
Webmind Inc. was *convinced* by your ideas. Some were intrigued, and some
thought they were absurd. But intrigued is not the same as convinced.
My role was definitely one of advocate in this case. I tried very hard to
get the others to take your ideas seriously, and it was only possible in
some cases.
> Right; what I'm saying is that nudging Novamente's architecture into
> hierarchicality is one thing, and nudging it into Friendliness is quite
> another.
Sure.
The two concrete things you've pointed out are:
1) the links in the hierarchy in question should be composed of
ImplicationLinks not AssociativeLinks (true enough, and that was my
proposal)
2) the nodes in the hierarchy should be "specially shaped for Friendliness"
(I'm not so sure what this means; there are ways to implement such things in
Novamente but I'm not at all sure of the necessity)
>Incidentally, if you think that the CFAI architecture is more
> "rigid" than Novamente in any real sense, please name the resulting
> disadvantage and I will explain how it doesn't work that way.
I will, but not in this e-mail, which is long enough...
> A directed acyclic network is not *forced* upon the goal
> system; it
> is the natural shape of the goal system, and violating this shape
> results in
> distortion of what we would see as the natural/normative behavior.
I guess this gets to the crux of the matter.
According to my own measly human intuition, this view of goal systems is is
as dead wrong as dead wrong can be.
In my view,
1) The "natural shape" of a goal system is an unstructured heterarchical
graph.
2) A working goal hierarchy ("directed acyclic network" if you prefer) must
emerge from a goal heterarchy without structural constraints.
I feel that you want to *force* what should *emerge*.
> What
> is the system property that requires continual intense interaction to
> enforce, and how does the continual intense interaction enforce it? Or
> alternatively, what is it that requires continual intense informational
> inputs from humans in order to work right?
There may be many things in this category.
One of them is, as I said: Ensuring that the system's Friendliness
ConceptNode remains reasonably well-aligned with the human concept of
Friendliness, rather than drifting far astray...
-- Ben G
This archive was generated by hypermail 2.1.5 : Wed Jul 17 2013 - 04:00:37 MDT