RE: Seed AI milestones (was: Microsoft aflare)

From: Ben Goertzel (
Date: Wed Feb 27 2002 - 11:15:12 MST


> (1) and (2/3) seem to correspond to the distinction between weakly
> self-improving processes - i.e., learning processes - and strongly
> self-improving processes.

Yes, this is basically right, although I'm not so sure you've drawn the
distinction quite carefully enough.

Nor did I in my post, and nor will I in this post -- maybe later; it will
take some careful thought.

You say
> Both cases of
> "weak" self-improvement involved the operation of an external
> improvement process, already powerful, which did not further
> increase in power during the timespan considered.

Sure. Human learning *does* improve its ability to improve itself, over the

But as you say, there is a background learning process, i.e. brain dynamics,
which is not changed, but which enables the self-modification on the level
of neurally-based thoughts & procedures.

> The difference between (2) and (3) seems to be
> the difference between consciously modifying cognitive content, and
> consciously modifying source code.

Yes, this is reasonable.

But I also think that the difference between "cognitive content" and "source
code" is fuzzier than you explicitly make it out to be here.

When the cognitive content in question is procedural knowledge, perhaps very
abstract procedural knowledge, then it may effectively serve as "source
code" for a wide variety of mental procedures...

I don't disagree with your re-phrasing of my distinction, but nor am I sure
your rephrasing is any clearer. It just involves *different* ill-defined
terms ;>

I would like to write something more clear and careful on this topic, and
may do so later when I have more time for it...

-- Ben

This archive was generated by hypermail 2.1.5 : Wed Jul 17 2013 - 04:00:37 MDT