From: Matt Mahoney (matmahoney@yahoo.com)
Date: Thu Apr 24 2008 - 12:33:23 MDT
--- Lee Corbin <lcorbin@rawbw.com> wrote:
> Matt writes
>
> > But my point is that if you expect AGI to make boundless happiness
> possible, I
> > think you will be disappointed. How would you program a brain, or any
> > intelligent system, to experience accumulated happiness that grows without
> > bound?
>
> We don't know at present. We assume that 30 or 70 years from now
> happiness circuitry would be much better understood.
>
> And then in the next email, Matt writes
>
> > John K Clark wrote:
>
> > > I'll just change the mean happiness quotient in my code from a 5 to a 6,
> > > oh yes that's much better, I wonder what a 7 would feel like, wow that
> > > was even better than I expected, 8 is really not that much greater than
> > > 7 so it couldn't hurt to..
> >
> > You'll have to stop at BusyBeaver(2.91 x 10^122) (Bekenstein bound
> > of the Hubble radius).
>
> Very good! Few people understand the philosophical significance of
> the Busy Beaver. Add just *one* neuron and the capacity of a system
> is potentially increased by an enormous factor. We cannot begin to
> imagine just how much pleasure a cubic meter of material could be,
> or how smart it could be. Much less a Jupiter brain.
>
> So I do not take "unbounded" literally. It really cannot be taken
> literally. The reason for this is that the speed of light is constant.
> If a brain gets too big, it ceases to be a single entity.
(One correction. The Hubble radius is not constant, so you don't have to
stop).
I proposed a bound on happiness (or unhappiness) of K(S2|S1), where S1 is the
state of an intelligence before the reinforcement signal, S2 is the state
afterwards, and K is Kolmogorov complexity. It is intuitive in that a
stronger reinforcement signal induces a greater change in mental state (as
measured by the length of the shortest program that describes the change), and
that it is not possible to experience happiness without memory. By this
definition, happiness would be bounded by the complexity of the intelligence.
-- Matt Mahoney, matmahoney@yahoo.com
This archive was generated by hypermail 2.1.5 : Wed Jul 17 2013 - 04:01:02 MDT