Re: [sl4] Re: Uploads coming first would be good, right?

From: Johnicholas Hines (
Date: Tue Feb 24 2009 - 13:58:38 MST

On Tue, Feb 24, 2009 at 1:47 PM, Charles Hixson
<> wrote:
>> On Mon, 23 Feb 2009 "Johnicholas Hines"
>> <> said:
>>> My claim is I think we should consider the expert
>>> opinion that "there's no weather underneath" as relevant
> This isn't clear. If it models how clouds change accurately enough I would
> assert that temperature is implicitly encoded into the model. And if it's
> accurate *enough*, then it might be a good local model of weather. And if
> it's accurate ENOUGH, then it might be a reasonable datafeed for your World
> Weather Watch.
> Think of neural nets. They don't have separate variables for separate
> measures, and they don't have intelligible internal workings. But if you
> train one well enough, it produces reasonable results without having the
> separate variables that we have decided are good descriptors. Like
> temperature.
> Personally, I want to be able to understand my models, but I don't think of
> intelligibility as an essential property of a model.

Yes, an expert could extract an implicitly encoded temperature from
only images of clouds. Forrest Mims seems capable:

Yes, it is possible to build faithful models that have confusingly
different internal workings. (Of course, there is some explanation for
why the model is faithful. If we knew that explanation, then they
would be intelligible internal workings.)

Lest we get confused, let me recap. We're talking about criteria for
how one might determine that an emulation is a faithful emulation.

Dr. Mahoney pointed out that it might be possible to build a computer
program from only the public record that emulates a person
sufficiently well to convince their friends and family. My worry was
the claim that "good enough to fool your friends and family" was the
only, or primary, criterion for fidelity.

The weather metaphor goes like this:

wet weather : emulated weather :: wetware humans : emulated humans

The point of making the weather metaphor was to claim that we wouldn't
use the "friends and family criterion" as the only, or primary,
criterion for whether to trust a weather simulation. We would also use
experts. If we're interested in simulating never-before-seen
situations such as extreme global warming, then the experts (at least
in the near term) need to see the construction in order to argue that
the results of the simulation are correct by construction.



Though it might be possible to build a faithful emulation of a human
from only the public record, I don't think it's likely to happen soon.
Let me try to convince you of that, using two points:
1. A faithful emulation should act correctly even in quite extreme,
never-before-encountered situations. Your friends and family haven't
seen you in these situations, and lots of different behaviors would be
2. A faithful emulation should enact firm plans and intentions that
you made immediately before uploading. Guessing those plans and
intentions based on the public record might be possible, but I think
it would be quite difficult.

This archive was generated by hypermail 2.1.5 : Wed Jul 17 2013 - 04:01:04 MDT