There's one more thing.

So we've seen kind of the world model part on top relatively high up in the agent, right?

Things that answer questions like how's the world now?

What action should I do now, given the world state?

What do my actions do?

What's the world probably going to be like in three hours from now?

Those are things that if you want to have one of these higher, more versatile agents

that you have to answer, that you actually have to program.

And there the question is, the state of the world, how do you represent that?

What are the objects that you use for the state of the world?

And there's essentially three ways to do it, broadly spoken.

One is you can have atomic states, right?

You're in a situation and you have 3N plus K distinct states your world can be in, and

you call them state one, state two, state three, state four, and so on, up to 3N plus

K.

Okay?

That's perfectly fine.

The problem with this is, and the advantage of this is, that you cannot look into the

states.

You know that if I'm in state 317 and I see a red blinking light, the world is probably

going to be in, the world is going to be in a, say, static and deterministic environment,

is going to be in state 512, and if the world is in state 512, I run away.

Okay?

Very simple agent.

And what you basically have is kind of a graph with nodes that you can identify, but you

cannot look into, and you reason essentially about the connections of what state can follow

what other state if a certain percept happens.

The nice thing is you can just basically have very simple algorithms for that.

The bad thing is typically there are lots of states, right?

Things of them, which make even very simple algorithms difficult to do in practice.

The opposite to that is to have world states that you can completely look into.

Think about, you can think of these here as black box states.

You cannot look into them, and these here, you have crystal block box states.

You can see everything.

You have lots of things, objects in there, and they're related in certain way, and so

on and so forth.

And you can basically single out, say, instead of talking about state 317, you may actually

choose to look at what happens to this little piece of the state.

And if that piece of the state doesn't change, you don't have to worry.

The advantage of this is that by just concentrating, say, on a little piece of the state and disregarding

everything else, you can talk about thousands or billions of states at the same time.

Rather than having to talk about every single state, you can actually, when you look into

them, you can concentrate on certain things.

And there are things in between which we call factored, which is basically you, instead

of having objects there that can in and of themselves have certain states and have changing

relations between them, you basically have what we call factored relations, which are

essentially states where you cannot look at, where you have sub-objects and can reason

about the values of them, but not how they internally change and how they're connected.

And it looks like this is the best thing.

But of course, to actually maintain such a thing costs you a lot.