Last semester, we basically looked at agents that lived in what we called a deterministic world.

And we all know and we all talked about that this is very unrealistic.

So what we want to do this semester is actually look at more complex environments that are not deterministic anymore.

And what do we have to do to that? It will turn out that logic is not the answer.

Surprise, surprise. So let's see where this comes, right?

If we go back to the Wumpa's world.

This is kind of an idealistic view of the thing where actually there's light in the cave.

That's so that you know what's going on.

But for the agent here, it looks different.

They cannot see the whole cave and see where the Wumpa's is and where the pits are and so on.

So in a real Wumpa's world, there's lots of uncertainty.

If there wasn't, it wouldn't be any fun.

So the only things you can sense, so the agent cannot sense where the pits are.

They cannot sense where the Wumpa's is. They cannot sense where the gold is.

And they can't, they don't, they can sense certain things.

They can sense whether there's a breeze in one cell, but only if they're in that cell.

So at the moment, the only thing the agent knows is I'm in cell 1-1.

And I'm safe because it doesn't stink and there's no breeze.

That's all the agent knows.

And it's uncertain about where the Wumpa's is, where the pits are, where the gold is.

That's something we want to study.

Now it must make a decision in the midst of all this uncertainty.

It must move, and it can move, I think, in those two cells.

And then it knows more, but it's still uncertain about all the important stuff.

But it has to make the decision whether to go there or where to go there.

And depending on the environment, it may even know less.

It might not even know where it is.

In an environment where your actions are uncertain, where you say, oh, I'll go forward one.

Whereas, in fact, you went sideways.

When your actions don't always succeed, you lose the ability to be sure where you are.

And in this Wumpa's example, we've kind of implicitly assumed that we have reliable sensors.

But what if the agent has a cold and his nose is blocked?

Then you don't know whether there is a stench.

You might think it stinks, but that might be wrong.

Or you might think that the air is clear and it actually stinks, and you'll be eaten by the Wumpa's next.

So we have various realistic worlds, even such a very simple one.

We have various sources of uncertainty.

Our sensors, our actuators as an agent, might be unreliable.

And you might also be completely mistaken about the rules of the world.

Imagine going onto your car, driving over to the Czech Republic.

And you know in Germany, there's no speed limit.

But you've crossed the border.

How fast can I go?

Is it 100, 130, no speed limit?

I don't know.

So you might have a very different wrong model on how the world actually works.

Typical thing, one of the big problems of self-driving cars, for instance, is the car needs to know where it is.

And where on the road it is, where in the city it is, in which country it is, and all of those kind of things.

And then you might want to do certain things as an agent.

Say your GPS broke down.