So essentially what we did was a vocab lesson, right? One, two, three, four, five, ten or

so new concepts about environments. We're calling a, so one of the questions you can

ask yourselves about environments is how much of it can you observe? Is there anything,

any essential information about an environment that you don't have? And usually there is.

Only in very artificial environments, like playing chess or something like this, do you

have fully observable environments. And needless to say that our algorithms actually like fully

observable environments because that makes them easy. If you know everything, you can

take everything into account, you can optimize. If you know nothing, optimization is very

difficult. The real world is somewhere between knowing everything and knowing nothing. Depending

where you go and whether it's foggy or dark or something like that, it's more here or

more there. But our algorithms need to be able to cope with both. And we're going to

start with the easy algorithms obviously. So we're going to go for fully observable

worlds. We can ask ourselves, even if we can observe, what will our actions do? Will our

actions always succeed? Will my sensors always give me the right information? Is an action,

does that actually give me the next state in a deterministic way or in a stochastic

way? That is actually important to know how the world evolves. And again, deterministic

worlds are easy. We're going to prefer those for the moment.

Then there's the question, and once time is involved, is it that I can kind of stop time

for a while, do something and not care about what goes on in the world around me, and then

kind of look at the world again? And of course, the real world is sequential, whereas we prefer

episodic environments if there is time involved at all.

And then there's the question of whether an environment is dynamic or static. Dynamic

means something happens without you actually doing it. And needless to say, all interesting

environments are dynamic. Then we have questions about being discrete. Can I describe the environment

as a countable set of states? The answer is usually no, but for some environments like

the ones we're going to look at next, for instance games or something like that, this

is the case, which is why we do search and games first.

And finally, you can ask yourself how many agents are there in this world? Is it just

you? Or is there an opponent in your game? Or are there crowds of other things, of other

agents that do stuff?

And of course, if you think about say a taxi driving agent, then of course there's lots

of other cars. So you have to kind of see what they're doing. Will this guy break when

he's coming onto my road? Probably yes. Let's go on. Those kind of things become interesting

in a multi-agent world where you actually have to kind of reason about and find out

what might the plan of this other agent be. And does it hinder me or does it help me?

Can I take advantage of it? That's all.

Those are things that you want to ask yourselves. Before you start an AI project, right? Start

an AI project, something like build a Go player. In essence, you're building an agent, a Go

playing agent. And you have to ask yourselves what kind of environment is this Go player

going to behave in? And depending on what your answers to all these questions are, these

classifications, you will need different algorithms.

We're going to look at what we call search agent, search algorithms, search based agents,

at first. And they're kind of almost the same as game playing agents, except that they're

the single agent version of it. You fight against chance or intrinsic difficulties or

something like this in search problems. Whereas in say chess, chess is not only intrinsically

difficult but there's also somebody else who makes life especially difficult for us by

trying to win.

That's why we want to classify the environment. We looked at a couple of examples.