So yesterday and starting last week, we talked about agents and environments. An agent being

an entity that can sense the environment, like you can, and can act on the environment.

And the interesting thing is what's in there. That's the AI question. How can we build agents?

And when it's about sensors and actuators, AI collaborates with people from mechanics

or these kind of things, in robotics for instance. But the main question of AI is how can we

build agents? And if you think about it, that's all AI needs. We want to make intelligent

agents. But we don't know what intelligent is, so we'll settle for making rational agents,

because that empirically comes very close to intelligence. And rationality is something

we have a much better handle on. That gives us a way of going forward, building stuff.

And so we first want to understand what agents are. And you talked about that last week a

little bit. And we can characterize agents' tasks by these four things, the performance

measure and environment and actuators and sensors. And what we're going to do, what

we started doing yesterday, was looking at what kind of environments there are.

So essentially what we did was a vocab lesson. One, two, three, four, five, ten or so new

concepts about environments. So one of the questions you can ask yourselves about environments

is how much of it can you observe? Is there anything, any essential information about

an environment that you don't have? And usually there is. Only in very artificial environments,

like playing chess or something like this, do you have fully observable environments.

And needless to say that our algorithms actually like fully observable environments, because

that makes them easy. If you know everything, you can take everything into account, you

can optimize. If you know nothing, optimization is very difficult. The real world is somewhere

between knowing everything and knowing nothing. Depending where you go and whether it's foggy

or dark or something like that, it's more here or more there. But our algorithms need

to be able to cope with both. And we're going to start with the easy algorithms obviously.

So we're going to go for fully observable worlds. We can ask ourselves, even if we can

observe, what will our actions do? Will our actions always succeed? Will my sensors always

give me the right information? Is an action, does that actually give me the next state

in a deterministic way or in a stochastic way? That is actually important to know how

the world evolves. And again, deterministic worlds are easy. We're going to prefer those

for the moment. Then there's the question, once time is involved, is it that I can kind

of stop time for a while, do something and not care about what goes on in the world around

me, and then kind of look at the world again. And of course, the real world is sequential,

whereas we prefer episodic environments if there is time involved at all. And then there's

the question of whether an environment is dynamic or static. Dynamic means something

happens without you actually doing it. And needless to say, all interesting environments

are dynamic. Then we have questions about being discrete. Can I describe the environment

as a countable set of states? The answer is usually no, but for some environments like

the ones we're going to look at next, for instance games or something like that, this

is the case, which is why we do search and games first. And finally, you can ask yourself

how many agents are there in this world? Is it just you? Or is there an opponent in your

game? Or are there crowds of other things, of other agents that do stuff? And of course,

if you think about say a taxi driving agent, then of course there's lots of other cars.

So you have to kind of see what they're doing. Will this guy break when he's coming onto

my road? Probably yes. Let's go on. Those kind of things become interesting in a multi-agent

world where you actually have to kind of reason about and find out what might the plan of

this other agent be? And does it hinder me or does it help me? Can I take advantage of

it? And so on. Those are things that you want to ask yourselves. Before you start an AI

project, start an AI project, something like build a Go player. In essence, you're building

an agent, a Go playing agent. And you have to ask yourselves what kind of environment

is this Go player going to behave in? And depending on what your answers to all these