Now, we want to extend that and the upshot of everything we did last time is we need

to extend this model to not a model of the world state, but since we don't know the

world state and cannot know the world state, to what we believe about the world state.

And that really needs two things, right, we need to have a belief state, what do we believe

the world is like, and we need to extend our model to allow multiple possible worlds, because

some things we don't know, so we have to kind of plan for both contingencies, and to have

a transition model that actually updates our belief state with new information coming in.

For instance, if I'm unsure whether I'm on, I don't know, I can see outside, so, if I'm

unsure whether I'm on the bottom floor of this building or there's something beneath,

right, I need to have a plan for both contingencies, but I can also just basically go down, see

whether there are some stairs and look what's beneath here, and then I can actually restrict

the number of possible models, making my belief state more accurate.

But of course I have to plan for my actions being unreliable, and when I thought I was

going down, I was actually going up, and get false information.

And of course the environment tells me which of those things I really have to do, if I

have full observability, then my belief state can become a world state, and if I have reliable

sensors and actions, then the transition model can actually be deterministic.

So essentially we're looking at a generalization of the agents we were talking about naively

last semester.

So here we have basically, we've reviewed our kind of agents we talked about last semester,

isn't it nice that we can kind of condense a whole semester onto one slide.

So we look at what our environments are like, they're all fully observable, and what our

world states are essentially, and how we do the updates.

We do it either without any inference or with constraint propagation, or with full logic

based inference, and the planning was different from the first couple, in that we had a dynamic

environment which could actually change, which is the whole point of planning.

We're planning for the changes, so we have to have all these delete and add lists and

all of those kind of things.

So the inference is basically state or plan based search, and we have a transition model

which is essentially strips, meaning I can describe the world by a couple of assertions,

and some I have to leave out after an action, and some I have to add after an action.

Right and what we're going to look at now is essentially the generalization, which we

call probabilistic agents, where we have a belief model, which is essentially a set of

possible worlds, together with their likelihoods of being true, we're going to use probability

theory for expressing that, and we're going to come up with a inferentially efficient

representation here, which means we can actually do inference on this difficult problem, and

the inference of course is going to be probabilistic inference.

Which means we're going to learn about probability theory first.

And then we are going to extend those to what we call the decision theoretic agents, which

are a special kind of utility based agents that actually try to predict how good the

actions are and tries to optimize the positive effects of their actions.

We've essentially just set the stage, so we know what kind of the setting is in which

we are actually now going to extend our repertoire of theoretical methods.