Let's start.

So yesterday we essentially completed the first part of building probabilistic agents.

Remember, we're in a situation.

Where we have an agent interacting with the environment.

It's a model-based agent which really models the world as a probability distribution over the possible world states.

And somehow that's influenced by the percepts.

And really the things we're interested in is the probability of something or the other that might be helping me to choose an action.

Given a bunch of random variables that correspond to my observations.

The EIs come via the percepts. That's the idea.

And all the machinery is just to fuel those things.

And we know how to basically, when we design an agent, to set up a Bayesian network, get a Bayesian network's library or program one yourself.

You can do that by now.

And we can maintain a world model.

The next thing to look at is how to do decisions.

And the trick in this is really to think of the actions as special random variables.

It's kind of a trick we use so that we can keep using the same math, the same implementation.

All along.

And so we're really going to ask ourselves what's the probability distribution of something, the world state, given that the action is whatever the possible actions are.

Okay? So nothing new.

Just the same as always.

And so we want to take the best action.

And the main thing we've been talking about is what does that the best even mean?

And we were talking about utilities.

So mathematically it's very easy.

We just assume a utility function falls from the heavens.

And we can do the math keeping the utility function under specified.

We can do the math for any of those.

Doesn't help AI people a bit.

Because if we design a concrete agent, the utility function is an input.

Meaning you have to supply it.

And so that is kind of our problem.

And so the things we've been doing is basically thinking about where could the utility functions come from.

And how does, if we have one, does that work?

So we've been talking about the expectation of a random variable.

And we have been building our way towards this maximizing expected utility thing.

What that means mathematically.

And that's really relatively simple.

Okay?

We're just basically treating this as a Bayesian network essentially.

The actions of a random variable.

Given a utility, we know what the world is going to look like.

Or what the utility, what the probability distribution is like given our observations.

And given any of the actions we're choosing.

And we're just basically going to use the utility function as the function we're waiting with the world state.

Given the observations and the actions.

And that gives us a number.

And then we're just going to take the argmax of that.

Mathematically trivial.

Right?

Even the expectation, which is a big formula out there, is mathematically trivial.