The next thing we started looking at was what we call statistical learning.

And this will culminate in actually learning the parameters of Bayesian networks.

Remember, Bayesian networks were models which, once we have them, are nice to predict certain things, to make decisions.

We've understood most of how to do essentially decision theory with Bayesian networks.

The problem, of course, is that the assumption that somewhere God sits and puts Bayesian networks into agents' minds

is religiously loaded, let's say, that way.

Okay, some people have no problems imagining that, and others do.

So in AI, we always take the path of least assumptions.

What we really need to do is we need to be able to say, if we have these Bayesian networks as good models, where do they come from?

And the answer, of course, is as always, learning. Yes, of course.

So we started looking at statistical learning.

And so the thing is, how do we estimate these kind of parameters in Bayesian networks and then use them for prediction? Or can we even take shortcuts?

And that's what we want to look at now.

So essentially what we've been doing is looking at learning, not as something where we have to decide on a current best hypothesis.

But basically, instead of just singling out one hypothesis, we look at a probability distribution over all hypotheses.

So you basically, instead of saying the zero one question, you have this huge hypothesis space, and you say, this one, that's the one.

That's so last semester.

So what instead we want to do is we want to say, well, but we don't know everything, so let's just do a probability distribution over this.

Where that's high, we might just look at certain things.

This is good because it also lends itself well to approximative techniques.

So if you approximate in the kind of zero one thing where you have exactly one best hypothesis, then just approximation will give you something near, but you don't really have a clue how well this fits.

And so this is the idea of full Bayesian learning is that you say, well, we'll just make a probability distribution.

What's the real need for that?

And obviously, if you want to have a probability distribution over hypotheses, we give ourselves a random variable H, which is the random variable for the hypothesis, acknowledging that we have no idea what it might be.

But we can compute the probability.

And so we have to assume that there's some kind of a prior for this random variable, which in the grand scheme of things we might know or might not know.

OK.

And then we make observations, which is essentially exactly what we've been doing before.

We have a random variable, which we know certain things about.

We have observations and then we're interested really in the posterior probability that for any hypothesis H I.

How probable is that given the observations?

OK.

Same game we've always played.

We've got slightly different wrinkles about where the conditional in dependencies are and those kind of things.

And we can we can compute that as always by Bayesian techniques, right, turning around the posterior probabilities.

Normalizing.

We have something where we have the prior, where we have the likelihood, right, the likelihood of the data under a certain hypothesis.

And that gives us this, which we're really interested in.

Well, actually, we're not really interested in what the right hypothesis is.

We want to make predictions.

But if we can compute this posterior probability, then we can make predictions in the usual way.

What we're interested in, we have some kind of a data in vector X.

Given the observed data D, we can just compute by summing out.

We have an independence here that we can use and get this term.

Right.

All of that looks nice.

And indeed, in our examples, remember we have this candy example where we had.

Where the crucial fact was that the candies come in big bags and there are a couple of kinds of bags in this case, five,

which will be our hypothesis.

We will hypothesize about what kind of a bag do we have.