The next topic we have to look at is base law, because base law is the mathematically correct way of updating your belief about reality if data comes in.

That's something that is very important in data science.

We will start with the so-called non-singular case, the elementary discrete case,

and then in the next lecture we will talk about random variables and how to condition them,

because that's a bit harder.

But we will try to build up our intuition about conditioning and about conditional probabilities

and about base law in a discrete context, because then we can count things and understand things more clearly.

So, again, the obligatory probability space, triple here, which you can totally ignore.

So we take two events, A and B, and we have to assume that those two events have non-zero probability.

This is something that will trouble us later, but not for now.

So let's say you have two events and they have not zero probability,

and then we can define the conditional probability of A given B as just this quantity.

It's just a definition. There's nothing magic about that.

Something we can compute and that's it. That's just a conditional probability.

And base law is a beautiful thing that we can invert this conditioning,

and the probability of B given A is the same as the probability of A given B times probability of B divided by the probability of A.

We call this the posterior, and this we will call the prior.

So what is that?

The prior is something that we start with. So we have some belief about B.

Let's say B is the event that it's raining.

So we have some prior belief about the weather tomorrow.

And this conditioning is the correct way of updating your inner probability distribution about the weather tomorrow in the light of some data A.

So maybe someone, a trusted friend of you tells you, well, I've seen it in the news that the weather will be bad tomorrow.

So that should, of course, change your inner probability distribution about the weather tomorrow.

And the correct way to do that is the following.

So you take the prior, which is the old probability distribution.

You multiply this by the so-called likelihood, which is the probability that the data was generated in the event that B is true.

Then we have to normalize by this factor of P of A, the evidence.

So those are at this point, completely arbitrary names.

So why should we call this a likelihood? Why should we call this the evidence?

This will become clearer later.

I can recommend that you learn those names as well, because that will make thinking about more complicated ideas more easily.

So why is that true?

Baselaw is very quickly proven. Probability of B given A is by this definition the same as the probability of A and B divided now by the probability of A.

This is commutative, so we can just change the order.

And this means the probability of A and B is the same thing as the probability of A given B times the probability of B.

And that we plug in here and then we get to that.

So this is we get those two terms instead of that term. That's the whole proof.

OK.

So far, I have not motivated why this is why this has anything to do with taking information into account.

It was it was just purely mathematics. But the next example, which we'll look at, will hopefully help you make this more intuitive.

There's a second form of baselaw. It's not really a second law, but it's a more detailed version of baselaw, which will be very important.

And that is making this so-called evidence explicit, because often this will be something weird.

So we'll see in another context why this is weird. So this is usually something which has too little information.

So what we have to do is we have to specify this decomposition. So this is this is true.

So the probability of A is the same thing as the terms down here.

We can also quickly prove that.

The first thing to notice is that A is the same thing as A and B. And this is the joint, disjoint union with A and B complement.

So why is that true?

That is A, that is B, then A and B. That's set.