We're still deep in probabilistic reasoning, this time probabilistic reasoning with special

considerations of time. We've been talking about inference procedures for Markov chains

and in principle they can also do higher order Markov processes but we've mainly done

Markov chain because that's the, I would say, good modeling decision. You're always

trying to get it down to Markov chains possibly by introducing you random variables.

And we've talked about the three main algorithms. The main one is essentially filtering which

is kind of doing state estimation forward taking the existing evidence into account. There

are two kind of extensions to this one is prediction which just means we're overrunning

the current time. We don't have evidence but we still keep doing the same, the other

variant of this is smoothing where we basically take the evidence and run it backwards to

get better estimates of the state of the world in hindsight. And we briefly talked about

the Vitalee algorithm which really does similar things but based on sequences rather than

on single states at single time points. We looked at the algorithms and the math behind

this and the most, I would say, important algorithm was kind of the forward backwards algorithm

where you do the filtering steps forward and if necessary in smoothing run the backwards

coming back to the past. And in the most likely explanation of the Vitalee algorithm,

we're doing kind of the same with similar ideas only that. We're recursively descending

with a slightly different term in the recursive equations. Here basically the real difference

is whereas in filtering and smoothing and prediction we have a probability of a single

state we're really looking at the probabilities of sequences given the evidence. Make things

a little bit more difficult because we essentially have bigger distributions but conceptually

it's not such a big step. In all cases we have very simple recursive equations which

gives a very simple recursive algorithm. Okay, and I have better pictures now. Right.

The second thing we talked about yesterday is that if we're prepared to go down to one

state variable and one evidence variable we can reformulate all of those in terms of matrices.

This is typically much better implemented in software, I think numpy or matlab or something

like this, than doing unspeakable things to conditional probabilities. Okay. So this

is actually something very nice. We can implement a whole solver in essentially two lines of

code if we have a matrix capable system in the back. That is very, very useful. It gives

you lots of opportunities for experimentation. And there are nice applications we looked

at the robot localization example here and see that these things converge nicely. And

a lot of practical things are done exactly this way with hidden Markov models if one in

one out variable are enough for you. Okay. And sometimes you can even do better, but that's

an improvement, a practical improvement. Okay. The next thing we looked at or we started

looking at was dynamic base networks. Now you might ask yourselves why are we doing this?

We already have hidden Markov models. One to remind you of the fact that one of our example,

the umbrella example which is not really a real world example which was picked to be easy

to talk about on slides or in box. That is indeed one state variable, one evidence variable,

we can do hMM how nice and we did. Okay. But our other example, robot motion already has

one to three state variables and one to evidence variables. Okay. We cannot do hMMs directly.

And this is not a very big model actually, right. Think about our decision diagrams from

medicine that had 50 or 60 nodes in them, all most of them actually state variables. Every

diagnosis of doctor runs is an evidence variable. Okay. So you typically don't have just one

state, just one evidence variable. So what can you do? And the typical thing we're doing

here often is that we're saying yeah, but we can do a little trick and then it becomes

an hMM. Right. The trick here is to say ah, but it's easy. Instead of having three variables

here, we can have one variable for a triple. Okay. Easy peasy. It's an hMM after all. Okay.

Now sometimes that's a good idea. Doing these tricks and sometimes it's not. And if you

think about our trick from a Bayesian network point of view, right, namely what depends

on what. And if you look at all of those things here, these arrows, right. Every one of these