Welcome everybody to the Monday morning session. As usually Monday morning we will briefly sketch

the big picture such that you do not get lost with a storyline and then we will dig into one

learning algorithm that is round about 40 years old but a fundamental principle in pattern analysis

machine learning the Rosenblatt's perceptron and it's quite technical to a certain extent so be

prepared that we will see a proof that the algorithm converges brief sketch of the proof

because that's important for the general understanding. So what is pattern analysis

the summer semester about? This is all about modeling or estimating the a posteriori probability

for the class Y given a feature vector X right and what we have considered was first logistic

regression as a method to directly model the a posteriori probability using the so-called

sigmoid function just for you to remember what is the shape of the sigmoid function it looks like

a step function. That was a direct representation of the a posteriori probability and we also have

seen that we can right away read out from the sigmoid function the decision boundary. Then we

looked into the modeling of the a posteriori probability using the following factorization

P of Y times P of X given Y so we have the prior and here we have the class conditional

density function and we have modeled the class conditional density function in different ways

for instance we have seen that we can use a Gaussian for that and implement a Gaussian

classifier based on this assumption and while looking at the Gaussian classifier we found out

that we get linear decision boundaries we also talked a little bit about PCA data normalization

how to get spherical data and all that stuff so basic algorithms that are partially known

from pattern recognition in winter semester at least to a few of you. So very important section

where we combine PCA LDA we also have reconsidered basic concepts from linear algebra like the SVD

type of normalization and last week we had a very technical lecture on norms and norm approximations

and it was important to give a brief overview of different norms vector norms matrix norms

because as soon as we set up minimization problems to estimate parameters we have to deal with a

similarity measure and the similarity measure can be based on different norms we have seen the L0

type norm that is more or less the the hemming distance you know from from information theory

we have seen the L1 norm that's the sum of the absolute values we have seen the standard L2 norm

everybody who is not knowing anything about norms it's usually using the L2 norm to set up

his optimization tasks we have seen the L infinity norm we also have seen the quadratic LP norm so we

have seen a whole bunch of norms and we have looked at unit balls and the related problems

we also looked into constrained optimization just remember the rich regression that we have

considered and the rich regression and the lasso tip shirani's lasso we have considered where we

use the L1 norm combined with the L2 norm that bounds the length of the vector so there are

various degrees of freedom and I also pointed out that in modern pattern recognition and modern

pattern analysis research the usage of the norm is a hot topic and many different algorithms are

or many new algorithms are born out of the idea replace the Euclidean norm by any other norm and

solve the corresponding optimization problem and the fact that these norms are more and more

successful in our community is related to the fact that optimization theory has done a huge

progress in the recent ten years or so so convex optimization convex constraint optimization problems

can be solved today very easily using the interior point method some of you might have heard of if

not I will give a brief introduction in one of the following lectures so that's the big picture and

basically we have considered linear decision boundary for two class classification problems

so we are still more in well more academic type of questions in in pattern recognition because in

many applications you have far more than two classes but to understand the basic concepts I

think the two class problem is sufficient and we also have seen that for a given set of training

samples you will find an infinite number of linear decision boundaries if they are linearly

separable and we have seen different ways to compute these linear decision boundaries and

today we will look into Rosenblatt's perceptron this is a very old neural network type of approach

that computes linear decision boundaries but it's a very smart way to compute that and I think it's

still worth presenting this method in the lecture here so I have a new presentation software let's