Welcome back to Pattern Recognition. So today we want to start looking into a special feature

transform that is called the independent component analysis and essentially today we want to

introduce the idea and why independent components may be useful in terms of a feature transform.

So let's have a look at our slides and the independent component analysis tries to address

the cocktail party problem. So here imagine the situation that you have two microphones at

different locations and these microphones record some signal x1 and x2 that is dependent on the

time. Now each recorded signal is essentially a weighted sum of the speakers in the room. For

simplicity let's assume that there are two speakers in the room and we can model this

essentially as a weighted sum of the two speakers what we collect in microphone one and in microphone

two. So the parameters depend essentially on the distance between the microphones and the speakers.

So now we would be interested of course to reconstruct the signal from the two speakers

and this can be done using factorization methods and in particular the idea of the

independent component analysis. So for simplicity we just assume a very simple mixing model without

any time delays or further factors here and this then means if we knew the AIJ the problem of

reconstructing S is to solve the linear equations by classical methods because we simply need to

compute the inverse of this so-called mixing matrix. Now the problem is that we don't know

the AIJ so thus the problem is considerably more difficult and we can't just use the matrix inverse

of these unknown coefficients so we have to estimate them as well. Now let's look into a

simple example. So here I brought two signals two audio signals one is me speaking of course about

machine learning. Machine learning is a great tool that is revolutionized. And the other one is a

piano that is playing some background music. Now you can see that what we would record at the

microphone would then typically be a superposition of the two signals so in one you can hear that my

voice is louder. Machine learning is a great tool that is revolutionized. And in the other one my

voice is not as loud. Now the idea is that we want to find the unmixing matrix and with the unmixing

matrix we are able to reconstruct the original audio signals and you can see this is actually

a pretty hard task but still the results are quite impressive. So here you can hear my

reconstructed voice from the superposition of the two signals. Machine learning is a great tool that is revolutionized.

And here you can hear the reconstructed music.

So generally the cocktail party problem has many many more applications so it's not just for

unmixing two speakers or a sound source in the speaker but generally you can recover speech

signals from telecommunications. You can recover images from mixed signals like an MRI and functional

MRI. Then there's also examples where you can essentially reconstruct electrical recordings

of the brain activity. So this is used in EEG and MEG signals and of course it is a popular way of

extracting features and you can even do multispectral image analysis with this. So let's look into the

separation of natural image. So here you see that we have four different sources so it not just works

with two sources but you can also just increase the size of the mixing matrix and then let's look at

a total of four sources. Then you mix them somehow with the input to our independent component

analysis you can see that essentially we are close to not be able to recognize anything on those

pictures and then we perform the independent component analysis and you can see that it is

very well reconstructed so we see that the different animals can be reconstructed. We also get a

reconstruction of the noise so generally this is a method that has a broad range of applications.

Note that the sequence of the images has changed so this is a general drawback of the independent

component analysis. We don't get a ranking of the inputs so we still have to look at the outputs and

then identify which source has been mapped onto which output channel. Now this was an application

in imaging so we can also use this to analyze brain activity and here we are actually looking

into an MEG acquisition and you see that the MEG field is actually acquiring a superposition of all

the different things that are happening in the brain so we have a mixed signal that we're observing

and if we now apply an independent component analysis then we are able to reconstruct different

activities in the brain so we can use this method in order to localize different independent

components and regions of activity in the brain. So the common framework is that we have some