The following content has been provided by the University of Erlangen-Nürnberg.

Okay, so good morning everybody. We dig into the last chapter of the winter semester, which is the most exciting chapter as all the other chapters as well.

We will continue with rigid image registration, just to give you again the big picture without writing things down because I forgot my tablet PC at home.

We have basically four chapters in this lecture. One was on different modalities just to get a good idea what imaging modalities are used in medical diagnosis or in radiology.

Then we talked a lot about preprocessing. So what can we do once we have captured an image to improve the image?

So we know the physics, we know what happens during the acquisition procedure, which artifacts come in, and then we try to eliminate these artifacts by using smart algorithms.

That was the second chapter. We looked at X-ray imaging, image undistortion, we looked at defect pixel interpolation, we looked at MR imaging and the inhomogeneities.

We have learned tons of different methods to remove and eliminate the inhomogeneities and so on.

The third chapter was on how can I generate on the, sorry, the third chapter was on generating higher dimensional information from multiple images that were acquired by a single modality.

So we capture multiple X-ray images and we want to generate out of these images higher dimensional information like a volume, CT volume.

And we learned algorithms to do so. Multiple images, reconstruction. We learned algebraic methods, we heard about statistical methods, we learned about exact reconstruction approximations,

we learned about weighting schemes if we have different projection geometries. So we have a very good understanding how we can generate higher dimensional information from lower dimensional measurements.

And the fourth chapter, and that's what we are now considering, is on what can we do if we have images from multiple modalities.

So think about the patient who goes to the hospital, who gets an X-ray, who gets a CT scan, who gets an MR, who gets some ultrasound images.

And then the doctor has all these images side by side on the lightbox. If you go to a mid-range hospital, they will still have film sheets and lightboxes.

Where are you from? Yeah? For instance? Klein-Sendelbach. Maybe there is a hospital. Maybe they still have a lightbox. Klein-Sendelbach. Very interesting.

Do you have electricity there? Internet. You have internet. Good. Lightboxes. And then you have your multiple views like in Klein-Sendelbach.

They are sitting there and looking at the ultrasound and then they go three meters to the X-ray image and then they mentally try to combine things.

And image registration means let's take all these images and transform them into a joint coordinate system.

Such that you can look at the image and you can fade in and fade out the information of different modalities.

And we know that different modalities show different features inside the body. So we can use all the advantages of all the modalities at the same time.

And we will now consider software-based methods to find a transformation such that the images fall into the same coordinate systems.

In industry, there are not so many software engineers. In medical engineering industry, they built hybrid scanners where they combine CT with a PET device, for instance, and they simultaneously acquire these images.

And what I want to do here in winter semester is I try to explain to you how image registration is done if there is no deformation allowed.

So the images are just rotated and transformed but not deformed. Once we get deformation into the game, things become way much harder than just the rigid transformation.

And we will now look into different applications. And oh, I should stop here.

We will talk about image registration. If you hear about the term image registration, that's basically nothing else but the process of transforming the different images into one common coordinate system.

The registration of volumes is also subsumed by the term image registration if you combine all these things.

If you combine it and visualize it, then we call it image fusion. That's just the way we call it within medical image processing.

So whenever you hear something about, oh, I do research on image registration, he's trying to find a transformation between two images.

Depending on these properties of the transformation, we basically distinguish two classes. We talk about rigid registration.

The German term is rigide Registration, rigid registration or starrere Registration, rigid.

So we have no deformation. And the term non-rigid registration also allows for deformations.

And if you look at the math from a distance of 30,000 feet, you will notice that this appears way simpler than this one.

But I can guarantee both problems are really very, very hard problems.

And some people consider them to be solved. But if you look into the details and if you look how things work, basically in practice,

there are so many open research questions that we are far away from having a solution which is totally satisfying.

And we will learn methods that can basically compute a rigid transformation between different images.

And if you talk to doctors and radiologists and if you ask them what is very important for you,

quite often you end up with a statement like we want to have image fusion.

When I started to work here at the university and I started to define my research field,

I was not considering image registration to be a very demanding and challenging and interesting field.

But I learned that people want to have it. And then we started also to work in the field.

Then you discuss with doctors and doctors ask you what do you need to do in image registration?

And I told them or we, the computer scientists, the image processing people told them,

look, if you can provide some markers and some points which do not change while the patient is moving around

and while the patient is transformed basically, then we can easily compute the transformation out of point correspondences.

Geometry and basic mathematics allows us to compute the transformation.

And then doctors said no problem at all, so I screw in here a marker into the skull

or I screw in here a whole rack with a lot of markers that you can easily perform the image registration