Okay, so welcome back everybody to interventional medical image processing.

Everything okay? Yes, okay, good. So welcome. Wow, you're going so far back.

So there's competition. Okay, so today we will talk about the last chapter in pre-processing

and what we are going to look at today is image super resolution. So it says super resolution

image reconstruction and you will find that the way of processing super resolution images

is very similar to a reconstruction process. So we will first discuss what is super resolution,

what is the idea of super resolution, then we will talk a bit about cameras and sampling

and what actually happens when you discretize an image and this is followed by a single and

multi-camera super resolution. So there's two different kinds of methods. So you can do super

resolution with a single frame using prior knowledge or you use multi-frame super resolution

that you have a sequence of frames and you use an estimate of the motion in between to

compute super resolved images. Based on that we will then look into

like statistically based super resolution. So the idea is that there is also a relation to the image

statistics to noise statistics that you can build in order to design super resolution approaches

and we will finish by take home messages. Okay, so what is super resolution? So you may know CSI Miami

or one of these nice television series and there they do super resolution all the time. So they take

a video and then they zoom in into the mirror of a car and then there is a reflection and you zoom

further in and this is basically the idea of super resolution and you will see today that if you

actually try to do that it's actually much harder than in the videos. So we won't be zooming into

images and looking into reflections and then recognize faces and reconstruct them. So we won't

be able to do that. But it's still a fancy technique so it's still very nice and we will

have a couple of examples of different modalities where we apply super resolution. One nice thing is

you can use for example multi-frame or multiple sensors and in this way you can get quite high

resolution images from a pretty cheap sensor. So this is one application where super resolution

became popular in the last few years because you can use it in a very cheap sensor technology.

The downside is of course the money you save on your sensor you have to spend on computation time.

So you probably can use very cheap sensors and then you ship a cluster with it in order to

compute the super resolved images. Now you can do that on for example on graphics hardware

very efficiently and but there's a considerable amount of computing required.

Okay so let's go to the problem description. The real imaging system performs a non-ideal mapping

of a scene to some kind of image plane. This is the image plane of your camera and it does a

non-ideal mapping and there's a couple of steps happening in between. So if you consider your

image plane and you're sampling there first of all you have to to sample or essentially down sample.

So the signal that you get actually has if you consider it as a continuous signal it is in general

it may contain very high frequencies and very good resolutions therewith and you have to map it

onto a discrete space and what you assume typically for a super resolution approach

is that the original image that you want to reconstruct has more higher frequencies

that then you actually collect in your down sample signal. So you have you're always

collecting a down sampled version of the original signal. So this has a lower resolution than

the actual image that you're expecting and of course there's also other things happening.

So generally there is a certain blur introduced by your sensor and there's of course noise.

So by photon statistics if you don't have enough light it may introduce noise or generally introduced

by the sensor by quantization. So there's different kinds of noise that could affect the image

and the result is that small details like high frequency spatial high frequency details get lost

in this 2D image. And now the super resolution is I want to estimate a good approximation of

the ideal image or the high the super result or the high resolution image from this real

low resolution image data. And one example for example one example are typically surveillance

cameras. So you have plenty of surveillance cameras around and you want to improve image quality.

So you want to zoom into the reflections of a car and then recognize faces and things like that.

And then there's of course military applications. So you want to super resolve for example satellite