The following content has been provided by the University of Erlangen-Nürnberg.

Yesterday we made a few very basic considerations that are required to do image processing.

And let me motivate again what we are currently looking at.

For many procedures in the future we need to find interesting parts in the image.

We need to find edges in the image, for instance vessels.

We need to find corners for tracking objects and endoscopy or whatever you might think of.

And the question we have tackled yesterday was how can I analyze the image function that is basically in the case of a 2D image,

a 2D function where you have the row index, the column index and then you have the function value.

So we can imagine in our mental model that an image is just a surface on a 2D plane.

And then we started to analyze the surface that is basically defined by the image function.

And we looked into analytical methods like the computation of derivatives, the computation of gradients

and the analysis of the surface structure based on the information of the slopes and directions of maximum slope and things like that.

That is what we have considered yesterday and we have first of all seen a discrete version of derivatives

where we for instance considered df dx is just f of x plus one minus f of x.

We took the forward difference as an approximation.

There is the backward difference.

There are also many other ways to model it, finite element methods you might have heard of.

And then we generalized this to the gradient.

The gradient and we introduced the NABLA operator saying this is df dx df dy in the 2D case.

And in the n-dimensional case we can lift this properly.

And then we have introduced a very interesting operator and that is the so-called structure tensor.

Tensors are linear mappings and if you have two dimensional tensors these are basically matrices.

So this structure tensor should be called in the 2D case structure matrix.

But we will see later on that we do not have just a slice through the object but a stack of slices.

So we have three dimensional cubes of image data and so we can compute a three dimensional derivative

and a three dimensional structure tensor and we leave the concept basically of matrices at this point.

So the structure tensor was defined as and now there are two ways how you can remember things.

Structure tensor you can look it up in the web.

There will be formal definitions how the structure tensor looks like.

Or you can have more pictorial illustration in mind to remember what the structure tensor is doing.

The structure tensor is basically considering the gradients in the local neighborhood of the current point.

It looks at all the gradients in the local neighborhood.

It's an engineering constant to decide what is the size of the neighborhood.

Usually in image processing we take three by three neighborhoods or five by five neighborhoods.

Let me just draw this figure.

You will find also detailed slides on these things but I think it's better to explain it here with the pen and the whiteboard.

So these are the pixels we are considering.

All the little circles represent an entry in our image matrix.

Now we want to compute the structure tensor for this point here.

We can consider here the local neighborhood.

Let's say a three by three neighborhood.

Do we have British people here?

So we can have a U. It depends on U what you prefer.

We know that we have gradients here.

I have to terminate this because it will never come back.

Everybody understands what this picture means.

For each pixel we can compute a gradient.

In which direction does the gradient point to?

The biggest difference between two grid points to the highest change.

You don't have the situation that the gradient is pointing towards a grid point.