Welcome back to Deep Learning.

Thanks for tuning into the next video.

What I want to show to you today

are a couple of limitations of Deep Learning.

Is it unbeatable,

has it solved all of the problems,

what is working well,

And where are we approaching the limits?

So where are the problems of deep learning?

So I hope you will enjoy this video as well.

Well, of course, there are some limitations.

For example, take tasks like image captioning.

Here on this slide, you see really impressive results.

You can see that the networks are able to identify a baseball player, a girl in a pink dress jumping in the air, or even people playing the guitar.

So let's look at some errors. Here on the left you can see this is clearly not a baseball bat.

Also, there isn't a cat in the center of the image.

And there are also slight errors like the one on the right hand side. The cat on top of the suitcases isn't actually black.

Sometimes there are even really big errors like the one here in the left image.

I don't see a horse in the middle of the road. Also on the right image there is no woman holding a teddy bear in front of a mirror.

So, the reason of course is that there is a couple of challenges and one major challenge is training data.

Deep learning applications require huge manually annotated datasets and these are hard to obtain.

Annotation is time consuming, expensive and often ambiguous.

So as you've seen already in the ImageNet challenge, sometimes it's not clear which label to assign.

And obviously you would have to assign a distribution of labels.

Also, we can see that in the human annotation, there are typical errors.

What you have to do in order to get a really good representation of the labels,

you actually have to ask two or even more experts to do the entire labeling process.

Then you can find out the instances where you have a very sharp peaked distribution of labels.

These are typically prototypes and broad distributions of labels are images where people are not sure.

If we have such problems, then we are typically getting a significant drop in performance.

So the question is how far can we get with simulations, for example to expand training data.

Of course there are also challenges with trust and reliability.

So verification is mandatory for high risk applications and regulators can be very strict about those.

They really want to understand what's happening in those high risk systems

and to end learning essentially inhibits us to identify how the individual parts work.

So it's very hard for regulators to tell what part does what and why the system actually works.

We must admit at this point that this is largely unsolved.

It's difficult to tell which part of the network is doing what.

Modular approaches that are based on classical algorithms may be one approach to solve these problems in the future.

So let's look at some future directions.

Something that we like to do here in Allen is learning algorithms.

So for example you can look at the classical computed tomography which is expressed in the filtered back projection formula.

You have a filter along the projection direction and then a summation over the angle in order to produce the final image.

So this is a convolution and a back projection that can actually be expressed in terms of linear operators.

As such they are essentially matrix multiplications.

Now those matrix multiplications can be implemented as a neural network and you essentially then have an algorithm or a network design that can be trained for specific purposes.

So here we extend this approach in order to apply to fan beam projection data.

This is only a slight modification of the algorithm and there are still cases that cannot be solved like the limited angle situation.

In this image you see a full scan and this produces a reasonable CT image.

However, if you are only missing 20 degrees of rotation, you already see severe artifacts.