Okay, so welcome everybody. Sorry for the short delay. And we are back to deep learning

this morning. And today we want to discuss object detection and segmentation. So now

we are approaching imaging problems and we want to figure out not just classify an image

but really figure out what's going on in an image. And this is done with these two techniques.

So we will first clarify what we mean with object detection, what we mean with segmentation.

And once we have done that we will talk a bit about object detection and different approaches

and then we will talk about segmentation and how to get really pixel-wise annotation of

some image. Okay, so what do we mean by these two techniques? So far we have been looking

at images and there was some input image, it was probably showing some object and we

had this input and said okay this is a cat. But this is not really how most images look

like. Typically you don't have one object that fills your entire view but you have more

complicated scenes where you actually have several things happening and obviously there

could be scenes like that. So you could have for example the object localization and this

would then mean that you have a bounding box that tells you okay this is the object of

interest so the cat is located here. So this would be merely some object recognition and

localization but actually if you want to detect objects what you would be interested in is

detecting one bounding box per object. So this is what we are dealing with when we are

talking about object detection. So we are interested in getting the bounding box and

the identifier so cat here for example. In contrast to that we have image segmentation

and image segmentation is identifying every single pixel of the image and it's assigning

it a class. So here this would be the class cat and all of the pixels that are associated

with cat are delineated here. Often this is also segmentation is related to finding the

outlines for example of organs is very classical image medical image processing type of organ

segmentation. So this would be a semantic segmentation and then in the third part today

we will even talk about semantic instance segmentation. So here the task is done not

just identifying all of the pixels that belong to a certain class but also detecting several

instances of that object over here the three cats that are shown here in the image and

figuring out which pixel is actually associated with which cat. So these are the different

things that we want to talk about today and I can tell you so this is one of the let's

say advanced topics and when you are preparing for the oral exam of course we want to know

how the like how object detection works how segmentation works there's a couple of algorithms

that can be explained really well and I think that's a very good preparation for the oral

exam. So we want to know the basic ideas how those algorithms work and how you can get

them implemented in a neural network. Okay so let's see how we can what we can do about

object detection. Well in object detection this is not something that just started with

deep learning there's actually quite a bit of history to that and in particular there's

the two tasks that you want to localize the object and you want to classify. So that's

that's the two things that have to happen and typically what you do is you try to find

some some bounding boxes then you re-sample the selected boxes such that you can classify

and then you apply the classifier. So you find the bounding box and the finding the

bounding box also helps you with figuring out a size of the object that makes sense

and if you have a bounding box and then you have perspective distortion so let's say one

cat is bigger than another because it's just closer to the camera you can solve that by

re-sampling and then you re-sample to a fixed grid of a certain size and this then allows

you to do the classification. So practically what you want to do a clever combination of

putting those steps together in order to get better speed or accuracy. So what are we looking

for? Well we're looking for bounding boxes and yeah so what you want to find is the smallest

box that fully contains the object in question and typically you define that by the top list

in the left corner some width and some height and the classifier confidence. So that's how

we describe typically a bounding box and this would be the bounding box for the plane and