So welcome. Welcome to the Monday session, 45 minutes, let's say 35 minutes today, because

I have to leave a little earlier. We will talk about light fields and virtual endoscopy

today. So far we have considered epipolar geometry, magnetic navigation that is used

for interventions. So far we have considered ultrasound structure from motion approaches

for 3D ultrasound imaging during interventions. So far we have considered endoscopy using

calibrated endoscopes. We talked a lot about hand-eye calibration so far. And now we will

talk about the following scenario. Think about that, you have an endoscope and this endoscope

has some markers. We have done the hand-eye calibration properly and what we want to do

is the following. We want to do a sweep, no we want to put the endoscope into the human

body, then we do a sweep, capture images and then we pull it out again and we have a sequence

of images, including the information on the markers and the markers positions. And now

we would like to compute a new image from a direction that was not part of the acquired

path or of the acquisition path. So the idea is I have captured images and now I want to

look from a different point of view on the image. So think about the following, you have

for instance here a path where the focus of your camera is moving and you get the images,

here is the bundle of projection lines, that's always the picture we have in mind, bundle

of projection lines. And now somebody says, okay now you have all these images, capture

a sequence of images and you want to compute a new image where the focus is here. You are

smiling, you know parts of it, I know. And the question is how can I compute an image

from this position that is not at a point where the camera was before, completely a

new image. Or to show or to tell you the extreme, you are looking somebody from this side and

then you say how does he look from the other side. That's the question we can answer, roughly.

We won't do that but you can of course think about that, that you move it this way and

then you say can I look a little bit from this side or that side and you can compute

artificial images that are as close as possible given the information that was acquired by

this camera motion. Of course we will not be able to look behind the scene, right. I

mean no information was acquired regarding this so you will not gain anything out of

the set of images, of course. So there was a joke. You see I'm always running back and

forth. So I can try this like a priest. Good. And now let's do first a few historical remarks

on the whole story before we dig into the math. The term virtual endoscopy, that is

used in the literature in completely different contexts. First of all if you read something

in the web about virtual endoscopy the probability is quite high that something completely different

is meant than what I was describing so far. Virtual endoscopy in medical engineering,

in medical imaging usually means you have a set of CT slices like these here. You see

the lung here. This for instance is the heart. These are the two lungs. And you get a 3D

data set out of the CT data, out of the CT scan. And what you want to do is you want

to virtually fly through the lung for instance. So you put your virtual camera into the data

set and you look around. That's virtual endoscopy. Usually in CT you look at slices or you use

volume rendering and you look through an object. In virtual endoscopy you can use the 3D data

set and take a virtual camera and go into the data set and look around. For instance

it's used for virtual colonoscopy. Colon, that's the darm, virtual colonoscopy that

means you do a visual inspection of the colon by using CT data sets. Instead of using an

endoscope and put an endoscope into the human body you just do a CT scan and then you virtually

fly through the colon and check for polyps for instance. So there was I think three,

four, five years ago there was a huge announcement in the German press where virtual colonoscopy

was advertised. That's a system developed by Siemens GE and all the big CT system manufacturers

where they try to make people more and more aware of colon cancer and early detection

of colon cancer. So this is done by virtual colonoscopy. And here for instance you see

also the inner part of the human body. So the tube structures, now you can fly through.

You can also use a CT angio where the blood is carrying around a contrast agent. So you