Hello everybody, I guess we can start.

Our topic today will be on support vector regression, which is kind of repeating the

theory that you learned about support vector machines, but with slightly different examples,

just to check that we know about how to formulate an optimization problem and how to formulate

the dual problem of that.

I assume that you can see my screen, you can hear me, so if not then please write me.

So what we did in the last theory exercise, and this was the one before Christmas, you

should be able to find it online on FIU TV, and what we did there is basically what we

did the week before that in the programming exercise, and it was basically doing line

fitting, so we formulated an optimization problem either using the R2 norm or the Huber

norm, and our idea was that we have given some points, and for each of those points,

we have a pair of coordinates, one coordinate is more or less like our feature vectors,

and then our y, which will be the coordinate that we want to predict, so prediction space,

and what we have given is, we have all the coordinates from all these points, so we have

the x coordinates and the y coordinates, and our task is to find a model that kind of explains

best this data that we have, so that in the future when we get another x coordinate, like

for example this one, that we can just use a model and predict, okay for this x value

you should use this value y, and to find a model, we used a specific category of functions,

this example a line with some parameters, and our parameters, we just called them alpha,

well we could use any line, but we said to kind of say which one is the best model, we

formulated an optimization problem, where this one, this y comes from our data, and

this x, this f of x would be our model, given the data x points that we already have, and

we try to minimize the parameters of our model, optimize the parameters of our model such

that the distance, the discrepancy between the prediction of our model and our actual

data points is kind of minimal, so we measure all these distances here, in some norm, here

in this example we used the L2 norm, L2 norm of this distance, L2 norm of this distance,

plus this, plus this, plus this, plus this, plus this, and we add over all these errors,

and we say like a model is optimum with respect to our loss function here, when like the parameters

cause this loss to be minimal, so this is the one that we used last time, and what we

want to do now is, we just want to use a different optimization function, similar to support

vector machines, where we also try to find like a line, and a band around the line that

is kind of maximum, and we now try to find a model, so again this would be our model

here, this line here, with the same function, also with the same parameters, and what we

now say is, we try to find the model such that we can have like a epsilon band around

that model, so you have a upper border of this band, so it would be f of x plus epsilon,

so this would be this line here, and you have also a lower bound of this band, this would

be f of x minus epsilon, and what we try to achieve is, now that all the points from our

data fit within this band here, so this would be hard, hard as support vector regression,

as we are, all points must be in epsilon band, so in this case, this would not be a solution

because we have one point here outside, and in the hard vector, support vector regression,

we would try to find like the smallest epsilon, so that we can find a band such that all points

fit within this band, for example, maybe this band could look like this, I don't know,

so kind of the highest point, lowest point, we try to make this discrepancy as low as

possible, and all the points should fit within this band.

Is the session recorded?

Yes, it should be, yeah, the record button is pressed.

Sorry, why we measure the distance between the epsilon 1 to that line, why we don't

perpendicular to that line, but we just measure perpendicular to the y axis, right?

Yeah, because actually, you're asking why we do not measure the distance like this,

so perpendicular, and the reason for that is basically because what is the task of our