Yes, I'm going to talk today about a very generic title almost, but this is sort of
work that I've been, well, amongst other work I've been recently been working on, on the
intersection of machine learning and inverse problems in some ways.
And yeah, as Leon said, it will actually involve Breckmann distances, which I believe is one
of the few research groups where I can probably talk about Breckmann distances without even
having to explain them.
Although maybe this has changed, maybe it's not the case anymore, who knows.
We'll see.
Okay, let's get started.
What I should mention first is that this is predominantly work of my PhD student, Victor,
who you see here, who's actually not at Queen Mary, he's at Cambridge.
So when I was still at Cambridge, he became my PhD student and therefore, yeah, he's still
my PhD student.
And this is predominantly his work.
Okay.
And what am I going to talk today about in terms of the outline, in terms of details?
So what I first want to say, so Leon instructed me that I should try to give a talk more like
on a master level, on a basic level for a mixed audience.
So I will first talk about perceptrons and what perceptrons are.
It may very well be that you all very well know what perceptrons are, but it's probably
always good to also set the notation and to recall the basic properties and concepts.
And then we come to the training or the learning of perceptrons and how we've got to paraphrase
a very old classical algorithm, namely the Rosenblatt algorithm, and see how we can actually
formulate it as a proper gradient-based algorithm for which we will define a Breckmann loss
or a loss based on Breckmann distances.
And then if time permits, actually this might be a big if, I even will talk a little bit
about the extension to general artificial neural networks, which is current work in
progress.
So we don't have a paper on this yet.
We have a small paper on the first part.
And I will then conclude with some conclusions and outlook.
Okay.
Let's dive straight in into the concept of perceptrons.
So yeah, back in the fifties and sixties of the previous century, the idea of perceptrons
were born and there were basically simplistic models of biological neurons.
So the idea was to take a biological neuron and to create an artificial neuron in a simplistic
way where basic parts of the perceptron, such as the dendrites that take the inputs, the
cell body and the nucleus that process these inputs that the dendrites receive and the
sort of axons which transmit a potential output of a neuron that they are being modeled in
form of an artificial neuron.
And the most popular artificial neuron, originally proposed by Frank Rosenblatt, is the perceptron
or used to be the perceptron.
And what does the perceptron do?
So a perceptron takes a couple of inputs, which traditionally are binary.
Today they do not necessarily have to be binary anymore, which here in this graphical case,
and that actually reminds me that I do have a pointer and I should obviously also use
the pointer so that we have these inputs here, x1, x2, x3, for example.
So here we have a perceptron with three inputs and one output.
The idea is that the perceptron takes these inputs in the same way that a biological neuron
would process inputs and then that the cell body is being or that the process of whether
Zugänglich über
Offener Zugang
Dauer
01:10:53 Min
Aufnahmedatum
2020-12-16
Hochgeladen am
2021-01-08 17:08:43
Sprache
en-US
Martin Benning on "Generalised Perceptron Learning"
We present a generalisation of Rosenblatt's traditional perceptron learning algorithm to the class of proximal activation functions and demonstrate how this generalisation can be interpreted as an incremental gradient method applied to a novel energy function. This novel energy function is based on a generalised Bregman distance, for which the gradient with respect to the weights and biases does not require the differentiation of the activation function. The interpretation as an energy minimisation algorithm paves the way for many new algorithms, of which we explore a novel variant of the iterative soft-thresholding algorithm for the learning of sparse perceptrons. This work is joint work with Xiaoyu Wang.