Okay, cool.
Thanks Martin for the nice introduction and for having me, but most importantly to all
the organizers for pulling this seminar series off.
It's one of my Monday morning treats every other week to join the talks.
And it's so cool to be speaking here and giving this talk to, what is it, like a hundred of
my best new friends.
Okay.
So today's talk is going to connect machine learning and optimal transport.
And basically the idea is to have like a bi-directional exchange between these two areas that of course
both have been quite popular in recent years or decades or in the optimal transport case
even for centuries.
So the talk is divided into two parts.
The first one is a new tricks from learning.
So let's focus on optimal transport.
And I'm going to show you how techniques from machine learning can be used to solve optimal
transport problems in high dimensions.
So we are interested in high dimensional mappings of densities.
And basically what we are going to do is we will work with a dynamical perspective of
optimal transport.
Look at it in three different ways from the macroscopic, microscopic point of view and
also study the optimality conditions given by Hammond Jacobi-Bellman equations.
And based on that, we've come up over the last half a year or so with new solvers that
incorporate neural networks and combine a few other tricks in the numerical treatment
of neural networks that we've been developing over the last few years.
And that gives us actually quite a general framework that also extends beyond OT to something
like mean-fit games, mean-fit control problems.
And that part of the talk is based on an article that we've recently written with Stan Olsha
and his group during my time at IPM last semester.
And then in the second bit of the talk, I'm going to motivate why would you even care
about high dimensional OT problems.
Basically we're used to solving optimal transport in two or three dimensions, it seems, at least
in imaging.
But high dimensional problems arise in machine learning actually quite a lot.
So let's learn from these old tricks.
So let's apply OT in machine learning to a problem of variational inference.
So there's a branch of continuous normalizing flows, it's called in the machine learning
community.
And it turns out bringing in optimal transport knowledge really simplifies, regularizes the
problem.
And together with the right numerics can actually give quite drastic speed ups there, both in
terms of the training time, but also in terms of using these models, because you typically
want to generate them.
But then that's a subject to work some of my students have been doing that should be
out by the end of this week.
So let's get started with machine learning for high dimensional OT and more, because
it generalizes to mean-fit games.
So that work happened during my semester at IPAN last fall.
So I'm really grateful for the organizers of the long program on machine learning and
physics that I was part of.
And especially thankful for all the nice collaborations I had with these four guys, without whom nothing
Zugänglich über
Offener Zugang
Dauer
00:48:42 Min
Aufnahmedatum
2020-05-18
Hochgeladen am
2020-05-19 12:16:18
Sprache
en-US
This talk presents new connections between optimal transport (OT), which has been a critical problem in applied mathematics for centuries, and machine learning (ML), which has been receiving enormous attention in the past decades. In recent years, OT and ML have become increasingly intertwined. This talk contributes to this booming intersection by providing efficient and scalable computational methods for OT and ML.
The first part of the talk shows how neural networks can be used to efficiently approximate the optimal transport map between two densities in high dimensions. To avoid the curse-of-dimensionality, we combine Lagrangian and Eulerian viewpoints and employ neural networks to solve the underlying Hamilton-Jacobi-Bellman equation. Our approach avoids any space discretization and can be implemented in existing machine learning frameworks. We present numerical results for OT in up to 100 dimensions and validate our solver in a two-dimensional setting.
The second part of the talk shows how optimal transport theory can improve the efficiency of training generative models and density estimators, which are critical in machine learning. We consider continuous normalizing flows (CNF) that have emerged as one of the most promising approaches for variational inference in the ML community. Our numerical implementation is a discretize-optimize method whose forward problem relies on manually derived gradients and Laplacian of the neural network and uses automatic differentiation in the optimization. In common benchmark challenges, our method outperforms state-of-the-art CNF approaches by reducing the network size by 8x, accelerate the training by 10x- 40x and allow 30x-50x faster inference.