5 - Lars Ruthotto: Machine Learning meets Optimal Transport: Old solutions for new problems and vice versa [ID:16038]
50 von 694 angezeigt

Okay, cool.

Thanks Martin for the nice introduction and for having me, but most importantly to all

the organizers for pulling this seminar series off.

It's one of my Monday morning treats every other week to join the talks.

And it's so cool to be speaking here and giving this talk to, what is it, like a hundred of

my best new friends.

Okay.

So today's talk is going to connect machine learning and optimal transport.

And basically the idea is to have like a bi-directional exchange between these two areas that of course

both have been quite popular in recent years or decades or in the optimal transport case

even for centuries.

So the talk is divided into two parts.

The first one is a new tricks from learning.

So let's focus on optimal transport.

And I'm going to show you how techniques from machine learning can be used to solve optimal

transport problems in high dimensions.

So we are interested in high dimensional mappings of densities.

And basically what we are going to do is we will work with a dynamical perspective of

optimal transport.

Look at it in three different ways from the macroscopic, microscopic point of view and

also study the optimality conditions given by Hammond Jacobi-Bellman equations.

And based on that, we've come up over the last half a year or so with new solvers that

incorporate neural networks and combine a few other tricks in the numerical treatment

of neural networks that we've been developing over the last few years.

And that gives us actually quite a general framework that also extends beyond OT to something

like mean-fit games, mean-fit control problems.

And that part of the talk is based on an article that we've recently written with Stan Olsha

and his group during my time at IPM last semester.

And then in the second bit of the talk, I'm going to motivate why would you even care

about high dimensional OT problems.

Basically we're used to solving optimal transport in two or three dimensions, it seems, at least

in imaging.

But high dimensional problems arise in machine learning actually quite a lot.

So let's learn from these old tricks.

So let's apply OT in machine learning to a problem of variational inference.

So there's a branch of continuous normalizing flows, it's called in the machine learning

community.

And it turns out bringing in optimal transport knowledge really simplifies, regularizes the

problem.

And together with the right numerics can actually give quite drastic speed ups there, both in

terms of the training time, but also in terms of using these models, because you typically

want to generate them.

But then that's a subject to work some of my students have been doing that should be

out by the end of this week.

So let's get started with machine learning for high dimensional OT and more, because

it generalizes to mean-fit games.

So that work happened during my semester at IPAN last fall.

So I'm really grateful for the organizers of the long program on machine learning and

physics that I was part of.

And especially thankful for all the nice collaborations I had with these four guys, without whom nothing

Zugänglich über

Offener Zugang

Dauer

00:48:42 Min

Aufnahmedatum

2020-05-18

Hochgeladen am

2020-05-19 12:16:18

Sprache

en-US

This talk presents new connections between optimal transport (OT), which has been a critical problem in applied mathematics for centuries, and machine learning (ML), which has been receiving enormous attention in the past decades. In recent years, OT and ML have become increasingly intertwined. This talk contributes to this booming intersection by providing efficient and scalable computational methods for OT and ML.
The first part of the talk shows how neural networks can be used to efficiently approximate the optimal transport map between two densities in high dimensions. To avoid the curse-of-dimensionality, we combine Lagrangian and Eulerian viewpoints and employ neural networks to solve the underlying Hamilton-Jacobi-Bellman equation. Our approach avoids any space discretization and can be implemented in existing machine learning frameworks. We present numerical results for OT in up to 100 dimensions and validate our solver in a two-dimensional setting.
The second part of the talk shows how optimal transport theory can improve the efficiency of training generative models and density estimators, which are critical in machine learning. We consider continuous normalizing flows (CNF) that have emerged as one of the most promising approaches for variational inference in the ML community. Our numerical implementation is a discretize-optimize method whose forward problem relies on manually derived gradients and Laplacian of the neural network and uses automatic differentiation in the optimization. In common benchmark challenges, our method outperforms state-of-the-art CNF approaches by reducing the network size by 8x, accelerate the training by 10x- 40x and allow 30x-50x faster inference.

Einbetten
Wordpress FAU Plugin
iFrame
Teilen