Thank you.

And so they say today I would like to present a deep learning approach to reduce order modeling

of parameter dependent partial differential equation.

Now, one possible way to blend the classical let's say approaches for numerical approximation of partial differential equation that in some sense is the core business of the group here in Milano.

And the new techniques are based on machine learning definitely, my opinion has a potential in this in this field.

This is a talk in an activity in collaboration with many colleagues at the Laboratory of Modeling Scientific Computing of the Department of Mathematics or Polytechnic, in particular, Nicola R.

Franco, a brilliant postdoc, Stefania Fresca, an assistant professor and Professor Andrea Manzoni, a well known professor in the area of reduce order modeling.

So, let me start by giving some context. And so first of all, we will speak about reduce order models, which are

problems such as parameterized partial differential equations that typically need or are used in a sort of many query, many query scenarios. What is a many query scenario we can think of, for example, control problems where we have to design optimally

parameters that maybe govern the geometry or the physical properties of a system. And so we had to repeat the solution of the PD many times, but the same, let's say situation happens if, for example, if we want to quantify the sensitivity of some

quantity of interest with respect to the parameters or even better in the case of uncertainty quantification we will have to assign a probability distribution to the parameter space, and we want to propagate this distribution to the quantities of interest

through the model, which also requires a very large numbers of evaluation of the PD.

And if we handle the with the advanced numerical methods and computing platforms, maybe hundreds or thousands of evaluation still represent a significant cost. So in this context the new model really are the main methodology to address these problems.

So we start from a well known high fidelity model that represent our, let's say, a partial differential equation discretize with the model of choice with the desire that accuracy for the application, which generally leaves in a high dimensional

function, for example, the number of degrees of freedom of the finite element approximation. And we want to let's say they reduce the dimensionality of the problem by some suitable model or the reduction technique that basically aims at

quantifying the some representative basis functions that can capture the features of general finite element functions. And so this is the result of the model or the model or the reduction technique basically the reducer that model that leaves in a typical

let's say space which has a much less degrees of freedom of the original one.

Also, it is important to to consider that typically this strategy is based on the composition in two main steps. There is the so called offline steps so we typically we collect data which basically are, let's say input output pairs of the, let's say, inputs,

in this case the parameters of a PD and the outputs the solution of the PDs for many different values, many different points in the parameter space. And with this data we construct the reducer model.

And then there is the online phase the one that we want to solve repeatedly many times.

And then we take certain costs with respect to the high fidelity mode. And what I would like to discuss today is really to show how deep learning can really facilitate this or generalize this approach of reducer model.

And before going into the main topic of the of the seminar, I would like to give maybe it's as trivial but brief introduction to deep neural networks and what does it mean to train a deep neural network just a couple of slides.

So basically I suppose all know that the deep neural network, and in particular here we are we restrict to the specific architecture of the deep neural network, among many other possibilities, I consider the fit forward deep neural network so basically

we have a network that receive an input with a suitable let's say dimension, and through a sequence of linear and nonlinear information by multiplication of a weight matrix and addition of a bias vector.

And then to the output vector we apply an activation function that is typically nonlinear.

And in what we see we will use the whatever the parametric rectifier linear unit almost simply, more simply the rectifier linear unit, which is this type of function, but the activation functions can be different can be the single function, etc.

But this is where exactly the nonlinearity of the deep neural network comes into play into play.

And then we repeat this construction many times up to the final layer, which is the only difference that typically the activation function is not applied to the output.

I would like to stress here that this construction is really very general and powerful approximation properties in high dimensional spaces, which are somehow known as the universal approximation property of deep neural network and I would like to let's say

I'll give you some examples. For example, if we just consider a single layer percept perceptron. So basically one single layer of this construction with continuous activation function we can approximate any continuous function.

If we use a deep neural network basically a multi layer perceptron with at least one hidden layer. So imagine one layer in between the input and the output.

And we use a differential activation function, we can approximate any continuous function. And if we use at least two hidden layers and use a differentiable activation function we can approximate any function.

So you see we have really, we can hear how high generality in representation of, let's say, general continuous function.

And this is really exploited in what I will, I will present. And also just a couple of words on what does it mean to train a neural network, which is a problem where we

have a data space, which combined together, determine the data space the data are always couples between the input and the output. And we select a suitable points in the data space that constitute that generate our training data.

And we also identify some hypothesis set of function that we want to characterize. And in this case indeed this hypothesis set is exactly represented by the fit forward the neural networks uniquely characterized by the definition of the weights and the biases of every

layer, typically we don't change the activation function that division function is determined a prior.

We need a functional to minimize that is typically determined by the loss function, which is basically the measure of the discrepancy between the data and the representation of the data by the

let's say we evaluate through the last function and empirical functional to be minimized that is the evaluation of the loss function in every point of the data set.

So these are the fundamentals of machine learning and now I would like to let's say, state our general objective, which is indeed to develop model or the reduction strategies, which are for personalized partial differential equations, which are have this

particular features which are quite desirable features for this type of methods. First of all, we would like that this model reduction strategy is non intrusive.

Basically it means that doesn't require to the knowledge of the description of the operators and manipulation of these operators in order to obtain the reduce order model.

And of course, what I just said, in terms of the high fidelity model in order to obtain a reduce order model but this is typically not possible in many situations of industrial or let's say, real interest for example, if we want to use for the full

scale model a commercial platform or a legacy code, they, let's say we really need to exploit a non intrusive reduce order modeling technique.

And as a consequence of that typically our technique will be the will be data driven. And here we see that machine learning will really play a role.

And indeed, basically what we will set up is a supervised learning approach for the derivation of a suitable reduce order model for parameterized PDEs.

So let me just formulate the general model for parameterized PDE, which is stated here as an abstract model, the operator, the forcing terms the boundary conditions can depend on, let's say some parameters can be either physical parameters, geometrical parameters

of the problem so the approach is extremely general.

So the abstract model, we discretize the abstract model obtaining what I presented before that is the full order mode. And here we assume that we really apply the best, let's say numerical approximation techniques available with a really desired accuracy

by the application that we have in mind.