Thank you very much for the introduction. So today I'm going to talk about indeed. Yeah, how biomechanics can benefit from deep learning and how deep learning can benefit for from biomechanics.

So first of all, in our group, what we would like to do is, is predict human movement. And so why do we want to predict human movement is we want to be able to design or use it in a design of variables.

So for example we want to design a running shoe or maybe a prosthesis or an exoskeleton and already understand how this affects someone's gait before we actually, before we actually use the device or prototype it.

Because what currently happens is that you have this cycle of designing a prototype and testing it in some lab, and then adjusting the design and so on. And so this just requires a lot of effort to prototype but also you need participants to test out your system.

And so it's, it's in general a bit cumbersome. And so what we want to move to is to have instead of this two steps at this third step of simulation.

And so this is already very common for example in the aerospace and automotive industry where the simulations are used to assist the quality or the sound or the aerodynamics of engines for example, and in many other applications.

And so similarly, we want to be able to predict how people move when they're in some new environment.

Another reason that we're also interested in this is to, to estimate the state so you all probably have some sort of device that tracks you when you move around like your phone or a Fitbit, and you also have these tiny IMUs that you could put in your shoe

which are called Nike, Nike Plus. And recently also this gamer system was developed.

And what we can do with this too is actually extract more information from these sensors, if we really understand human movement better. And finally, yeah, what we want to do is also understand why do we walk the way we walk, because if you have seen

Monty Python's Ministry of Sailor Walk, you know that there are many ways that that we can walk, which are called silly, and we all kind of walk or move around in the very same way. And the question is why do we do that out of these infinite options that we have.

And so, we do this because we want to be energy efficient so basically people are quite lazy, and they always try to be as efficient as possible. And this is shown in this graph over here, around the horizontal axis, you have the step rate, so the number of steps per minute.

And on the vertical axis, you have the energy expenditure in calories per minute per kilograms of body weight.

And so in this experiment from over 50 years ago, or almost 50 years ago, people were asked to walk with different step length, and at different step frequencies. And what you can see is that there's this more or less this bowl shape.

So out of the range of step frequencies, there is this minimum where your energy expenditure is minimized.

And so, what these people found, and also has since been found again is that people actually also choose to walk at a set frequency that is close to the minimum of of this bowl.

And so if we put that somehow into the computer.

We can actually simulate or try to simulate and predict human movement.

So we do that by something what we call a trajectory optimization problem, which is actually the same as an optimal control problem. So we use tools from optimal control theory to create, and we create a model with multi body and muscle dynamics.

We also create some model of energy expenditure, which we'll talk about later. And then we solve a problem where we say, the goal is to move this human model forward, while minimizing energy, and the output is something like this.

Where you can see without.

Yeah, we were able to predict the motion the walking motion of the skeleton, without.

Yeah, without requiring an experiment.

So, at least this is the ideal case that we want to move through with even also adding accurate arm movement.

However, currently, when we solve our problems, what we get is something like this.

So here you see it's a two dimensional model in this case, and it's walking on a toe you can see this is not realistic. This is not how someone would would normally walk.

So what we normally do to solve this is that we add some reference data to track, which creates these makes these simulations much more accurate.

However, it's our goal to remove this tracking data. And so therefore, there are different avenues that can be explored.

First of all, this objective of energy is in itself probably not not enough to create accurate predictions, but there is more to it.

Then there's some model inaccuracies related to the physical background, but also the anthropometric so that is what we mean by that are the just the general weight and height parameters over human.

The problem is we can't just measure what the weight and center of mass of someone's lower leg is.

And so therefore we have to base our simulations on estimations for example from cadavers studies, which are obviously not always going to be accurate.

And so furthermore, we also do not take into account differences between people yet. So right now.

We use our generalized models to make generalized predictions. And also, if we want to really be accurate and uses for design, eventually we'll have to move to to including the personal, the other personal differences between individuals.

So today we're going to talk a little bit about all these things. And yet combine or explore how machine learning can actually be used to improve gate simulations and move towards this goal of predicting motion without using any reference data.

So first we'll start is because a lot of you are probably not familiar with biomechanics, we start with an introduction to gate simulations, and then talk about an example problem or an example study that we did recently in the within our group about exoskeletons.

We move on to more machine learning related research and we talked about how machine learning can improve gate simulations. And as an example we're going to look at metabolic energy models.

And then finally, we talked, we are going to talk about how gate simulations can improve machine learning or neural network machine learning models.

And we're going to talk about this study from someone in our group from about a year ago, where we, yeah, we use generalization through biomechanical simulations and basically improved CNN models using gate simulations.

But first, we're going to, I'm going to give you an introduction to gate simulations, and we'll start with the very basic and talk about a gates cycle. We'll talk about the models that we use and it should actually optimization problems that will solve.

So first of all a gate cycle, a gate cycle is just a simple walking cycle, where there is actually, but there's actually quite a lot to it. So there's two phases mainly there's a stance phase and there is a swing phase where the sense phase is about 60% of the

cycle, and the swing phase 40%. So that also mean that about 20%. So here there's this double support and here there's this double support. So there on these, in these times both the feet are on the ground.

And then very importantly, terms that we use a lot is flexion and extension. Perfection is the movement of the, the other bone that is further away towards the bone that is closer and extension is the opposite movement.

And so, yeah, so try to. So that is what we call the motion in in this side plane or the size of plane of the human body. And so obviously, gates are walking and running, mostly happen in this two dimensional side of the plane.

So that is why most of our studies are also in this plane.

Then, commonly what happens to yet to capture gate is we perform gate analysis experiments.

So here what we have is a motion capture system so here you see about six or eight different cameras that are all pointed to to this force plate over here.

And so then a person will wear these, these reflective markers, as you can see over here, which, yeah, which done and then these, the cameras that you see in the graph in the picture over here, they emit infrared light, which is then reflected by the

markers. And then if this is picked up by two or more cameras, you can.

Yeah, where the algorithms can calculate the three location of all the body markers.