Let's go back to learning.

If we look at what we did,

was something conceptually extremely simple.

We give ourselves a hypothesis space,

and we optimize the loss in that hypothesis space.

It kind of have a hidden problem that makes it

completely unrealistic for real-world applications.

Namely, we always search from nothing, from tabula rasa.

Which is not a problem if you're fast enough,

because then you can always do that,

because you can just retain the old data and just

throw it at the learning thing again.

But it's most certainly something that is not true for,

at least human beings.

It's not that you wake up every morning,

look at your collected works of computer science,

starting with computer science one,

just briefly read all of them,

go to the course, learn something new,

have a beer, forget everything,

wake up next morning and read this slightly longer book.

But that's exactly what we're making our systems do.

Okay? That's the problem.

You have to forget everything before you learn something new.

So, what we would like to do,

we would like to do, what you're doing.

You wake up in the morning,

brush your teeth, go to the course,

and then you remember stuff.

Some of you. Not everything.

We don't need perfect memory.

It's good actually to lose some stuff while you're dreaming.

Our brain has dedicated cleanup procedures for those things,

reinforce certain things and forget other things.

Very important. But we retain some of the knowledge of

yesterday to build up on today.

Most of us remember quite a lot from the last couple of years.

Wouldn't it be nice if we could do something like that?

Unfortunately, we don't have a very good way.

Now, we have a way of doing that,

which I'm going to try and tell you about.

We do not have a good way

of integrating that with machine learning techniques.

I consider that as one of the holy grails.

I'm hoping for your generation to actually solve that.

So, we have a way of incrementally learning,

and looking at examples and then compiling them

down into a knowledge base,

which we can then use to learn more,

and hopefully learn more efficiently.