Learning is very attractive for a variety of reasons, because it lets us deal with changing

environments, but it also makes our life easier.

If we have a good way of learning, then instead of actually us doing the work, we can let

the agent do the work.

There are many things which you actually learn in this course by us giving you homework

assignments.

I could explain everything and even assume that you are awake all the time.

For some things it's just more effective by letting you do the work.

There are much more of you than there is of me.

Having an agent that can learn gives us the opportunity to just basically give the essential

shape of the model or something like this only and then say oh go and learn

the go and learn the parts of the model yourself.

Okay, for that we need to modify the agents decision, decision procedures and we've looked at this kind of a design which I want to assume in the future where we have all the agents we have had so far kind of like shrink that down.

And we have that down to a particular box and we have at par with that learning element which can essentially change every part of the performance element and actually gets to know whatever the performance element does and possibly why and so on to find out.

The important thing here is that we want to talk about learning and I can't over emphasize this.

We can only talk about learning if we have an outside given standard of performance.

Learning is about improving stuff, improving your behavior.

The environment itself really doesn't care how I behave.

If I behave by walking in front of a bus, it will happily kill me.

But the environment doesn't really care.

There must be something that ultimately triggers my improvement

and getting me to learn not to walk in front of a bus.

In front of a bicycle maybe, but not in front of a bus.

So we need some kind of a performance standard against which we can optimize.

And that cannot be part of the environment.

Because otherwise I could just change the environment.

I can act on the environment.

Which would actually make it possibly easier for me instead of changing my behavior,

which is actually cheating and changing the rules.

So this is external.

The only real change to the architecture.

The idea is that these performance standards are kind of hardwired into agents.

Hunger, fear of death, reproduction rate, all those kind of things.

And if you think about the philosophy of this, I don't know whether you know this theory of selfish genes.

If you don't, I encourage you to read that.

Reproduction rate is kind of fundamentally built into evolution and genetics.

So there's a very, very low level and fundamental trigger.

So this theory of selfish genes is essentially that all humans and any kind of life form is actually a shell around the genes that want to actually replicate.

And all this complexity that you see here and there is just to make the chance of replication higher.

Which really, and the chance of replication is actually kind of built into the system.

Why?

Because genes that can replicate, will replicate and drown out all others and then therefore evolve to more efficient genes.

Which is really what drives evolution.

These are external performance standards.

And the whole of evolution is essentially a mechanism driven by numbers that tries to optimize that.

Okay, and the last thing we looked at was, I tried to give you a kind of a feeling of what learning elements could look like.

By looking at a couple of examples, right?

We always, in all of these agents, we had components which we could learn.

For instance, in alpha beta search, we could learn the evaluation function, which is something which last semester we assumed somebody gives us.

Which is quite unrealistic, because even very good chess players who must have an evaluation function,