Okay, let's start.

We've started looking at machine learning as a technique, as a technique of understanding

learning in and of itself, and of course, for building learning agents.

The attraction of learning agents is that there are lots of things you don't have to do

as the God-like agent designer, right?

So far, the idea was that we'll build an agent, we design a Bayesian network, we give preferences

and maybe at the end we are ending up with a bomb D.P. or something like this.

The wonderful thing about learning agents is that some of those things, for instance, coming

up with the CPTs in a Bayesian network is something the agent can learn itself, okay, which

means A, we can build agents that survive in a changing world, right?

We all know whether patterns are changing, so if we hard-code them into our model, our

agents will be disadvantaged.

And if we build in learning stuff anyway, we can just tell them to go out and learn the

necessary stuff anyway, right?

In a way, put them to kindergarten and school, okay?

So we've briefly talked about the makeup of learning agents.

The idea there is that we have, well, I should probably do this, yes.

We have a performance agent element which is what our agents used to be.

We have a learning element which can kind of change any part of the performance element.

Think about learning or changing the model because the agent discovered that maybe there

should be an edge from here to there and learns about that a certain conditional independence

actually doesn't hold and it's better to not assume it.

The important thing in learning is we have this external performance standard which is something

that is one of the two inputs, right?

An agent and that is part of the basic design of agents only gets sensory data, right?

That's the only way of interacting with the environment.

As for a learning agent that's not true, it gets some access to a performance standard,

right?

Exams being hungry, being too hot, all of those kinds of things, typically things that give

you a performance standard, okay?

It's external from the agent.

It's not that you make your own or the agent makes its own goal and then measures how close

the agent is to its goal.

That's not an external performance standard.

You can do this too as an agent but it's not actually what we're learning from mostly.

So we need this external incorruptible, if you will, performance standard that tells

the agent how it's doing.

We might or might not have a problem generator that is part of the information gathering

behavior.

We looked at the math of this and I always feel a bit disappointed when I compare this

to that, right?

What is learning?

Learning is given a partial function given by a couple of input output pairs, find a function

on all input, output pairs that actually somehow corresponds to the example set, to the training

set.

Oh yes, and it has to be from a predefined set of hypotheses, okay?

So that's the math view of learning, somewhat disappointing.

And then there's the agent view which already probably shows you that the inputs might not

just be real numbers, right?

They might be complex states of the or state sequences even of the performance element.