Yeah, welcome back everybody to our seminar Meta-Learning.

Today we will have a presentation by Anil Boga-Jajak and the presentation is entitled

Neural Architecture Search with Reinforcement Learning and I'm very much looking forward

to this presentation.

Anil, the stage is yours.

Okay, thank you, Professor.

Hello everyone, today I will present the paper Neural Architecture Search with Reinforcement

Learning by Beretsofen Kockfaholia of Google.

First, I will make an introduction and talk about the motivation of the proposed methods.

Then in the second part, we will see this method in detail.

And finally, we will see how this method is applied to the different problems as well

as their results.

Now let's begin.

As opposed to the fifth-shot learning approaches we've seen in the preceding weeks, this paper

is exclusively about hyperparameter search.

The method proposed in this paper focuses on neural network architectures only.

A hyperparameter is any tunable parameter of a framework.

As you see in the examples of hyperparameters, they are the ones that are tunable by us,

so they can be changed.

The purpose is to find the optimum values that gives the minimum loss and maximum score,

but most methods are still limited that they only search models from a fixed-length space,

and therefore it's difficult to ask them to generate a variable-length configuration that

specifies both the structure and connectivity of a framework.

This is crucial for a neural framework since a fixed-size configuration may not give the

best overall results even with the most suitable hyperparameters.

Therefore a neural network shouldn't be restricted to the fixed-length architecture.

Besides this, connectivity among parameters also must be considered since a decision on

one parameter may interact with another.

This figure shows an overview of the neural architecture search.

The process starts with the generation of an architecture by a controller recurrent

neural network.

Then, a child model is built with the generated hyperparameters and trade on a validation

set.

The NAS is a gradient-based method.

Hence after we get an accuracy R from the training phase, it's used as the reward signal

for reinforcement learning.

And now we can compute the policy gradient to update the controller.

As a result, the controller will learn to give higher probabilities to the architectures

that may give higher accuracies over time.

As we've seen, as a controller we can use a recurrent model to generate the model descriptions

of neural networks.

As mentioned in the introduction part, the structure and connectivity of a neural network

can be specified by a variable-length string.

Before diving in the deeper, let's get a better understanding of why using RNN makes more

sense here.

In the recurrent neural networks, the activations here denote that a superscript provides information

from previous time steps to the next time step.

In our case, next hyperparameters to be predicted.

The structure you see in the figure is the decoder of the sequence learning.

I directly focused on this since that's also the idea borrowed by our structure as well.