W1-Machine Learning Modeling Pipelines in Production

Question 1

Neural Architecture Search or NAS is at the heart of AutoML. There are three main parts to Neural Architecture Search, a ____, a ____and a _____.

Intro to AutoML 02:30

Answer 1

search space
search strategy
performance estimation strategy

Question 2

The search space defines the range of architectures which can be represented. To reduce the size of the search problem, we need to limit the search space to the architectures which are best suited to the problem that we’re trying to model. This helps reduce the search space, but it also means that ____ will be introduced, which might prevent Neural Architecture Search from finding architectural blocks that go beyond current human knowledge.

Intro to AutoML 02:45

Answer 2

a human bias

Question 3

Neural Architecture Search is a sub field of AutoML. True/False

Intro to AutoML 04:18

Answer 3

True

Nowadays, some researchers mistakenly equate AutoML with neural architecture search (NAS). However, a clear distinction is required, as the process of automating architecture engineering is strictly called NAS.

Question 4

There are two main types of search spaces, ____ and ____.

Understanding Search Spaces 00:00

Answer 4

macro, micro

Question 5

The search strategies in neural architecture search need to estimate the performance of generated architectures so that they can generate architectures that perform better. What are 3 approaches used for it?

Measuring AutoML Efficacy 01:16

Answer 5

1. Lower fidelity estimates
2. Learning curve extrapolation
3. Weight inheritance or network morphism.

The simplest approach is to measure the validation accuracy of each architecture that is generated.This becomes computationally heavy and IS NOT USED

Question 6

How does Lower fidelity estimate for performance estimation strategy work?

Measuring AutoML Efficacy 01:25

Answer 6

Lower fidelity or lower precision estimates try to reduce the training time by reframing the problem to make it easier to solve by training on a subset of data or using lower resolution images, for example, or using fewer filters per layer and fewer cells. It greatly reduces the computational cost, but ends up underestimating performance. (It’s not worth using)

That’s okay if you can make sure that the relative ranking of the architectures does not change due to their lower fidelity estimates. But unfortunately, recent research has shown that this is not the case

Question 7

How does Learning Curve Extrapolation for performance estimation strategy work?

Measuring AutoML Efficacy 02:13

Answer 7

Learning Curve Extrapolation is based on the assumption that you have mechanisms to predict the learning curve reliably, and so extrapolation is a sensitive and valid choice. Based on a few iterations and available knowledge, the method extrapolates the initial learning curves and terminates all architectures that perform poorly.

Question 8

How does Network Morphism work?

Measuring AutoML Efficacy 02:59

Answer 8

Another method for speeding up architecture search initializes the weights of novel architectures based on the weights of other architectures that have been trained before, similar to the way that transfer learning works. One way of achieving this is referred to as network morphism.

In simple terms, network morphism is a way of comparing and transforming one network into another network while preserving its overall structure and relationships.

Question 9

What are the steps of classifying images using AutoML Vision (Google Cloud)?

Assignment Setup

Answer 9

1. set up AutoML
2. create the dataset in Vision by first uploading images to Cloud Storage and then proceeding to create the dataset.
3. AutoML then performs the actual training
4. selects the best model for the task
5. Finally, we can deploy the model trained by AutoML Vision and test the model by generating predictions using the deployed model.