Machine Learning with Viya® 3.4® Lesson 4: Neural Networks

Question 1

What is the neural network term for a parameter estimate or slope?

Answer 1

Weight estimate

Question 2

What is the mathematical transformation that is applied to a linear combination of the input values in a MLP or Neural Network referred to as?

Answer 2

activation function

Question 3

Which neural network architecture is best for modeling nonstationary data?

Answer 3

A skip-layer perceptron is the best architecture to use with nonstationary data.

Question 4

Which hyperparameters or set of hyperparameters in Model Studio are used to control weight decay?

Answer 4

the L1 and L2 hyperparameters control weight decay

Question 5

What is the purpose of the Minibatch size option in a Neural Network mode?

Answer 5

It defines the number of training observations to calculate the model error and update the model coefficients.

Question 6

Can neural networks can select inputs in a similar fashion as a tree-based model?

Answer 6

neural network cannot select inputs like a tree-based model

Question 7

To which optimization method does the minibatch option in a Neural Network apply?

Answer 7

The Minibatch size option defines the number of training observations to use in the SGD instead of using all training observations.

Question 8

Why is a neural network referred to as universal approximators?

Answer 8

Neural networks are called universal approximators because they can model any input-output relationship, no matter how complex.

Question 9

What terms refers to a parameter estimate or slope that is associated with an input in a neural network?

Answer 9

Weight estimate is the neural network term for a parameter estimate or slope.

Question 10

Which activation function is commonly used in the target layer when modeling a binary target?

Answer 10

The logistic function is the target layer activation function (or target layer link function) that is typically used with a binary target.

Question 11

What is the limitation of modeling with a neural network?

Answer 11

Neural networks are generally considered to be “black boxes.” Because they are minimally interpretable, at best, neural networks are most useful in pure prediction scenarios.

Question 12

What is the value of a standardized variable called?

Answer 12

Z-score or standard score

Question 13

What is standardization?

Answer 13

rescaling your data to have a mean of 0 and a standard deviation of 1

Question 14

When is the identity target activation function appropriate when modeling a neural network?

Answer 14

when the target error function is normally distributed

Question 15

What is the neural network term for an intercept estimate?

Answer 15

Bias estimate

Question 16

When early stopping is used to build a neural network model, which data partition does Model Studio use to select the final model?

Answer 16

The validate partition

Question 17

What may be affected when multi-scaled variables are used in a multivariate analysis?

Answer 17

Model stability and parameter estimate precision are influenced during multivariate analysis when multi-scaled variables are used (i.e., a variable that ranges between 0 and 100 will outweigh a variable that ranges between 0 and 1.)

Question 18

What occurs during a neural network’s learning process?

Answer 18

Numerical optimization is an important part of the learning process.

Question 19

Why might you apply weight decay when building a neural network model?

Answer 19

Weight decay is one of two main methods used to avoid overfitting when building a neural network model

Question 20

What is the partition validation method?

Answer 20

With partition, you specify proportions to use for randomly assigning observations to each role.

Question 21

The Neural Network node can use weight decay to avoid overfitting. How are the L1 and L2 regularizations applied?

Answer 21

L1 penalizes the absolute value of the weights.

L2 penalizes the squared weights.

Question 22

What is an activation function?

Answer 22

a mathematical transformation that is applied to a linear combination of the input values.

Question 23

What does it mean to normalize your data?

Answer 23

Normalizing your data refers to rescaling numeric data between 0 and 1 where xmin is the variable’s minimum value, and xmax is the variable’s maximum value.

Question 24

How are missing observations handled when the box is checked to include them in a Neural Network mode?

Answer 24

Observations with missing values for interval inputs will be imputed with the mean and included.

Question 25

What are optimization methods used for?

Answer 25

Optimization methods are used to efficiently search the complex landscape of the error surface to find an error minimum.

Question 26

Which neural network architecture is best for modeling data with discontinuous input-output mappings?

Answer 26

a multilayer perceptron with two hidden layers

(adding a second hidden layer can improve performance by enabling the MLP to realize discontinuous input-output mappings)

Question 27

Is model generalization depends more on the number of weights or the magnitude of the weights when modeling a neural network?

Answer 27

Model generalization depends more on the magnitude of the weights than on the number of weights. In fact, large weights are responsible for overfitting.

Question 28

What techniques do deep learning models use to overcome the computational challenges associated with multiple hidden layers?

Answer 28

Deep learning models use fast moving gradient-based optimizations, such as Stochastic Gradient Descent, for this purpose.

Question 29

Which hyperparameters or set of hyperparameters in Model Studio controls weight decay?

Answer 29

The L1 and L2 hyperparameters control weight decay.

Question 30

How is early stopping performed in SAS VDMML?

Answer 30

SAS Visual Data Mining and Machine Learning treats each iteration in the optimization process as a separate model. The iteration with the smallest value of the selected fit statistic is chosen as the final model. This method of model optimization is also called stopped training.

Question 31

What is the global minimum?

Answer 31

A global minimum is a set of weights that generates the smallest amount of error.

Question 32

What are the two steps important in the learning process for a neural network?

Answer 32

1. find a good set of parameters that minimizes the error (avoid bad local minima)
2. ensure that this set of parameters performs well in new data sets

Question 33

What does the Number of tries property specify?

Answer 33

the Number of tries property specifies the number of times the network is to be trained using a different starting point

Question 34

What does early stopping prevent?

Answer 34

Early stopping keeps the sigmoids from becoming too steep (and steep sigmoids are thought to be responsible for overfitting)

Question 35

What property would you adjust to specify whether or not to stop training when the model begins to overfit?

Answer 35

Perform Early Stopping

Question 36

Which properties affect the number of iterations performed during model optimization?

Answer 36

Maximum iterations and Maximum time

Question 37

What are the methods available to prevent overfitting in the Perform Early Stopping property?

Answer 37

1. Stagnation – training stops after N consecutive iterations (Stagnation) without improvement in the validation partition (Note: Early stopping cannot be used if there is no validation partition. The default value is 5.)
2. Validation error goal – Specifies a goal for early stopping based on the validation error rate. When the error gets below this value, the optimization stops. This option is in effect only for networks with fewer than 6 hidden layers. The value of 0 indicates that no validation error goal is set. The default value is 0.

Question 38

Describe the architecture of a Multilayer perceptron model.

Answer 38

The first layer, called the input layer, connects to a layer of neurons called a hidden layer, which, in turn, connects to a final layer called the target or output layer.

Question 39

Which of the two optimization methods will be chosen when the Optimization method is set to Automatic?

Answer 39

The optimization method selected is based on the number of hidden layers:
For 2 or fewer hidden layers, LBFGS is used;

otherwise, SGD is used.

Question 40

In Model Studio, which two optimization methods are currently available in the Neural Network node?

Answer 40

limited memory Broyden-Fletcher-Goldfarb-Shanno (LBFGS) and variants of gradient descent

Question 41

Which optimization method is well suited for optimization problems with a large number of variables?

Answer 41

LBFGS

Question 42

Which iteration does the optimal model come from when early stopping is used?

Answer 42

one of the iterations that occur earlier than the final training iteration

Question 43

Which data partition is used to select the final model when early stopping is used to build a neural network model?

Answer 43

validate

Question 44

Neural Network equation

Answer 44

The predicted estimate, ŷ, is a weighted sum of the input features x0 to xp, weighted by the learned coefficients, or weights, w0 to wp.

Question 45

What does the Node Score Code window provide?

Question 46

Where is code to train the model based on different data sets or in different platforms located?

Answer 46

The training code window

Question 47

What shows the model’s performance based on the valid error throughout the training process when new iterations are added to achieve the final model?

Answer 47

The iteration plot

Question 48

How would you request a skip-layer perceptron in Model Studio?

Answer 48

In Model Studio, skip-layer perceptrons are constructed when the property Allow direct connections between input and target neurons, located on the Options tab, is selected

Question 49

How many hidden layers may be used in Model Studio?

Answer 49

The possible number of hidden layers in SAS Visual Data Mining and Machine Learning ranges from 0 to 10.

Question 50

What is the Bernoulli function?

Answer 50

An error function for a binary target produced by minimizing the error which is -2 times the log-likelihood.

Question 51

What is Model Stidio’s default hidden unit activation function in a Neural Network node?

Answer 51

hyperbolic tangent (tanh)

Question 53

What is a global minimum?

Answer 53

a set of weights that generates the smallest amount of error

Question 54

What is the goal of numerical optimization?

Answer 54

to minimize the error function

Question 55

How might you refer to local features that make these algorithms vulnerable to getting stuck in treacherous regions of the error surface that result in little improvement?

Answer 55

error plateaus or local minima

Question 56

The decision to stop is based on meeting at least one of what three convergence criteria?

Answer 56

1. The specified error function stops improving.
2. The gradient has no slope (in other words, the rate of change of the error function with respect to the weights is zero).
3. The magnitude of the weights stops changing substantially.