Topic 2: Machine Learning: Introduction to Algorithms

Question 1

Explain why we estimate a function with data

Answer 1

For prediction or inference reasons

Question 2

What is the role of error terms (reducible and

irreducible) and why is the irreducible error larger than zero?

Answer 2

Reducible error occurs when the estimate for f can be improved (e.g. using a better statistical model.

Irreducible error is the part of the error that cannot be reduced (because Y is also a function of the error)

The error is larger than zero because the error may contain unmeasured variables that are useful in predicting Y.

Question 3

Difference between prediction and inference

Answer 3

Prediction uses X to predict Y (f is treated as black box)

Inference is estimating f but not necessarily make predictions for Y (f cannot be tread as a black box)

Question 4

Difference between a parametric and non-parametric approach when applying a statistical learning method to the training data.

Answer 4

Parametric approach involves a two-step model-based approach.

1. Make an assumption about the functional form, or shape.
2. Select procedure that uses the training data to fit or train the model (e.g. OLS).

Non-parametric approach do not make an explicit assumption about the functional form or shape of f. Downside is that you need a large dataset.

Question 5

Describe the trade-offs between prediction accuracy, flexibility, and model interpretability, including the role of overfitting.

Answer 5

As the flexibility of a method increases its interpretability decreases. Highly flexible methods have a greater potential for overfitting.

Question 6

Reason that we might prefer a more restrictive model

Answer 6

When we are mainly interested in inference restrictive models are more interpretable.

Question 7

When is a supervised learning model preferable to unsupervised?

Answer 7

Supervised learning models are used when you want to fit a model that relates the response to predictors.

With unsupervised learning models there is no response variable that can supervise the variable (e.g. clustering).

Question 8

Difference between quantitative and qualitative problems?

Answer 8

Quantitative -> regression problems (numerical values)

Qualitative -> categorical problems (classes)

Question 9

Interpret the Mean Squared Error (MSE)

Answer 9

MSE will be smaller if the predicted responses are very close to the true responses.

Question 10

Explain the goal of measuring the quality of fit by minimizing training and test mean square errors (MSEs)

Answer 10

quality of fit -> how well predictions match observed data

The quality of fit is measured by MSE, you want to choose the method that has the lowest test MSE.

Question 11

Implications of different levels of flexibility

(degrees of freedom) for both training and test MSEs.

Answer 11

As model flexibility increases, training MSE will decrease, but the test MSE may not.

Overfitting the data (small training MSE but large test MSE)

Question 12

What does it mean when a method is overfitting the data?

Answer 12

When the model is working too hard to find patterns in the training data, and may be picking up some spurious patterns.

Question 13

Explain the purpose of cross-validation.

Answer 13

A method for estimating test MSE using the training data

Question 14

Explain the bias-variance trade-off with an MSE decomposition into three fundamental quantities.

Answer 14

Expected test MSE can be decomposed into:

1. variance of f
2. the squared bias of f
3. the variance of the error

Lower bias (better fit of training data) can lead to higher variance (in the testing data)

Question 15

Ideal machine learning algorithm characteristics

Answer 15

The algorithm has low bias (can model the true relationship accurately) and low variance (by producing consistent predictions across different datasets)

Question 16

Describe the features of a Bayes classifier (two classes)

Answer 16

A Bayes classifier assigns each observation to the most likely class given its predictor values.

The Bayes decision boundary is the line which represents the points where the probably is exactly 50%.

Question 17

What is the Bayes error rate?

Answer 17

It is the lowest possible test error rate in classification which is produced by the Bayes classifier

Question 18

Apply/calculate the Bayes error rate

Answer 18

1 - [the class with the highest probability of Y belonging to class J]

or 1-E[maxj PR(Y = j|X)]

Question 19

How is K-nearest neighbor classifier related to the Bayes classifier?

Answer 19

KNN attempts to estimate the conditional distribution of Y given X, then classifies a given observation with the highest estimated probability.

Question 20

What are the effects of a low K and high K in K-nearest neighbor?

Answer 20

Lower K on training data gives higher flexibility (or low bias) but has a very high variance when applied to test data.

High K will give lower flexibility (high bias) but lower variance when applied to test data (decision boundary will become more linear with higher Ks)

Question 21

Describe the use of a training set as an alternative to a rules-based program to recognize digits.

Answer 21

A rules-based program would lead to a lot of exceptions and caveats while using training data you can use examples to infer rules for recognizing digits.

Question 22

What is a perceptron and how does it work?

Answer 22

A perceptron is a artificial neuron used in a neural network. It takes several binary inputs (x1, x2, x3) and produces a single binary output.

Question 23

Calculate the output of a perceptron neuron

Answer 23

Sum of WjXj + bias = 1 when positive, 0 when negative

Question 24

Describe the intuition of a perceptron as a decision-making device

Answer 24

You use perceptrons to weigh evidence to make decisions.

Question 25

Describe a perceptron as a NAND gate and what it implies for perceptron networks concerning computing logical functions.

Answer 25

A combination of the same integers (eg. 11 or 00) will produce a negative output (and thus 0)

Question 26

Explain how perceptron neurons are more than new types of NAND gates.

Answer 26

With perceptron you can create learning algorithms that can automatically tune the weights and biases of a network of neurons. (ie more than just a conventional circuit with a NAND gate)

Question 27

Name a limitation of perceptron neurons that can be overcome by sigmoid neurons?

Answer 27

Perceptrons are not able to learn, while sigmoid neurons are able to be modified so that small changes in their weights and bias only cause a small change in output.

Question 28

How does a perceptron neuron differentiate from a sigmoid neuron?

Answer 28

A perceptron takes on the value 1 or 0 while a sigmoid ranges between 0 and 1. It takes on a value close to 1 for very high positive values of x and a values close to 0 very high negative values of x.

Question 29

Explain the importance of the smoothness of the sigmoid function.

Answer 29

The smoothness of the function means that a small change in the weights and in the bias will produce a small change in the output (so not a 0 or 1)

Question 30

Identify components of a simple network with appropriate terminology.

Answer 30

Most left you have the input layer with input neurons, and most right you have the output layer with output neurons. Any layer in between is called a hidden layer.

Question 31

Describe the central feature of a feed-forward network.

Answer 31

Information is always fed forward, never fed back

Question 32

Compare and contrast feedforward networks with recurrent networks.

Answer 32

• Recurrent networks allow for feedback loops
• Less influential and less powerful
• Closer in spirit to how our brain works

Question 33

Compare and contrast feedforward networks with recurrent networks.

Answer 33

• Recurrent networks allow for feedback loops
• Less influential and less powerful
• Closer in spirit to how our brain works

Question 34

Calculate the required input neurons for classifying an individual digit in an image of a specific size in pixels.

Answer 34

width in pixels times height in pixels

Question 35

Explain the choice to use ten output neurons instead of four for classifying an individual digit.

Answer 35

The network would have trouble determining what the most significant part of the digit was (in the hidden layers it looks at multiple parts of a digit)

Question 36

Explain why minimizing a quadratic cost function is preferable to working with other types of cost functions.

Answer 36

It is a smooth function that makes it easy to measure improved performance by changing weights and biases.

Question 37

How can you apply gradient descent in a neural network?

Answer 37

You can use gradient descent to find the weights and biases which minimize the cost in the cost function

Question 38

Explain how quickly stochastic gradient descent can speed up learning given a training set size, n, and a mini-batch size, m.

Answer 38

You estimate the gradient by computing the cost function for a small sample of randomly chosen training inputs.

 training size (n) = 6.000
mini batch size (m) = 10
speedup in estimating gradient = 600 (n/m)

Question 39

Describe the role of hyper-parameters and their impact on output for each epoch.

Answer 39

Hyper-parameters are the variables that set the network structure and are set before training (e.g. learning rate). If you choose your hyper-parameters poorly, you can get bad results.

Question 40

Describe deep learning in terms of neural networks and their performance relative to networks that are not based on deep learning methods.

Answer 40

Deep learning are based on stochastic gradient descent and backpropagation among other things. They have more layers and perform far better on many problems than shallow neural networks.

Question 41

Describe Reinforcement Learning (RL)

Answer 41

A subfield of ML that teaches an agent how to choose an action from its action space, within a particular environment, in order to maximize rewards over time.

Question 42

Construct a task as an Reinforcement problem.

Answer 42

An RL has four essential elements:

1. An agent (the program you train)
2. Environment (The world, real or virtual)
3. Action (A move made by the agent)
4. Rewards (Evaluation of an action, positive or negative)

Question 43

Compare and contrast RL with supervised and unsupervised learning.

Answer 43

1. Static vs. Dynamic -> RL = Dynamic
2. No Explicit Right Answer -> RL = trial and error
3. RL Requires Exploration -> RL looks for new ways
4. RL is a Multiple-Decision Process -> RL = decision chain

Question 44

Define the Markov property and Markov chain.

Answer 44

Markov property: memoryless property of a stochastic - or randomly determined - process.

Markov Chain: Stochastic model whereby the probability of each event depends only on the state attained in the previous event.

Question 45

Describe how the Markov property works.

Answer 45

The state of X at time t+1 only depends on the preceding state of X at time t, and is independent of past states.

Question 46

Describe how the Markov chain puts the Markov property into action.

Answer 46

the Markov Chain, which works with S, a set of states, and P, the probability of
transitioning from one to the next.

Question 47

How does a Markov Decision Process differ from the Markov Chain?

Answer 47

It brings actions into play. The next state is not only related to the current state but also the actions taken in the current state.

Question 48

Define the MDP

Answer 48

An MDP is a 5-tuple (S, A, P, R, y) where:

1. S is a set of states
2. A is a set of actions
3. P is the probability that an action in state s at time t will lead to state s’ at time t+1
4. R is the immediate reward received after a transition from state s to s’ due to action a
5. y is the discounted factor

Question 49

Explain why and how to use discounted rewards.

Answer 49

Future rewards can be valued differently than current rewards.

You can use the discount rate to discount rewards from time t to T.

Question 50

Describe the rewards in relation to the discount rate at different discount rates.

Answer 50

If the discount rate is close to 0, future rewards won’t count for much in comparison to immediate rewards.