Chapter 1,2 - Overview

Question 1

Statistical Learning refers to…

Answer 1

A vast set of tools for understanding data. These tools can be unsupervised or supervised.

Question 2

Supervised vs Unsupervised Learning:

Answer 2

Supervised: Predicting/estimating an output based on one or more inputs. We have labels for the data already.

Unsupervised: inputs with no supervising output,; but we can still learn relationships and structure from data.

Question 3

When we want to predict our estimate a continuous or quantitative variable, what kind of problem is this?

Answer 3

Linear Regression

Question 4

What is a Classification Problem?

Answer 4

When we are trying to predict a non-numerical value. Eg. Will stock prices go UP or DOWN (ignoring by how much)

Question 5

What is a clustering problem?

Answer 5

Where we want to group variables based on observed characteristics.

No output variables for corresponding input variables, however we want to see data groups, structures and relationships.

Unsupervised.

Question 6

What model/method could we use to predict a qualitative variable?
Eg patient lives or dies, stock market goes up or down?

Answer 6

Logistic Regression or Linear Discriminant Analysis

Question 7

In statistics, what does this notation stand for:
n
p
x(small i,j)
X
T
y(small i)

Answer 7

n = number of total distinct data points
p = number of variables
x(small i,j) = ith observations of jth variable
X = n x p matrix whose (i,j)th element is xij
T = Transpose
y(small i) = ith observation of variable we wish to make predictions on

Question 8

What are some of the names given to input vs output variables in ISLR.

Answer 8

Input: predictors, independent variables, fewtures, also just variables

Output: response or dependent variable - often denoted by symbol Y

Question 9

Describe Y = f(X) + €

Answer 9

Assumptions:

f is some fixed but unknown function

€ is a random error term, which is independent of X and has a mean of zero

In this formulation, f represents the systematic information that X provides about Y.

Question 10

In essence, statistical learning refers to a set of approaches for estimating:

Answer 10

f

Some unknown function f that we can use to predict/estimate Y

Question 11

What are the two main reasons for estimating f(or the function)?

Answer 11

Two main reasons:

Prediction and Inference

Question 12

The accuracy for Y(^hat) as a prediction for Y depends on two quantities….

Answer 12

Reducible error: in general, f(^hat) is not a perfect estimate of f, and this inaccuracy will introduce some error. This error is reducible because we can potentially improve the accuracy of f(hat) by using the most appropriate statistical learning technique for estimating f.

Irreducible error:
Even if we could perfectly predict f(X), we would still see error in our predictions. No matter how well we estimate f(X), we cannot reduce the error caused by €.
This is because Y is also a function of €, which by definition cannot be predicted by X.

Question 13

Inference vs Prediction to estimate f

Answer 13

Prediction: estimate value for Y based on inputs X

Inference: what effect does change in input X have on Y.
We want to understand the relationship between X and Y

Question 14

Inference vs Prediction example

Answer 14

Inference:
How much extra will a house be worth if it has a view of the river?

Prediction:
Is the house under-valued or overvalued?

Question 15

If we want to find a function f(hat) that estimates Y ~= f(X) for any observation (X,Y), what type of statistical learning approaches can we take?

Answer 15

Parametric:
Involve a two-step model-based approach.
1. Make an assumption about the functional form. Eg. Assume that f is linear in X(linear model).

2. After selecting a model, we need a procedure that uses training data to fit or train model.

Non-parametric: no assumption of functional form. Seeks to estimate f by getting as close to data points ad possible.

Question 16

Estimating f.

Parametric approach: Advantages vs disadvantages

Answer 16

Advantages: assuming form for f simplifies the problem and makes it easier (generally speaking) to estimate a set of parameter.

Disadvantages: the model we choose will generall not much the true unkown form of f. If form is too far from f, estimate will be poor.

Question 17

Estimating f.

Non parametric methods advantages vs disadvantages:

Answer 17

Advantages: potential to fit a wider range of possible shapes for f. Avoid danger of functional form used to estimated f being different from real f as no assumptions are made.

Disadvantages:
Major one is since they don’t reduce problem of estimating f to a small number of paramaters, a very large number of observations is required in order to obtain an accurate estimate for f.

Question 18

What is the trade-off between prediction accuracy and model interpretability?

Answer 18

As Interpretability increases:
Flexibility decreases….

And vice versa