Brainscape's Knowledge GenomeTM

Browse over 1 million classes created by top students, professors, publishers, and experts.


See full index

422 > Chp3 Regression > Flashcards

Chp3 Regression Flashcards

1
Q

Linear Regression Theory

A

A statistical process for estimating the relationship among variables

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
2
Q

What two variables are in regression?

A

Response Variable (dependent variable y)
Predictor Variables (independent variables, X)

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
3
Q

What is regression used for?

A

Predicting and forecasting

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
4
Q

What does linear regression try to do?

A

Use past data to predict future outcomes

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
5
Q

What are model coefficients/parameters/weights?

A

multiply them against the input values to generate your response variable

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
6
Q

What is error called between observed response Yi and predicted response Yhat

A

Residual

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
7
Q

Residual Formula

A

Yi - Yhat
observed response - predicted response

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
8
Q

Residual Sum of Squares

A

RSS, sum of each residual squared

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
9
Q

Residual Formula mathematical simple

A

Ei = (B0 + B1Xi) - (Bhat0 + Bhat1X1)

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
10
Q

Which is the best regression line?

A

The one that minimizes the sum of squared residuals

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
11
Q

Multiple R-squared

A

Will always increase as you add more predictors because increasing variance and every predictor is increasing multiple R-squared, but not every predictor is a good predictor

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
12
Q

Adjusted R-squared

A

Captures how many of those predictors you have added are actually good predictors as you add those predictors. Mult r sq and adj r sq values go up, but there will be a time where the adj r sq will plateau and drop, stop adding variables at that point

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
13
Q

Adjusted R-squared shows

A

When adding more predictors makes it worse

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
14
Q

F statistic

A

Captures how good model is, bigger the better

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
15
Q

Degrees of freedom

A

How much wiggle room you have in your data set

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
16
Q

In hypothesis testing, what must be true to support the Null hypothesis H0

A

Pvalue > alpha

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
17
Q

In hypothesis testing, what must be true to not support the Null hypothesis

A

Pvalue <= alpha

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
18
Q

What is alpha in hypothesis testing

A

The probability of Rejecting the null hypothesis given that the null hypothesis is true

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
19
Q

What is pvalue in hypothesis testing

A

The probability of getting a result as extreme as you have given that the null hypothesis is true

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
20
Q

What are the only two outcomes from hypothesis testing?

A
  • Reject H0 in favor of H1
  • Do not reject H0
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
21
Q

In hypothesis testing, we never accept //

A

H1

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
22
Q

If we are looking if a drug has an effect, what is null and alternate hypothesis?

A

null - drug has no effect
alt = drug has some effect

23
Q

What are the four questions to evaluate the fit of a regression model?

A

Is at least one of the predictors useful in predicting the response?
Do all the predictors help explain the response, or is only a subset of the predictors useful?
How well does the model fit the data?
Given a set of predictor values, what response value should we predict and how accurate is our prediction?

24
Q

What is the hypothesis test to determine if at least one predictor is useful in predicting the response?

A

H0: all betas are 0
H1: at least one beta is nonzero

25
Q

What does it mean if the F statistic is close to 1?

A

There is no relationship between response and predictors, H0 is true

26
Q

What is f statistic if H1 is true

A

F-statistic > 1

27
Q

If at least one predictor is useful, what does that mean about the p value associated with the f statistic?

A

it is very small

28
Q

What is hypothesis testing for determining if all the predictors explain the response, or if only a subset of predictors are useful?

A

For each predictor
H0: Bi = 0 (there is no relationship between predictor and response)
H1: Bi != 0 (There is some relationship between predictor and response)

29
Q

What p value makes a predictor useful? not useful?

A

Very low P value
a non low p value means we cannot reject the null hypothesis

30
Q

T value

A

Tells you how many std your beta value is from 0

31
Q

T value the farther

A

the better

32
Q

In example, what is significance that radio and newspaper have correlation in correlation matrix?

A

If we spend more on radio we are also spending more on newspaper, which should not be done because newspaper is not very correlated with sales

33
Q

Given p predictors how many models could we create

A

2^p

34
Q

two strategies for feature selection

A

Forward Selection
Backward Selection

35
Q

Forward Selection

A

Start with null model (intercept) and add predictors

36
Q

Backward Selection

A

Start with all predictors and remove variables with largest p-values

37
Q

Which statistics 3 describe how well the model fits the data?

A

Residual Standard Error RSE
R^2
RMSE

38
Q

what is r^2

A

Correlation between y and yhat

39
Q

What do we want r squared to be?

A

1, we want what we are predicting to be very correlated to the training data

40
Q

Residual Analysis

A

Sees how well the model fits the data

41
Q

Residuals should be 4

A

homosceadastic
Centered on 0 through a range of fitted values
normally distributed
uncorrelated with each other

42
Q

Homosceadastic

A

Variance in residual must not vary too much

43
Q

RSE must be

A

minimized

44
Q

If residual plot is nonlinear, what type of regression would be best?

A

quadratic or higher power regression

45
Q

Significance level (alpha)

A

Probability of making the wrong decision, given H0 is true

46
Q

Confidence Interval

A

The range of results that would be expected to contain the population parameter of interest

47
Q

Confidence Level

A

Probability that if an experiment was repeated multiple times, results will be the same

48
Q

Confidence level formula

A

1 - alpha

49
Q

What does 95% confidence interval mean?

A

If I ran the experiment 20 times, then my true value (actual mean maybe) will be present in this interval 19 times.

50
Q

Higher confidence means

A

Higher margin for error, wider confidence intervals

51
Q

prediction interval is _ than confidence interval

A

wider

52
Q

Difference between prediction interval and confidence interval

A

Prediction interval is trying to predict something, so we have to take into account the variation in the data and uncertainty in knowing the true population parameter
Confidence interval is basically drawn from the data you have

53
Q

Multi-collinearity

A

When one predictor is a linear combination of another predictor

54
Q

In the presence of multi-collinearity

A

Regression model will not work. Cannot trust the weights because when updating one weight you hold the others constant, which cant be true in multi-collinearity.

422 (10 decks)

Brainscape helps you reach your goals faster, through stronger study habits.
© 2024 Bold Learning Solutions. Terms and Conditions