Test Prep

Question 1

What is the formula for the simple linear regression model?

Answer 1

y=β _0 +β_1*x_1

Where:
y: Dependent (response) variable
x_1: Independent (predictor) variable
𝛽_0: Intercept
𝛽_1: Slope (coefficient)

Question 2

What does the dependent variable (y) represent in simple linear regression?

Answer 2

y is the dependent variable (response). It’s the outcome we are trying to predict or explain, like house price in a real estate model.

Question 3

What does the intercept (𝛽_0) mean in simple linear regression?

Answer 3

β _0 is the intercept. It’s the predicted value of y when x=0. It shows where the line crosses the y-axis. For example, it could represent the price of a house with zero square feet.

Question 4

What does the slope (𝛽_1) mean in simple linear regression?

Answer 4

β_1 is the slope (coefficient). It shows how much y changes when x increases by 1 unit. For example, it might show how much house price increases for each extra square foot.

Question 5

What is the error term (ϵ) in simple linear regression?

Answer 5

ϵ represents the error or residual, the difference between the actual y and the predicted y. It accounts for variation in y that isn’t explained by x.

Question 6

What does the intercept tell us when the predictor variable (x) is zero?

Answer 6

It tells us the predicted value of y when x=0, essentially giving the baseline value of the response variable.

Question 7

What does covariance measure?

Answer 7

Covariance measures whether two variables tend to move in the same direction (positive) or in opposite directions (negative).

Question 8

What does a positive covariance indicate?

Answer 8

A positive covariance indicates that the two variables increase or decrease together.

Question 9

What does a negative covariance indicate?

Answer 9

A negative covariance indicates that as one variable increases, the other decreases.

Question 10

Why is covariance difficult to interpret?

Answer 10

Covariance is hard to interpret because its value depends on the scale of the variables and can be any large positive or negative number.

Question 11

What does correlation measure?

Answer 11

Correlation measures both the direction and the strength of the relationship between two variables.

Question 12

What are the value boundaries of correlation?

Answer 12

Correlation is always between -1 and 1, with:

1 meaning a perfect positive relationship.
-1 meaning a perfect negative relationship.
0 meaning no relationship.

Question 13

How is correlation different from covariance?

Answer 13

Correlation is standardized and bounded between -1 and 1, making it easier to interpret than covariance, which has no fixed scale.

Question 14

What does R-squared measure in a regression model?

Answer 14

R-squared measures the proportion of the total variability in the outcome variable that is explained by the predictor variable(s) in the model.

Question 15

What is the range of R-squared values?

Answer 15

R-squared values range from 0 to 1. A value of 0 means the model explains none of the variability, while 1 means the model explains all the variability.

Question 16

How is R-squared related to residuals?

Answer 16

R-squared is calculated as
1− (TotalSumofSquares(TSS)/SumofSquaredResiduals(SSR)). It reflects how much of the data’s variation is captured by the model compared to the residuals

Question 17

What does a high R-squared value indicate about residuals?

Answer 17

A high R-squared value indicates that the residuals are small, meaning the model’s predictions are close to the actual values and the model fits the data well.

Question 18

What does a low R-squared value indicate about residuals?

Answer 18

A low R-squared value indicates that the residuals are large, meaning the model’s predictions are far from the actual values and the model does not fit the data well.

Question 19

What are predicted values in a regression model?

Answer 19

Predicted values (ŷ) are the values estimated by the regression model for the outcome variable based on the predictor variables.

Question 20

What are observed values of y in a regression model?

Answer 20

Observed values (y) are the actual values of the outcome variable collected during data gathering.

Question 21

How is a residual calculated in a regression model?

Answer 21

Residuals are calculated as:

Residual = Observedvalue (𝑦) − Predictedvalue (ŷ)

Question 22

What does a positive residual indicate?

Answer 22

A positive residual indicates that the model under-predicted the outcome (the actual value is higher than the predicted value).

Question 23

What does a negative residual indicate?

Answer 23

A negative residual indicates that the model over-predicted the outcome (the actual value is lower than the predicted value).

Question 24

What is the range of correlation coefficients?

Answer 24

Correlation coefficients range from -1 to 1.

Question 25

What does a correlation coefficient of 1 indicate?

Answer 25

A correlation coefficient of 1 indicates a perfect positive correlation; as one variable increases, the other variable increases proportionally.

Question 26

What does a correlation coefficient of -1 indicate?

Answer 26

A correlation coefficient of -1 indicates a perfect negative correlation; as one variable increases, the other variable decreases proportionally.

Question 27

What does a correlation coefficient of 0 indicate?

Answer 27

A correlation coefficient of 0 indicates no linear relationship; the variables do not have a consistent pattern of moving together.

Question 28

How do you interpret a positive correlation coefficient (e.g., 𝑟 > 0)?

Answer 28

A positive correlation coefficient means that as one variable increases, the other variable also increases.

Question 29

How do you interpret a negative correlation coefficient (e.g., r<0)?

Answer 29

A negative correlation coefficient means that as one variable increases, the other variable decreases.

Question 30

What does a correlation coefficient close to 0 indicate?

Answer 30

A correlation coefficient close to 0 indicates a weak or negligible relationship between the variables.

Question 31

What does a correlation coefficient around 0.3 to 0.7 (or -0.3 to -0.7) suggest?

Answer 31

This suggests a moderate relationship between the variables. (Subject to interpretation)

Question 32

What does a correlation coefficient close to 1 or -1 indicate?

Answer 32

This indicates a strong relationship; the variables move closely in sync with each other.

Question 33

What should you remember about correlation in relation to causation?

Answer 33

Correlation does not imply causation; a high correlation doesn’t mean one variable causes the other.

Question 34

What kind of relationships does the correlation coefficient capture?

Answer 34

The correlation coefficient captures linear relationships only; non-linear relationships are not well represented.

Question 35

How can outliers affect the correlation coefficient?

Answer 35

Outliers can heavily influence the correlation, making it seem stronger or weaker than it actually is for most of the data.

Question 36

How do you interpret a positive slope coefficient (𝛽) in a regression model?

Answer 36

A positive slope coefficient means that as the predictor variable increases, the outcome variable is expected to increase as well. It shows the rate of increase in the outcome for every one-unit increase in the predictor.

Question 37

How do you interpret a negative slope coefficient (𝛽) in a regression model?

Answer 37

A negative slope coefficient means that as the predictor variable increases, the outcome variable is expected to decrease. It shows the rate of decrease in the outcome for every one-unit increase in the predictor.

Question 38

What does the magnitude of a slope coefficient tell you in a regression model?

Answer 38

The magnitude of a slope coefficient indicates how strong the relationship is between the predictor variable and the outcome variable. Larger coefficients mean a bigger effect on the outcome, while smaller coefficients indicate a weaker effect.

Question 39

How would you use the intercept and slope coefficients to make predictions in a regression model?

Answer 39

Use the intercept as the baseline value of the outcome when predictors are zero. Add the product of the slope coefficients and their corresponding predictor values to the intercept to make predictions about the outcome.

Question 40

What does the t-distribution help with in regression analysis?

Answer 40

the t-distribution helps determine if the estimates from the regression are statistically significant and reliable.

Question 41

What is a t-statistic in regression?

Answer 41

The t-statistic tells us if a variable in your model has a strong impact on what you’re trying to predict. It does this by comparing how big the variable’s effect is to the variability of that effect. A large t-statistic means the effect is strong and likely real, while a small one means it might just be random noise.

Example:
If you’re analyzing how hours studied affects exam scores:

The t-statistic for the coefficient of hours studied helps determine if the relationship you observe is likely due to a true impact of studying hours on scores, or if it might be just a coincidence in your sample.

Question 42

What does a p-value represent?

Answer 42

The p-value tells you how likely it is to observe your data (or something more extreme) if H₀ were true. A low p-value means that it’s unlikely the data you observed could have occurred just by random chance under H₀.

Question 43

How do you interpret a p-value of 0.03?

Answer 43

A p-value of 0.03 means there’s a 3% chance that the observed effect is due to random chance rather than a real effect.

Question 44

How does the t-distribution relate to p-values?

Answer 44

the t-distribution is used to calculate the p-value from the t-statistic, which tells you if your results are likely due to chance or if they reflect a true effect.

Imagine you’re checking if a new study method really works. After analyzing the test score changes, you calculate a t-statistic of 2.45. Using the t-distribution, you find that there’s only a 3.7% chance you’d see such a result if the method didn’t actually improve scores. Since this is a small chance, you conclude the method likely does have an effect.

Question 45

What is the goal of the least squares method in relation to the arithmetic mean?

Answer 45

The goal is to find the line or model that minimizes the total squared differences between the observed values and the predicted values from the model.

Question 46

Why do we use squared differences in the least squares method?

Answer 46

Squaring the differences amplifies larger errors more than smaller ones, ensuring the model minimizes the biggest discrepancies, making the model more robust overall.

Question 47

How does the arithmetic mean act as the least squares deviation estimator?

Answer 47

The arithmetic mean minimizes the sum of squared differences between itself and all data points, making it the best estimate to represent the data.

Question 48

What happens if you use a number other than the arithmetic mean to calculate squared differences?

Answer 48

Using any number other than the arithmetic mean will result in larger total squared differences compared to using the mean.

Question 49

Why is the arithmetic mean considered the “best” average?

Answer 49

The arithmetic mean is the “best” average because it’s the number that fits the data most evenly (Its locationed mostly at the center of the data). It makes the total of all the squared gaps between itself and the data points as small as possible.

Question 50

What does “least squares” refer to?

Answer 50

mathematical method used to find the best-fitting line or model for a set of data points by minimizing the sum of the squared differences (errors) between the observed values and the values predicted by the mode

Question 51

How does choosing any number other than the mean affect squared deviations?

Answer 51

Choosing any number other than the mean results in a larger total of squared deviations compared to choosing the mean.

Question 52

What is the effect of the arithmetic mean on squared deviations?

Answer 52

The arithmetic mean minimizes the total squared deviations (differences) between itself and the data points. This means the sum of the squared differences between each data point and the mean is the smallest possible compared to any other number.

Question 53

What does a statistically significant p-value (e.g., < 0.05) for a predictor mean?

Answer 53

It means there is evidence that the predictor has a significant effect on the outcome variable.

Question 54

What does a high R-squared value (close to 1) in regression output suggest?

Answer 54

It suggests that the predictor(s) explain a large portion of the variability in the outcome variable.

Question 55

What does a confidence interval that does not include zero around a coefficient indicate?

Answer 55

It indicates that the predictor has a meaningful effect on the outcome variable.

Question 56

If the R-squared value of your regression model is 0.75, what hypothesis could you make?

Answer 56

Hypothesis: “Approximately 75% of the variation in the outcome variable can be explained by the predictor(s).”

Question 56

How do you interpret a 95% confidence interval for a predictor of (3, 7)?

Answer 56

Hypothesis: “The effect of the predictor on the outcome variable is likely between 3 and 7 units.”

Question 57

What is the purpose of a confidence interval in regression?

Answer 57

A confidence interval provides a range of values within which the true parameter value is likely to fall, accounting for uncertainty in the estimate.

Question 58

How do you construct a confidence interval around a regression parameter estimate?

Answer 58

Place a “window” (interval) around the parameter estimate, using a margin of error to define the range from a lower bound to an upper bound.

Question 59

How do you interpret the confidence interval for a regression parameter?

Answer 59

The interval represents a range of values within which you are confident the true parameter value lies. For example, a 95% confidence interval means that 95 out of 100 intervals constructed this way would contain the true parameter.

Question 60

If a regression coefficient estimate is +5 and the margin of error is ±2, what is the confidence interval?

Answer 60

The confidence interval is from 3 to 7.

Question 61

What does a confidence interval of 3 to 7 for a regression parameter tell you?

Answer 61

It suggests that the true effect of the predictor is likely between 3 and 7 units, and you can be confident about this range based on your sample data.

Question 62

What does a 95% confidence level mean when interpreting a confidence interval?

Answer 62

It means that if you took 100 different samples and constructed intervals in the same way, approximately 95 of those intervals would contain the true parameter value.

Question 63

Why is it important to understand the margin of error in a confidence interval?

Answer 63

The margin of error determines how wide the confidence interval is, reflecting the degree of uncertainty about the parameter estimate.

Question 64

What does the residual indicate in regression analysis?

Answer 64

The residual indicates the part of the outcome that the model could not predict, showing the error or difference between the actual and predicted values.

Question 65

What does Adjusted R-squared account for when comparing models?

Answer 65

Adjusted R-squared accounts for the number of predictors in the model and penalizes adding too many predictors, helping to prevent overfitting.

Question 66

Why might a simpler model perform better than a more complex one?

Answer 66

A simpler model is less likely to overfit the data, meaning it may perform better on new, unseen data.

Question 67

How can cross-validation help in comparing models?

Answer 67

Cross-validation tests how well each model performs on different subsets of data, providing a more reliable sense of how it will perform on new data.

Question 68

What is the trade-off when comparing a simple model versus a complex model?

Answer 68

A complex model may fit the current data better but risks overfitting, while a simple model may be less accurate but better generalizes to new data.

Question 69

What is overfitting in the context of regression models?

Answer 69

Overfitting occurs when a model is too complex and captures noise in the data rather than the true underlying pattern, leading to poor performance on new data.

Question 70

How does R-squared help in evaluating model performance?

Answer 70

R-squared shows how well the model explains the variation in the outcome. However, to compare models with different predictors, Adjusted R-squared and other criteria should also be considered.

Question 71

What are residuals in regression?

Answer 71

Residuals are the differences between the observed values and the predicted values from a regression model. They represent the part of the outcome that the model couldn’t predict, showing how far off the model’s predictions are from the actual data points.

Question 72

What does the F-test help you determine in model comparisons?

Answer 72

The F-test helps you determine if adding more predictors to your model significantly improves its ability to explain the outcome, comparing a more complex model to a simpler one.

Question 73

What question does the F-test answer when comparing models?

Answer 73

The F-test answers: “Does this model with more predictors do a better job than a simpler model?”

Question 74

In layman’s terms, how does the F-test work when adding predictors?

Answer 74

The F-test checks if adding a new predictor to your model meaningfully improves the predictions. If the F-test result is significant (low p-value), the new model is better.

Question 75

Why is Adjusted R-squared more reliable than regular R-squared?

Answer 75

Adjusted R-squared is more reliable because it only increases if new predictors truly improve the model, rather than just adding predictors that don’t help much.

Question 76

What does Adjusted R-squared prevent in your model?

Answer 76

Adjusted R-squared prevents you from adding too many unnecessary variables, as it adjusts the fit measure to account for the number of predictors.

Question 77

How are the F-test and Adjusted R-squared useful together?

Answer 77

The F-test shows if adding predictors improves the model, while Adjusted R-squared ensures that you’re not adding predictors that don’t really help, giving you a better quality fit.

Question 78

In simple terms, what is the difference between the F-test and Adjusted R-squared?

Answer 78

The F-test checks if adding more predictors improves the model overall, while Adjusted R-squared gives a more accurate measure of how well the model explains the data without adding unnecessary complexity.

Question 79

What is the null hypothesis (H₀) in a linear regression model?

Answer 79

The null hypothesis states that the predictor has no effect on the outcome, meaning the coefficient (slope) is zero.

Question 80

What is the alternative hypothesis (H₁) in a linear regression model?

Answer 80

The alternative hypothesis states that the predictor does affect the outcome, meaning the coefficient (slope) is not zero.

Question 81

What is a t-test used for in regression analysis?

Answer 81

A t-test is used to check how far the estimated coefficient (slope) is from zero, determining if the predictor has a significant effect on the outcome.

Question 82

What do we do if the p-value is less than 0.05?

Answer 82

If the p-value is less than 0.05, we reject the null hypothesis, meaning we have evidence that the predictor affects the outcome.

Question 82

What does the p-value tell us in a hypothesis test?

Answer 82

The p-value tells us the probability that we would observe this coefficient if the null hypothesis (no effect) were true. A small p-value suggests that the predictor has a significant effect.

Question 83

What do we do if the p-value is greater than 0.05?

Answer 83

If the p-value is greater than 0.05, we fail to reject the null hypothesis, meaning we do not have enough evidence that the predictor affects the outcome.

Question 84

What does a small p-value suggest in hypothesis testing?

Answer 84

A small p-value suggests that the predictor has a significant effect on the outcome, and we should reject the null hypothesis.

Question 85

What does it mean to reject the null hypothesis?

Answer 85

Rejecting the null hypothesis means we’ve found evidence that the predictor a

Question 86

What does it mean to fail to reject the null hypothesis?

Answer 86

Failing to reject the null hypothesis means there is no evidence that the predictor has a significant effect on the outcome (the slope could be zero).

Question 87

What happens when you add an additional predictor to a regression model?

Answer 87

Adding another predictor provides more information to the model, helping explain the outcome more effectively by considering additional factors.

Question 88

Does adding a new predictor replace the existing ones?

Answer 88

No, each predictor keeps its own effect and gets its own coefficient (slope), showing how it affects the outcome independently of the others.

Question 89

Why might adding a predictor improve the model?

Answer 89

Adding a predictor can improve the model’s accuracy because it helps explain more of the variability in the outcome by accounting for additional factors.

Question 90

What does each new predictor get in a multiple regression model?

Answer 90

Each new predictor gets its own slope (coefficient), which shows how much the outcome changes with that predictor, assuming other factors are held constant.

Question 91

Can adding a predictor always help the model?

Answer 91

No, sometimes adding a predictor doesn’t help much if it doesn’t explain much more about the outcome. It may not improve the model’s predictions.

Question 92

What is the effect of adding a useful predictor to a model?

Answer 92

A useful predictor improves the model by explaining additional variability in the outcome, making predictions more accurate.

Question 93

How do predictors affect the outcome when there are multiple in the model?

Answer 93

Each predictor affects the outcome independently, and the model calculates how much the outcome changes with each predictor while keeping the others constant.

Question 94

What are qualitative input features in a multiple regression model?

Answer 94

Qualitative input features are categorical variables that represent groups or categories, such as gender, education level, or location

Question 95

Why do we need to code qualitative features in regression analysis?

Answer 95

We need to code qualitative features because regression models require numerical input, and coding converts categories into a numerical format.

Question 96

What is dummy coding in regression analysis?

Answer 96

Dummy coding converts a qualitative feature into multiple binary variables, each representing a category, with one category serving as the reference group.

Question 97

How would you code the qualitative feature “Color” with categories Red, Blue, and Green using dummy coding?

Answer 97

You would create two dummy variables:

Color_Red: 1 if Red, 0 otherwise
Color_Blue: 1 if Blue, 0 otherwise (Green would be the reference category)

Question 98

In a regression model, what does a coefficient for a dummy variable indicate?

Answer 98

It indicates how much the outcome variable is expected to change when that category is present compared to the reference category.

Question 99

What would a regression output look like for a qualitative feature “Location” coded as Urban, Suburban, and Rural?

Answer 99

You could have:

Intercept = $200,000
Coefficient for Location_Urban = +$50,000
Coefficient for Location_Suburban = +$30,000

Question 100

How would you interpret the coefficient for Location_Urban in the previous example?

Answer 100

If the house is in an Urban location, the predicted price is $200,000 + $50,000 = $250,000.

Question 101

What does coding qualitative features allow you to do in regression analysis?

Answer 101

Coding allows you to include non-numeric categories in the regression model, helping you understand their influence on the outcome variable.

Question 102

In layman’s terms, how can you think of qualitative features in a regression model?

Answer 102

Think of qualitative features as different flavors; coding them lets you see how each “flavor” affects the overall “taste” (outcome) you are predicting!