W5: RQ for Predictions 2

Question 1

What is a full regression equation involving 2 IVs

Answer 1

Yi = a + b₁X_1i + b₂X_2i + e_i

• b₁ and b₂
  
  • _​partial regerssion coefficients
• a
  
  • intercept

Question 2

What is a model regression equation involving 2 IVs

Answer 2

Y^i = a + b₁X_1i + b₂X_2i

Question 3

What is an intercept in a regression equation. What is it signified by

Answer 3

• a
• Expected score on DV when all IVs = 0

Question 4

What is a partial regression coefficient in a regression equation. What is it signified by

Answer 4

• b1, b2,…,
• Expected change in DV for each unit change in IV, holding constant scores on all other IV

Question 5

What is the Sum of Squares in a regression equation. What is SS Total Indicated by

Answer 5

SS_total = SS_reg + SS_res

• OLS gurantees SS_reg will be as large as possible because it ensures SS_res is as small as possible!
  
  • Unbiased in this regard

Question 6

What is the aim of OLS. Use sum of squares to explain

Answer 6

Maximize SS_reg; Minimize SS_res

Question 7

What is R^2. Alterative name. Is it an effect size?

Answer 7

• Coefficient of Determination
  
  • Effect size
• Strength of prediction in a regression analysis
• R^2 = SS_reg/SS_total
  
  • _​​Proportion of SS_total accounted for by SS_reg
  • Proportion of variance in DV that is predictable from IVs

Question 8

What is the range of R^2

Answer 8

0 to 1.

Closer to 1 = Stronger = Greater proportion of DV explained by regression model (IVs) = SS_reg great

Question 9

To calculate a confidence interval on R, what are the 4 things we require

Answer 9

• Estimated R² value
• Numerator df on F statistic (no. of IVs)
• Denominator df on F statistic (n-IVs-1)
• Desired confidence level
  
  • Smaller width = Greater precision

Question 10

Under which conditions will the observed R-squared value be most biased (all other aspects being equal)?

Answer 10

When sample size is small and the number of IVs is large

Question 11

Typically what is the biasness/consistency of R^2

Answer 11

OLS estimate of R² is biased (often overestimated), but consistent (if sample size increase, it will get increasingly closer to 95%)

• Note
  
  • OLS estimate of slope is unbiased, but it is bias for R²

Question 12

How is an adjusted R² better than R²

Answer 12

It is usually less biased, but we should always report both values

Question 13

What are the two ways we can make meaningful comparison between IVs in a multiple regression

Answer 13

• Transform regression coefficient to standardised partial regression coefficient
  
  • Z-scores
• Semi-Partial and Squared Semi-Partial Correlations

Question 14

Making meaningful comparison between IVs in a multiple regression: Method 1. When interpreting coefficients, what is the difference. When is this method useful?

Answer 14

• Coefficients are interpreted in SD units
  
  • Example
    
    • One SD increase in variable X will increase in 0.5 SD decrease in variable Y, holding constant scores on all other IV
• No (intercept)
• Only useful when IV has an arbitrary scaling

Question 15

Making meaningful comparison between IVs in a multiple regression: Method 1. What is the intercept

Answer 15

Always 0

Question 16

Making meaningful comparison between IVs in a multiple regression: Semi-Partial and Squared semi-partial Correlations. Overview.

Answer 16

• Semi-partial correlatons
  
  • Correlation between DV and each focal IV, when effects of other IVs have been removed from that focal IV
  • Correlation between observed (not predicted) DV and scores on focal IV that is not accounted for by all other IV in the regression analysis
  • Effect size estimate
• Squared semi-partial correlation
  
  • Proportion of variation in observed (not predicted) DV uniquely explained by each IV
  • Directly analgous to R² of the overall model

Question 17

Making meaningful comparison between IVs in a multiple regression: What is the SQUARED semipartial correlation? What is the analogous to?

Answer 17

• Indicates proportion of variation in DV UNIQUELY explained by each IV.
• Directly analogous to R^2 in regression model but telling about each IV on its own.

Question 18

What are the 4 statistical assumptions underlying the linear regression model

Answer 18

• Independence of Observations
• Linearity
• Constant variance of residuals/ homoscedastiscity
  
  • (Not homogenity)
• Normality of Residual Scores

Question 19

Statistical Assumption Linear Regression: Independence of Observations. How do we meet this

Answer 19

• Scores are not duplicated for bigger sample
• Responses on one variable does not determine person’s response to another variable

Question 20

Statistical Assumption Linear Regression: Linearity.

(1) What is it (2) How do we meet this

Answer 20

• Scores on the dependent variable are an additive
  linear function of scores on the set of independent variables

1. Scatterplot Matrix
2. Residual Plot/ Scatterplot of Residual Scores
3. Marginal Model Plots
4. Marginal Conditional Plots

Question 21

Statistical Assumption Linear Regression: Linearity. 1. Scatterplot Matrix

Answer 21

• Y-Axis
  
  • DV
• X-Axis
  
  • IVs
• Look for U-Shapes/Inverted U-shapes

Question 22

Statistical Assumption Linear Regression: Linearity. 2. Scatterplot of Residual Scores

Answer 22

• Y-Axis
  
  • Residual Scores
• X-Axis
  
  • Observed IV Scores
  • Predicted DV Scores

Question 23

Statistical Assumption Linear Regression: Linearity. Marignal Model Plot

Answer 23

• Y-Axis
  
  • Observed DV Scores
• X-Axis
  
  • Observed IV Scores
  • Predicted DV Scores

Question 24

Statistical Assumption Linear Regression: Marginal Conditional Plot. Why is it especially good

Answer 24

• Y-Axis
  
  • Predicted DV scores
• X-Axis
  
  • Observed IV Scores

_Conditional regression slop_e shows partial regression line after other IVs are partialled out with each other and the DV

Question 25

Statistical Assumption Linear Regression: Homoscedasticity, What is it. What are the 2 ways

Answer 25

Constant residual variance for different predicted values of the dependent variable.

1) Residual Plots (Similar to linearity)
2. ) Breusch-Pagan Test (nCV)

Question 26

Statistical Assumption Linear Regression: Homoscedasticity, Residual Plots

Answer 26

• Y-Axis
  
  • Residual Scores
• X-Axis
  
  • Observed IV Scores
  • Predicted DV Scores
• Examine fanning out

Question 27

Statistical Assumption Linear Regression: Homoscedasticity, Breusch-Pagan Test Overview

Answer 27

• Null-Hypothesis Test that assumes the variance of residuals are constant/homoscedastic
  
  • Individual IVs
  • IV together
  • Regresion model
• Evidence from the residual plots and from the ncvTest
  results are consistent, but it may not be the case. Report both if so.

Question 28

Statistical Assumption Linear Regression: Homoscedasticity, Breusch-Pagan Test. What happens when the P value is small

Answer 28

We have reason to reject the assumption of constant variance for the residuals

Question 29

Statistical Assumption Linear Regression: Normality of Residuals. What are the 3 ways

Answer 29

1. qqplot
2. Histogram
3. boxplot

Question 30

Statistical Assumption Linear Regression: Normality of Residuals. qqplot. What does it contain

Answer 30

• Middle line: Strict normality
• Side Lines: Confidence envelope
  
  • See residuals inside confidence

Question 31

Statistical Assumption Linear Regression: Normality of Residuals. What are studentized residuals.

Define Outliers and Influential Cases.

How are they checked?

Answer 31

Stundentized Residuals = Standardized form of Residuals Value, where SD is one

• Outliers (Measured by studentized value)
  
  • Vary large studentized residual value in absolute terms
    
    • Usually about 3
• Influential Case (Measured by Cook’s D)
  
  • If removed from analysis, results in regression coefficient changing notably in value
• Both are checked by influence index plot, which labels largest Cook’s D and largest absolute studentized reisdual value
  
  • Note: Outliers IS NOT NECESSARILY infleuntial

Question 32

Statistical Assumption Linear Regression: Normality of Residuals. How do we find influential cases

Answer 32

Cook’s D

• Measure of overly influential values
  
  • Range: >0
  • Problem: >1

Question 33

Statistical Assumption Linear Regression: Normality of Residuals. How do we find outliers

Answer 33

• Vary large studentized residual value in absolute terms
• Usually about 3

Question 34

What does “Over the Long Run” mean in CI

Answer 34

Repeated sampling of a population and 95% of these CIs will capture the true population parameter value.

Question 35

What indicates the precision of R^2

Answer 35

Width of the interval between lower and upper bound. (Smaller width = More precise)

Question 36

What are the factors in which observed R^2 will be much larger than true population value

Answer 36

1.) Small samples 2.) Many IVs

Question 37

What is the size of each partial regression coefficient determined by

Answer 37

Scaling/metric of each IV (If scaling differ = cannot use relative size of b values to say which is a stronger predictor)

Question 38

Statistical Assumption Linear Regression: Normality of Residuals. How do we find outliers

Answer 38

Studentized residuals (absolute value 3 problematic)

Question 39

What is a studentized residual

Answer 39

A particular form of standardized residual