week 5 & 6 Regression

Question 1

What is the primary difference between ANOVA and regression in terms of the types of studies they are used for?

Answer 1

ANOVA is for experimental studies while regression is for observational studies.

Question 2

What does regression fundamentally begin with?

Answer 2

Regression begins with correlation.

Question 3

What is the relationship between correlation and causation?

Answer 3

Correlation does not imply causation.

Question 4

What is the common measure of effect used in both correlation and regression?

Answer 4

The correlation coefficient (r²) which represents the proportion of variance explained.

Question 5

What can restriction of range on the independent variable lead to regarding the relationship?

Answer 5

It can underestimate the relationship.

Question 6

How can extreme cases or outliers affect linear models in regression?

Answer 6

Outliers can skew the correlation, inflating or deflating results.

Question 7

What is a potential issue with using poor or proxy measures in correlation?

Answer 7

It may underestimate the correlation.

Question 8

What does the strength of effect in regression correspond to in terms of r² values?

Answer 8

r² values indicate the proportion of variance explained, with higher values indicating stronger relationships

Question 9

A statistical method for predicting the value of one variable from another, using one or more predictors.

Answer 9

Regression

Question 10

A measure that quantifies the direction and strength of a linear relationship between two variables.

Answer 10

Correlation Coefficient (r)

Question 11

The portion of variability in a dependent variable that can be attributed to the independent variable(s) in a regression model.

Answer 11

Variance Explained

Question 12

What is the difference between simple regression and multiple regression?

Answer 12

Simple regression uses one predictor (independent variable), while multiple regression uses two or more predictors.

Question 13

A data point that differs significantly from other observations and can substantially affect the results of statistical analysis.

Answer 13

Outlier

Question 14

The variable that is manipulated or varied in an experiment or regression analysis to assess its impact on the dependent variable.

Answer 14

Independent Variable (IV)

Question 15

Why is it beneficial to use regression over simple correlation?

Answer 15

Regression allows for prediction of the outcome variable while accounting for multiple predictors, enhancing the understanding of variable relationships.

Question 16

What does a linear relationship express in the context of regression?

Answer 16

A linear relationship is expressed as a straight line.

Question 17

What is the relationship between a line and a model in regression?

Answer 17

Your line is your model.

Question 18

What are the two fundamental features that all lines possess in regression analysis?

Answer 18

All lines have a slope and an intercept.

Question 19

What does the term ‘error’ refer to in the context of regression?

Answer 19

Error refers to the difference between your modeled line and the actual data points.

Question 20

What does b1 represent in regression analysis?

Answer 20

b1 is the regression coefficient for the predictor and represents the gradient (slope) of the regression line, indicating the direction and strength of the relationship.

Question 21

What is represented by b0 in a regression equation?

Answer 21

b0 is the intercept, which is the value of Y when X = 0, marking the point where the regression line crosses the Y-axis.

Question 22

How can one estimate the outcome using multiple predictors in regression?

Answer 22

By entering the value of the predictor, multiplied by the coefficient, and adding the intercept, one can estimate the outcome.

Question 23

A statistical process for estimating the relationships among variables, allowing for the prediction of one variable based on the values of others.

Answer 23

Regression

Question 24

The slope of a regression line, represented by b1, indicates the direction and strength of the relationship between the independent and dependent variables.

Answer 24

Slope

Question 25

The intercept of a regression line, represented by b0, is the value of the dependent variable when all independent variables are set to zero.

Answer 25

Intercept

Question 26

The discrepancy between predicted values from the regression model and the actual observed values.

Answer 26

Error

Question 27

A variable that is used in a regression model to predict the outcome of another variable.

Answer 27

Predictor

Question 28

What is the method of least squares used for?

Answer 28

It is used to find the line of best fit for a set of data by minimizing the sum of the squares of the residuals.

Question 29

What is a residual in the context of least squares?

Answer 29

A residual is the difference between the observed data and the predicted values generated by the model.

Question 30

A straight line that best represents the data points in a scatter plot, minimizing the sum of the squares of the vertical distances of the points from the line.

Answer 30

Line of best fit

Question 31

A statistical method used to measure the total variability in a dataset, often decomposed into different components such as total variability, model variability, and residual variability.

Answer 31

Sums of Squares

Question 32

How do you assess the quality of a regression model?

Answer 32

By analyzing how well the model fits the observed data through metrics like Sums of Squares, ANOVA output, and Mean Squared Error

Question 33

Total Sum of Squares, representing total variability in the data, calculated as the variability between individual scores and the mean.

Answer 33

SST

Question 34

Sum of Squares for Residuals, indicating the variability between the actual data and the values predicted by the regression model.

Answer 34

SSR

Question 35

Sum of Squares for the Model, measuring the improvement in variability explained by fitting the regression model compared to the mean.

Answer 35

SSM

Question 36

What does a higher SSM compared to SSR indicate

Answer 36

It suggests that the model provides better predictions than simply using the mean of the observed data

Question 37

What does ANOVA stand for in the context of regression analysis?

Answer 37

ANOVA stands for Analysis of Variance, which tests the differences between the means of several groups and is used to evaluate the performance of the regression model.

Question 38

A metric that quantifies the average squared difference between the observed values and the values predicted by the model, reflecting the error of the model.

Answer 38

Mean Squared Error (MSE)

Question 39

What does the F-ratio represent in regression analysis?

Answer 39

The F-ratio is a statistic calculated to compare the mean of the sums of squares from the model to the mean of the sums of squares from the residuals, indicating whether the model effectively explains the variance.

Question 40

A measure that represents the proportion of variance accounted for by the regression model, indicating the strength of the relationship between the predictors and outcome variable.

Answer 40

r² (R-squared)

Question 41

How is r² similar to Pearson Correlation

Answer 41

r² is similar to squaring the r value obtained from Pearson Correlation, as it provides an understanding of the proportion of variance explained, but it can include multiple predictors in regression.

Question 42

The measure of how much the values in a dataset differ from the mean of that dataset, reflecting the spread of the data points.

Answer 42

Variance

Question 43

What role does the residual play in evaluating the fit of the regression model?

Answer 43

The residual indicates how well the model predicts the observed data; smaller residuals signify a better fit.

Question 44

A line that best describes the relationship between independent and dependent variables, derived from the least squares method.

Answer 44

Regression Line

Question 45

What factor determines whether a regression model is preferred over the mean?

Answer 45

If the model results in significantly lower error and better predictions compared to the mean, it is considered preferred.

Question 46

The process of using a regression model to estimate future values of the dependent variable based on new values of independent variables

Answer 46

Prediction

Question 47

What does a regression analysis output typically include?

Answer 47

It includes statistics such as R-squared, F-ratio, coefficients of the predictors, p-values, and sums of squares (SST, SSR, SSM) to evaluate model performance

Question 48

Write up the regression equation:

Answer 48

Yi = b0 + b1Xi
outcome = intercept + slope * predictor

Question 49

What is multiple regression?

Answer 49

Multiple regression is a model to predict the value of one variable from multiple predictors, extending simple regression to include several variables.

Question 50

How does multiple regression relate to simple regression?

Answer 50

Multiple regression is a natural extension of the simple regression model, which involves predicting a variable from just one predictor.

Question 51

What type of relationship does multiple regression model?

Answer 51

Multiple regression models the hypothetical relationship between several variables.

Question 52

What is the structure of a multiple regression equation?

Answer 52

The equation of multiple regression is similar to that of simple regression but includes additional predictors, forming a straight line

Question 53

In a regression equation, what does b0 represent?

Answer 53

In a regression equation, b0 represents the intercept, which is the value of the Y variable when all X variables are equal to zero.

Question 54

What role do regression coefficients play in a multiple regression equation?

Answer 54

Regression coefficients (bi) quantify the effect of each predictor (Xi) on the outcome variable (Y).

Question 55

Can a multiple regression model have more than one predictor?

Answer 55

Yes, a multiple regression model can include multiple predictors.

Question 56

What is the significance of the intercept in regression analysis?

Answer 56

The intercept indicates the expected value of the dependent variable when all independent variables are zero.

Question 57

General Linear Model

Answer 57

A statistical framework that encapsulates multiple regression, involving the relationship between a dependent variable and multiple independent variables.

Question 58

Multiple Regression

Answer 58

A statistical technique used to model the relationship between one dependent variable and two or more independent variables.

Question 59

A value that represents the effect of a unit change in a predictor variable on the dependent variable in a regression analysis.

Answer 59

Regression Coefficient

Question 60

The point at which the regression line crosses the Y-axis, indicating the expected value of the dependent variable when all predictors are zero.

Answer 60

Intercept

Question 61

What dimensionality does multiple regression introduce?

Answer 61

Multiple regression introduces additional dimensions as it includes multiple predictors, visualized as a regression plane.

Question 62

The outcome variable that a researcher aims to predict or explain in a regression analysis.

Answer 62

Dependent Variable

Question 63

A variable that is hypothesized to influence or predict the dependent variable in a regression analysis.

Answer 63

Independent Variable

Question 64

How is the regression plane defined in a multiple regression model?

Answer 64

The regression plane in a multiple regression model is defined by the intercept and regression coefficients of the predictors

Question 65

Predictors

Answer 65

Variables that are used in a regression analysis to predict the outcome of the dependent variable.

Question 66

What remains constant in the structure of multiple regression?

Answer 66

Despite the number of predictors, the model’s structure remains linear.

Question 67

A statistical method that models the relationship between two variables by fitting a linear equation to observed data

Answer 67

Linear Regression

Question 68

Why is the General Linear Model considered a foundational aspect of statistical analysis?

Answer 68

The General Linear Model provides a comprehensive framework for analyzing and understanding relationships between variables in various types of data.

Question 69

What are the three main methods of regression?

Answer 69

Hierarchical, Forced Entry, Stepwise.

Question 70

A regression method where all predictors are entered into the model simultaneously, relying on strong theoretical reasons for variable inclusion.

Answer 70

Forced Entry Regression

Question 71

What is the purpose of Hierarchical Regression?

Answer 71

Hierarchical Regression allows the experimenter to decide the order in which known predictors are entered, thereby assessing their unique contributions to the outcome.

Question 72

A data-driven approach where predictors are selected based on their semi-partial correlation with the outcome, using mathematical criteria rather than theory.

Answer 72

Stepwise Regression

Question 73

Why is Hierarchical Regression considered the best method?

Answer 73

It is based on theory testing and allows the unique predictive influence of new variables to be assessed while holding known predictors constant.

Question 74

Variables that can take on nominal values, such as different species in a study, which can be recoded for mathematical analysis in regression.

Answer 74

Categorical Predictors

Question 75

What is a major drawback of Stepwise Regression?

Answer 75

It may depend on slight numerical differences in semi-partial correlations, leading to significant theoretical implications.

Question 76

A statistical measure that indicates the unique contribution of a predictor to the outcome variable after controlling for other predictors.

Answer 76

Semi-partial Correlation

Question 77

How does a researcher determine the order of variables in Hierarchical Regression?

Answer 77

The researcher uses their theoretical understanding based on past research to decide which predictors to enter first.

Question 78

The portion of variance in the outcome variable that is accounted for by a specific predictor after considering other predictors in the model.

Answer 78

Unique Variance Explained

Question 79

What is the first step in Stepwise Regression when using SPSS?

Answer 79

SPSS looks for the predictor that can explain the most variance in the outcome variable.

Question 80

A variable that has two possible levels, such as having a characteristic versus not having it, which can be used in regression analysis.

Answer 80

Dichotomous Variable

Question 81

What is a critical consideration when using Forced Entry Regression?

Answer 81

The importance of having strong theoretical justifications for the inclusion of specific variables in the model.

Question 82

The process of using statistical methods, like Stepwise Regression, primarily to discover relationships in the data without strong theoretical backing.

Answer 82

Data Exploration

Question 83

What does it mean to recode categorical data for regression

Answer 83

It involves transforming nominal variables into numerical form that can be analyzed mathematically in regression analyses.

Question 84

What is the main goal of generalizing a sample model in multiple regression?

Answer 84

The main goal is to generalize the findings from the sample model to the entire target population.

Question 85

The process of applying findings from a sample model to the entire population, provided that certain assumptions are met.

Answer 85

Generalisation

Question 86

What type of outcome variable must be present in multiple regression analysis?

Answer 86

The outcome must be continuous.

Question 87

A condition where predictors in a regression model must display variance; otherwise, no estimation can occur.

Answer 87

Non-Zero Variance

Question 88

What is the assumption regarding the linearity in multiple regression?

Answer 88

The modeled relationship must be linear; it should not be curvilinear.

Question 89

The assumption that all values of the outcome variable should come from different individuals.

Answer 89

Independence

Question 90

What does multicollinearity refer to in multiple regression?

Answer 90

Multicollinearity exists when predictors are highly correlated with each other.

Question 91

The assumption that the variance of the error term should remain constant for each value of the predictors.

Answer 91

Homoscedasticity

Question 92

What does Cook’s Distance measure in the context of multiple regression?

Answer 92

Cook’s Distance measures the influence of a single case on the overall model.

Question 93

The residuals that have been transformed into Z-scores; typically, 95% should lie between ±2 in a normal distribution.

Answer 93

Standardised Residuals

Question 94

How can we identify potential outliers in standardized residuals?

Answer 94

Any case with an absolute standardized residual value of 3 or more is considered likely to be an outlier.

Question 95

Cases that have a disproportionate impact on the overall fit of the regression model, as measured by metrics like Cook’s Distance.

Answer 95

Influential Cases

Question 96

What should you check regarding the multivariate outliers in regression?

Answer 96

Look for extreme combinations of scores across multiple variables, not just single variable outliers.

Question 97

A measure used to identify multivariate outliers based on the distance of a case from the mean of a distribution, considering the covariance of the variables

Answer 97

Mahalanobis Distance

Question 98

What is the typical threshold for a case’s value in Cook’s Distance to determine influence?

Answer 98

A value greater than 1 indicates that a case is likely to be influential on the model.

Question 99

The differences between observed values and the values predicted by the regression model; they represent the error in prediction.

Answer 99

Residuals

Question 100

What does it mean if predictors exhibit high correlation in a regression model?

Answer 100

It indicates the presence of multicollinearity, which can obstruct the estimation of individual predictor effects.

Question 101

An assumption that the error terms in a regression analysis should be uncorrelated for any pair of observations.

Answer 101

Independent Errors

Question 102

Why is it essential to check for no multicollinearity in multiple regression?

Answer 102

High multicollinearity can inflate standard errors and make it difficult to assess the individual contribution of predictors