Regression

Question 1

what is regression analysis used for in statistics?

Answer 1

regression analysis is used to explore relationships between variables, allowing for the prediction of one variable based on another

Question 2

what is a residual in regression analysis?

Answer 2

a residual is the difference between the observed value and the predicted value for a given data point

Question 3

what does the line of best fit represent in regression analysis?

Answer 3

the line of best fit represents the linear relationship between the dependent and independent variables, minimising the sum of squared residuals

Question 4

what is the method of least squares?

Answer 4

the method of least squares is a technique used in regression the find the line that minimises the sum of the squared residuals

Question 5

how can you interpret the regression equation y = b0 + b1x?

Answer 5

in the equation y = b0 + b1x, bo is the y-intercept, representing the value of y when x = 0, and b1 is the slope, indicating how much y changes for a one-unit increase in x.

Question 6

what does a negative residual indicate?

Answer 6

a negative residual indicates that the predicted value is higher that the observed value for that data point

Question 7

what is the significance of the sum of squared residuals in regression analysis?

Answer 7

the sum of squared residuals is minimised in the method of least squares to find the best-fitting line, ensuring the model’s predictions are as close as possible to the observed data

Question 8

what is extrapolation, and why is it dangerous in regression analysis?

Answer 8

extrapolation involves using a regression line to predict y values for x values outside the observed data range. it is risky because the trend might change, leading to poor predictions

Question 9

what are forecasts, and what assumption is made when using a regression line to predict future values?

Answer 9

forecasts are predictions about the future using time series data. the assumption made is that the past trend will remain the same in the future, which can be risky

Question 10

what is an influential outlier in regression analysis, and how does it affect results?

Answer 10

an influential outlier is a data point that significantly impacts the regression line and correlation, especially when the point is both far from the trend and has an extreme x value

Question 11

what is the difference between an outlier and a regression outlier?

Answer 11

an outlier is a point far from others in terms of x and y values, but a regression outlier is a point that is far from the overall trend, even if not an outlier on its own x or y values

Question 12

what should you do when you encounter an influential regression outlier?

Answer 12

investigate the observation to see if it was recorded incorrectly or if it is genuinely different. it may be useful to refit the regression line without the outlier to check its impact on results

Question 13

does correlation imply causation? why or why not?

Answer 13

no, correlation does not imply causation. an association between two variables may be due to a third variable, or there could be other explanations for the observed relationship.

Question 14

what is a lurking variable?

Answer 14

a lurking variable is an unobserved variable that influences the association between the response and explanatory variables

Question 15

what is simpson’s paradox?

Answer 15

simpson’s paradox occurs when the direction of an association between two variables changes after including a third variable and analysing data at separate levels of that variable

Question 16

how can lurking variables affect the interpretation of correlations?

Answer 16

lurking variables can create spurious associations or distort the apparent relationship between two variables, making it seem as if one causes the other

Question 17

what is confounding in statistics?

Answer 17

confiding occurs when two explanatory variables are associated with both the response variable and each other, making it difficult to determine which variable is causing the observed effect

Question 18

what is the difference between a lurking variable and a confounding variable?

Answer 18

a lurking variable is unmeasured and affects the relationship between the explanatory and response variables.

Question 19

how can confounding affect the interpretation of a study’s results?

Answer 19

confounding can distort the apparent association between variables, making it seem as though one causes the other when, in fact, a third variable is influencing the results.

Question 20

what role do statistical methods play in analysing confounding variables?

Answer 20

statistical methods can adjust for confounding variables, isolating the effect of the explanatory variable, but there’s always a risk of omitting important confounders

Question 21

what is the response variable inn regression analysis?

Answer 21

the response variable (y) is the variable you want to predict

Question 22

what is the explanatory variable in regression analysis?

Answer 22

the explanatory variable (x) is the predictor or independent variable used to predict the response variable

Question 23

what is the purpose of a scatterplot in regression analysis?

Answer 23

a scatterplot is used to visualise the relationship between two variables and to check if the relationship is linear

Question 24

what is the equation of the regression line?

Answer 24

the regression line equation is yn = a + bx, where a is the y-intercept and b is the slope

Question 25

what are residuals in regression analysis?

Answer 25

residuals are the differences between observed values and predicted values: residual = y - yn

Question 26

what does the regression model describe in terms of the mean of y?

Answer 26

the regression model describes how the mean of y changes with x, represented by µy = alpha + beta * x

Question 27

what is the significance of the straight-line assumption in regression?

Answer 27

the straight.line assumption implies that the relationship between x and y is linear, but this is an approximation and may not always hold in practice

Question 28

what is the conditional distribution in regression analysis?

Answer 28

the conditional distribution is the distribution of y values at each fixed value of x, and it describes the variability of y around the regression line

Question 29

how is the variability in y values represented in regression analysis?

Answer 29

variability is represented by the standard deviation sigma of the conditional distribution of y for each fixed x value

Question 30

what happens if the true relationship between variables is nonlinear?

Answer 30

if the relationship is nonlinear (e.g., U-shaped), using a straight-line regression model could lead to inaccurate predictions

Question 31

what does the correlation r describe in the straight-line regression model?

Answer 31

the correlation r describes the strength and direction of the linear association between two variables

Question 32

what is the range of values for the correlation r?

Answer 32

the correlation r always falls between -1 and +1, i.e., -1 ≤ r ≤ 1

Question 33

how is the strength of the linear association related to the value of r?

Answer 33

the larger the absolute value of r, the stronger the linear association. when r = 1, the data points fall exactly on the regression line

Question 34

what is the relationship between the correlation r and the slope b of the regression line?

Answer 34

the correlation r has the same sign as the slope b. a positive correlation corresponds to an upward trend, and a negative correlation corresponds to a downward trend

Question 35

how does the correlation r relate to the slope b and standard deviations sx and sy?

Answer 35

the slope b is proportional to the correlation r and the ratio of the standard deviations: b = r * (sy/sx)

Question 36

why can’t the slope alone describe the strength of an association?

Answer 36

the slope’s numerical value depends on the units of measurement, while the correlation r is a standardised version that does not depend on the units

Question 37

what is “regression toward the mean”?

Answer 37

regression toward the mean refers to the tendency for extreme values of one variable (e.g., very tall parents) to be associated with values closer to the mean for the other variable (e.g., shorter children)

Question 38

how does the correlation r affect the predicted value of y in regression?

Answer 38

the predicted value of y is relative closer to its mean than x is to its mean. if x is one standard deviation above its mean, the predicted y will be r times that distance from its mean

Question 39

what does r^2 represent in regression analysis?

Answer 39

r^2 represents the proportion of the variance in y explained by x. it is the proportional reduction in error compared to predicting y by its mean

Question 40

what is the interpretation of r^2 = 0,4?

Answer 40

if r^2 = 0,4, this means that the error using the regression equation to predict y is 40% smaller than the error when predicting y using its mean

Question 41

what is the advantage of using the correlation r over r^2?

Answer 41

the correlation r is easier to interpret because it is in the original scale of measurement, while r^2 involves squared units

Question 42

how can outliers affect the correlation?

Answer 42

outliers with unusually large or small x values that fall far from the trend can have a significant impact on the correlation and slope

Question 43

how does grouping data for observations affect the correlation?

Answer 43

grouping data, such as using summary statistics for countries rather than individual data, tends to increase the magnitude of the correlation

Question 44

what is the ecological fallacy?

Answer 44

the ecological fallacy occurs when results from grouped data (e.g., county averages) are incorrectly generalised to individual data

Question 45

how does the range of x values affect the correlation?

Answer 45

the correlation tends to be smaller when only a restricted range of x values is sampled, as it may not capture the full variability of the data

Question 46

what are the key assumptions needed for regression analysis to make inferences?

Answer 46

1. randomness: the sample must be representative of the population
2. normality: the residual should approximately follow a normal distribution
3. linearity: the relationship between the variables should be linear

Question 47

what does the null hypothesis (H0 : beta = 0) in a regression significance test represent?

Answer 47

it tests whether there is no linear relationship between the variables (i.e., they are independent)

Question 48

what is the formula for the t-statistic in a regression test of significance?

Answer 48

t = ((b-0)/se)
where beta is the sample slope and se is the standard error of the slope

Question 49

what does a small p-value in a regression test suggest?

Answer 49

a small p-value indicates strong evidence against the null hypothesis, suggesting that there is a significant linear relationship between the variables

Question 50

what does a 95% confidence interval for the slope tell us?

Answer 50

it gives the range of plausible value for the true population slope. for example, a 95% confidence interval of (1.2;1.8) means we are 95% confident that the true slope falls within this range

Question 51

how is the sample correlation r related to the sample slope beta?

Answer 51

r = beta * (sx/sy)
where sx and sy are the standard deviations of x and y, respectively

Question 52

what does it mean if the slope beta = 0 in a regression analysis?

Answer 52

if beta = 0, then r = 0, indicating no linear relationship between the variables

Question 53

what is the purpose of testing the population correlation using the t-statistic for the slope?

Answer 53

the t-statistic for testing whether the slope is zero is equivalent to testing whether the population correlation is zero. a small p-value indicates a significant correlation