Statistical Models

Question 1

Define parameters in regression equations (slope, Y-intercept)

Answer 1

The intercept and weight (measurement error) values are called the parameters of the model.

Question 2

What are the weight values?

Answer 2

Regression weights, or regression coefficients. Weight values are values that assume for the amount of error/random error in the model. They are calculated using the slope of the line, where the dependent Y [reliant] has a quantitative relationship with the independent X [controlled]. Y is assumed to have a constant standard deviation over multiple observations, which can then minimize error by calculating least squared estimates [or taking correlation by SD of Y compared to SD of X].

Question 3

What is the intercept?

Answer 3

Used to account for error when the mean of the residuals [Y] is theoretically equal to zero [X=0]. It tells you nothing about the relationship b/w X and Y, and serves as a constant that gives some idea of where Y crosses the y-axis.

Question 4

What do these two parameters estimate?

Answer 4

The slope and the intercept define the linear relationship between two variables, and can be used to estimate an average rate of change.

Question 5

What are predicted values?

Answer 5

The values of Y-hat, or the values of the dependent variable assumed by the parameters of the model.

Question 6

What are observed values?

Answer 6

The values of Y, or the actual values of the dependent variable.

Question 7

How should we understand model fit?

Answer 7

Model fit is a measurement/representation of how well the actual values of Y correspond to the predicted values of Y.

Question 8

What specific equation is used to assess/estimate model fit?

Answer 8

Error variance, or the average squared deviations of the model parameters. This is used because the degree of error in the model tells us how far the predicted values distance from reality.

Question 9

What is good error variance?

Answer 9

Small, minor differences in variability that do not detract or misrepresent the relationship b/w Y and Y-hat. Good error variance is not necessarily explainable, but also does not impact how accurate the model is assumed to be or can be corrected for.

Question 10

What is bad error variance?

Answer 10

Differences in variability that are not explainable and that confound the relationship b/w Y and Y-hat. In this case, the error that we are attempting to examine can be related to entirely different variables, or are not actually relevant to the question that the model is attempting to represent.

Question 11

What does least squares estimation do?

Answer 11

Finds a line determined by using an estimation of squared SD error variance relative to the constant Y [if the mean of X=0]. It makes the sum of squared errors as small as possible, and thus minimizes the total compounded error.

Question 12

What is a residual?

Answer 12

Error estimates that result from the model being unable to perfectly reconstruct the actual data b/c it cannot realistically represent the full population error.

Question 13

What is error variance?

Answer 13

The average squared differences b/w the predict and the observed values.

Question 14

What is R^2?

Answer 14

A squared correlation that represents the proportion of the variance in Y that is accounted for by the model. It estimates the strength of the relationship b/w the model and the response variable.

Question 15

Explain the purpose of statistical modeling.

Answer 15

To provide a mathematical representation of theories [about the relationships b/w different factors].

Question 16

Explain how changing different parts of a linear equation will alter the model.

Answer 16

The model changes depending on which variable is different in the equation. This change is unique if it is a linear or quadratic model.

Linear:

a-value: Y values increase as we move up. Elevation of the line changes.

b-value: the line becomes steeper or less steep.

Quadratic:

a-value: elevation of the curve increases.

b-value: the depth of the curve increases.

Question 17

What if the slope is equal to zero?

Answer 17

The predicted values of Y are equal to a.

Question 18

Explain the goals of parameter estimation.

Answer 18

To find weight and intercept values that allow us to calculate the smallest discrepancy between the predicted Y values and the actual values of Y.

Question 19

Explain the least-squares estimation.

Answer 19

Finds a line determined by using an estimation of squared SD error variance relative to the constant Y [if the mean of X=0]. It makes the sum of squared errors as small as possible, and thus minimizes the total compounded error.

Question 20

Explain 3 sources of error variance.

Answer 20

Sampling error - error caused by observing a sample instead of the whole population.

Incomplete Model/Missing Variables - variables that mattered, but were not identified or applied to the model.

Imprecision in measurement - nonsystematic problems in how data was collected, how subjects interpreted the questions of interest, or w/ the process of experimentation. Random error has a normal distribution; high and low scores are roughly symmetrical.

Question 21

Explain what the best guess of the y-intercept would be if we have no information about x-values.

Answer 21

The mean of Y, because it is the point at which the sum of the deviations above that point are balanced by the sum of the deviations below that point. (e.g. a least squares statistic) The mean is the only value that makes the variance as small as possible. This is equivalent to ignoring any X serving as a predictor variable in the model.

Question 22

Explain the goal of model comparison.

Answer 22

To approximate which of multiple theories gives a better account of the data and better represents reality.

Question 23

Explain why R-squared is useful.

Answer 23

R^2 is useful b/c it is a standard metric for interpreting model fit. It evaluates the portion of the variance of Y accounted for by the model relative to the actual variance in Y.

Question 24

Identify parameters in a model, given an equation.

Answer 24

Linear: Y = a + bX | Quadratic: Y = a + bX^2

a = Y-intercept (the value of Y when X = 0)
b = slope (the-rise-over-the-run, the steepness of the line); a weight

Question 25

Identify independent and dependent terms in a model, and identify the assumptions we are making about causal modeling.

Answer 25

The dependent Y [reliant] has a quantitative relationship with the independent X [controlled]. We are assuming that the predictor X-value has a relationship to the unknown Y, and that there is a causal interaction [i.e. X impacts the possibility of Y occurring].

Question 26

Compare predicted vs. observed values.

Answer 26

1. Predicted - Y-hat. A value of Y assumed by the parameters used to estimate the model; in general, an estimation of what the relationship b/w Y and Y-hat may resemble.

2. Observed - actual Y. How the relationship w/ the Y value plays out in reality as opposed to the fictitious model.

Question 27

Compare “good” vs. “bad” model fit (perfect vs. worst)?

Answer 27

Good/Perfect - an R^2 of 1, w/ the model accounting for all the error variance in Y.

Bad - an R^2 of 0, w/ the model accounting for error variance as if no predicting variables were involved in the relationship of X to Y. Effectively, error variance is the same as the variance in Y.

Question 28

Compare error variance vs. R-squared.

Answer 28

Error variance: the average squared error. Average squared difference b/w the predicted values and the actual values of Y.

R^2: the average squared correlation of one or more coefficients. Average squared correlation of the proportion of variance in Y accounted for by the model vs. the actual variance in Y.

Question 29

What are two questions to ask when attempting to model an accurate estimation of a theory?

Answer 29

1. Is a linear or a quadratic model more appropriate to capture the data we want to describe
2. Are the parameters correct and do they accurately represent the data we want to describe

Question 30

What is the value of error variance when the model is perfect?

Answer 30

0.

Question 31

What is the value of error variance when a model is performing as badly as possible?

Answer 31

The mean of the sample or population.

Question 32

Why is the mean the worst a model can do?

Answer 32

Because it is a least squares statistic, it represents the point where the sum of the deviations above are balanced by the sum of the deviations below (i.e. made to equal 0). This makes the variance as small as possible, which is the purpose of least squares statistics.

In other words: our model is predicting no better than our “best” guess might be able to.

Question 33

What does it mean when our model can only predict the mean value?

Answer 33

Our model is getting an error variance that is as large as the actual variance of Y.

Question 34

What are 3 advantages of using R squared?

Answer 34

1. It doesn’t matter how large the variance of Y is because everything is evaluated relative to the variance of Y
2. Set end-points: 1 is perfect and 0 is as bad as a model can be
3. It is standardized

Question 35

What do theories typically specify?

Answer 35

The form of the relationship b/w variables.