Chapter 1 Flashcards
What is a functional relation?
X-Independent variable and Y-Dependent variable then the functional relation is Y=F(X)
What is Regression Analysis?
Stat methodology that utilizes the relation between two or more quantitative variables so that a response/outcome variable can be predicted from the others.
What is a Statistical Relation?
Not exact like a functional relation. In example, book used scatter plot to show a linear trend…there is variation in the points on the graph. Relation could be curvilinear (not linear).
What is a regression model?
formal means of expressing the two essential ingredients of a statistical relation:
- A tendency of the response variable Y to vary with the predictor variable X in a systematic fashion
- A scattering of points around the curve of statistical relationship
2 characteristics embodied in a regression model by postulating that:
- There is a probability distribution of Y for each level of X
- The means of these probability distributions vary in some systematic fashion with X.
What is a major consideration in the selection of Predictor Variables during the construction of regression models?
The extent to which a chosen variables contributes to reducing the remaining variation in Y after allowance is made for the contributions of other predictor variables that have tentatively been included in the regression model.
List some other considerations in selecting Predictor Variables.
- The importance of the variable as a casual agent in the process under analysis
- The degree to which observations on the variable can be attained more accurately, or quickly, or economically than on competing variables
- The degree to which the variable can be controlled
How is the scope of a model determined?
Either by the design of the investigation or by the range of data at hand.
What are the 3 major purposes of Regression Analysis?
- Description
- Control
- Prediction
T/F: No matter how strong the statistical relation between X and Y, there is a cause-effect pattern implied by the regression model.
F: No matter how strong the statistical relation between X and Y, no cause-and effect pattern is necessarily implied by the regression model
Define the basic regression model:
Y_i = B_0 + B_1X_i + e_1
note: Y_i is the value of the response variable in the i-th trial, B_0 and B_1 are parameters (Beta), X_i is a known constant, namely, the value of the predictor variable in the i-th trial, e_i is a random error term with mean E[e_i]=0 and variance V[e_i]=(simga)^2
Why is this model: Y_i=B_0+B_1X_i+e_i called a first order model?
The regression model is said to be simple, linear in the parameters, and linear in the predictor variable. Simple because there is only one predictor variable, linear because no parameter is squared, multiplied or divided by another parameter.
E[Y_i] where Y_i = simple regression model
E[Y_i] = E[B_0+B_1X_i+e_i] = B_0 + B_1X_i + E[e_i] = B_0 + B_1X_i
because E[e_i]=0
V[Y_i] =
(sigma)^2
because the error term e_i has constant variance, (sigma)^2
T/F: Since the error terms, say e_i and e_j, in a regression model are assumed to be uncorrelated then Y_i and Y_j (any two responses) are also uncorrelated.
T
What are the regression coefficients in a simple regression model?
B_0, B_1 are the parameters
What is B_1 in the simple regression model?
Slope of the regression line. It is the change in the mean of the probability distribution of Y per unit increase in X.
What is the parameter B_0 in the simple regression model?
Y intercept of the regression line.
What happens when the scope of the simple regression model includes X = 0?
B_0 gives the mean of the probability distribution of Y at X = 0.
What does B_0 mean when the scope of the model does not cover X = 0?
B_0 does not have any particular meaning as a separate term in the regression model.
How is observational data obtain?
Observational data is obtained from non-experimental studies. Theses studies do not control the explanatory or the predictor variable(s) of interest.
What is one major limitation of observational data?
They often do not provide adequate information about cause and effect relationships.
When control over the explanatory variable(s) is exercised through random assignments the resulting experimental data provide much stronger information about cause and effect relationships than observational data. Why?
The reason is that randomization tends to balance out of any other variables that might effect the response variable.
T/F: Control over the explanatory variable(s) consists of assigning a treatment to each of the experimental units by means of randomization.
T
Describe Completely Randomized Design.
Makes randomized assignments of treatments to experimental units (or vice versa). Useful when experimental units are homogeneous. The design is flexible because it accommodates any number of treatments and permits different sample sizes for different treatments.
What is the Method of Least Squares?
A way to find “good” estimators of the regression parameters B_0 and B_1.
How do you use the Method of Least Squares?
For the observations (X_i,Y_i) for each case, the MLS considers the deviation of Y_i from its expected value: Y_i - (B_o +B_1X_i).
The MLS requires that we consider the sum of the n squared deviations: Q = (sum from i=1 to n) (Y_i - B_0 - B_1X_i)^2
What is another way to describe the Method of Least Squares?
The estimators of B_0 and B_1 are those b_0 and b_1 that minimize the criterion Q for the given sample observations (X_1,Y_1), …,()X_n,Y_n)
What are the point estimators of B_0 and B_1?
b_0 and b_1 (page 17 of applied linear models)
What is the Gauss-Markov Theorem?
Under the conditions of the simple regression model, the least squares estimators b_0 and b_1 are unbiased and have a minimum variance among all unbiased linear estimators (b_0 and b_1 sampling distributions are less variable than any other estimators).
E[b_0]=B_0 and E[b_1]=B_1
What is the estimate of the regression function?
Y(hat) = b_0 + b_1X
Y hat is the value of the estimated regression function at the level X of the predictor variable.
What is the difference between the response and the mean response?
We call a value of the response variable a response and E[Y] the mean response. Mean response is the mean of the probability distribution of Y corresponding to the level X of the predictor variable. Y hat is a point estimator of the mean response when the level of the predictor variable is X. Y hat is an unbiased estimator of E[Y], with minimum variance in the class of unbiased estimators (by Markov Theorem).
Define residual.
The i-th residual is the difference between the observed value Y_i and the corresponding fitted value Y_i hat. e_i = Y_i - Y_i hat
e_i = Y_i - (b_0 +b_1X_i)
The magnitude of a residual is represented by the vertical deviation of the Y_i observation from the corresponding point on the estimated regression function.
Distinguish between the model error term value epsilon = Y_i - E[Y_i] and the residual e_i = Y_i - Y_i hat.
The model error term is the vertical deviation from the unknown true regression line and hence is unknown. The residual is the vertical deviation of Y_i from the fitted value Y_i hat on the estimated line and is known. Residuals are highly useful for studying whether a given regression model is appropriate for the data at hand.
What is the sum of the residuals?
0
What is the sum of the squared residuals?
a minimum
What does the sum of the observed values Y_i equal?
The sum of the fitted values, Y_i hat.
What is the sum of the weighted residuals when the residual in the i-th trial is weighted by the level of the predictor variable in the i-th trial.
Sum from i=1 to n of X_ie_i = 0
What point does the regression line always go through?
(X(bar), Y(bar))
T/F: The sum of the weighted residuals is 1 when the residual in the i-th trial is weighted by the fitted value of the response variable for the ith trial.
F: Sum from i=1 to n of Yhat_i e_i =0
Why does the variance, (sigma)^2, of the error terms in a simple regression model need to be estimated?
To obtain an indication of the variability of the probability distributions of Y.
What is the variance (sigma)^2 of a single population estimated by?
s^2, the sample variance. The sample variance is also known as the mean square.
T/F: s^2, the sample variance, is an unbiased estimator of the the variance (sigma)^2 of an infinite population.
True
What is E[MSE], where MSE is the error mean square/residual mean square
(sigma)^2 because MSE is an unbiased estimator of (sigma)^2 for the regression model.
Describe the Method of Maximum likelihood.
This method chooses as estimates those values of the parameters that are most consistent with the sample data. Way to obtain the estimators of the parameters B_0, B_1, and (sigma)^2. The method of maximum likelihood chooses the maximum likelihood estimate that value of mu for which the likelihood value is largest.
