Lecture 8 (MULTIVARIATE REGRESSION ANALYSIS)

Question 1

MULTIPLE REGRESSION ANALYSIS

Answer 1

Regression analysis with two or more independent variables or with at least one nonlinear predictor.

Question 2

PROBABILISTIC MULTIPLE REGRESSION MODEL

Answer 2

y = β0 + β1x1 + β2x2 + β3x3 + … + βkxk + ͼ

y = the value of the dependent (response variable)
β0 = the regression constant
β1 = the partial regression coefficient of independent variable 
β2 = the partial regression coefficient of independent variable 2.
βk = the partial regression coefficient of independent variable k 
k = the number of independent values
ͼ  = the error of prediction

Question 3

What is the dependent variable y sometimes referred to as?

Answer 3

The response variable

Question 4

What does the partial regression coefficient of an independent variable, βi represent?

Answer 4

Represents the increase that will occur in the value of y from a one unit increase in that independent variable if all other variables are held constant.

Question 5

ESTIMATED REGRESSION MODEL

Answer 5

y-hat = b0 + b1x1 + b2x2 + b3x3 + … + bkxk
where:
y-hat = predicted value of y
b0 = estimate of regression constant
b1 = estimate of regression coefficient 1
b2 = estimate of regression coefficient 2
b3 = estimate of regression coefficient 3
bk = estimate of regression coefficient k
k = number of independent variables

Question 6

MULTIPLE REGRESSION MODEL WITH TWO INDEPENDENT VARIABLES (FIRST ORDER)

Answer 6

The simplest multiple regression model is one constructed with two independent variables, where the highest power of either variable is 1 (first order regression model).

In multiple regression analysis, the resulting model produces a response surface.
POPULATION MODEL:
y = β0 + β1x1 + β2x2 + ͼ
ESTIMATED MODEL:
y-hat = b0 + b1x1 +b2x2

Question 7

RESPONSE PLANE FOR FIRST ORDER TWO PREDICTOR MULTIPLE REGRESSION MODEL

Answer 7

In multiple regression analysis, the resulting model produces a response surface.
In the multiple regression model shown here with two independent first order variables, the response surface is a response plane.
Fit into a 3D space (x1,x2,y)

Question 8

DETERMINING THE MULTIPLE REGRESSION EQUATION

Answer 8

The simple regression equations for determining the sample slope and intercept given in earlier material are the result of using methods of calculus to minimise the sum of error for the regression model.
The formulas are established to meet an objective of minimising the sum of squares of error for the model.
The regression analysis shown here is referred to as least squares analysis. Methods of calculus are applied, resulting in k+1 equations with k+1 unknowns for multiple regression analyses with k independent variables.

Question 9

LEAST SQUARES EQUATIONS FOR k = 2

Answer 9

b0n + b1Σx1 + b2 Σx2 = Σy
b0Σx1 + b1Σx1^2 + b2Σx1x2 = Σx1y
b0Σx2 + b1Σx1x2 + b2Σx2^2 = Σx2y

Question 10

TESTING THE OVERALL MULTIPLE REGRESSION MODEL

Answer 10

H0 : β1 = β2 = β3 = … = βk = 0

Ha : At least one of the regression coefficients is =/ 0

Question 11

SIGNIFICANCE TESTS FOR INDIVIDUAL REGRESSION COEFFICIENTS

Answer 11

H0 : β1 = 0
Ha : β1 =/ 0
H0 : β2 = 0
Ha : β2 =/ 0
H0 : β3 = 0
Ha : β3 =/ 0
H0 : βk = 0 
Ha : βk =/ 0

Question 12

What does it mean if you fail to reject the null hypothesis of a regression model?

Answer 12

Stating that the regression model has no significant predictability for the dependent variable.
A rejection of the null hypothesis indicates that at least one of the independent variables is adding significant predictability for y.

Question 13

F-VALUE

Answer 13

MSR (MS - Regression) / MSE (MS - Residual)

Question 14

How are residuals calculated for in multiple regression analysis?

Answer 14

First, a predicted value, y-hat, is determined by entering the value for each independent variable for a given set of observations into multiple regression equation and solving for y-hat.

Question 15

What are residuals useful for?

Answer 15

Helpful in locating outliers.

Question 16

OUTLIERS

Answer 16

Data points that are apart, or far, from the mainstream of the other data.
They are sometimes data points that were mistakenly recorded or measured.
Because every data point influence the regression model, outliers can exert an overly important influence on the model based on their distance from other points.

Question 17

What is Se in multiple regression analysis?

Answer 17

Se = sqrt (SSE/n-k-1)
n = number of observations
k = number of independent variables

Question 18

COEFFICIENT OF MULTIPLE DETERMINATION (R^2)

Answer 18

Analagous to the coefficient of determination (r^2)
R^2 represents the proportion of variation of the dependent variable, y, accounted for by the independent variables in the regression model.
Range between 0 and 1
R^2 = SSR/SSyy
R^2 = 1 - SSE/SSY

Question 19

ADJUSTED R^2

Answer 19

Sometimes additional independent variables add no significant information to the regression model, yet R^2 increases. R^2 therefore may yield an inflated figure.
Statisticians have developed an adjusted R^2 to take into consideration both the additional information each new independent variable brings to the regression model and the changed degrees of freedom of regression.

Question 20

adj. R^2

Answer 20

1 - (SSE/n-k-1)/(SSY/n-1)