Regression Analysis, Time Series Analysis

Question 1

How is correlation coefficient notated/

Answer 1

If measured for a population, called ? (rho)

If estimated from a sample, sure r; i.e r estimates ?

Question 2

What is true of the correlation coefficient?

Answer 2

• Correlation and covariance are appropriate for use with continuous variables whose distributions have the same shape (e.g. both normally distributed)
• If these assumptions are not met, r will be ‘deflated’ and underestimates ?.

Question 3

What are the data in a regression analysis?

Answer 3

• One continuous response variable (called y - dependent variable, response variable)
• One or more continuous explanatory variable (called x - independent variable, explanatory variable, predictor variable, regressor variable)

Question 4

What is true of εi?

Answer 4

Mean of εi will be zero

Question 5

What does E(Y|X=x) = β0 + β1x mean?

Answer 5

The expected value of Y when X = x is β0 + β1x

Question 6

What is the true/population regression line?

Answer 6

• Yi = β0 + β1xi + εi
• β0 and β1 are constant to be estimated
• εi is a random variable with mean = 0 if our line is going through the middle of our data

Question 7

How is the population regression line estimated?

Answer 7

• ŷ = b0 + b1x
  
  • b0 and b1 are estimated values
  • ŷ is a fitted value of the response

Question 8

What is a residual?

Answer 8

vertical distance between observed response and fitted value of response

Question 9

How are residuals estimated?

Answer 9

ri estimates εi, the error variable

Question 10

What is SSE?

Answer 10

The error sum of squares

SSE = nΣi=1 (yi - ŷ)2

Question 11

What error assumptions do we make in regression analysis?

Answer 11

• In our fitting we assume the errors have a particular distribution - that is, ε ~ N(o, σε2)
  
  • Normal distibution
  • Mean = 0
  • Constant variance = σε2
  • Errors associated with any two y values are independent

Question 12

What is sε?

Answer 12

• sε = standard error of the estimate
• Interpretation - standard deviation of residuals; standard error in predicting Y from the regression equation - best definition: standard deviation around prediction line

Question 13

What are the t stats in regression analysis output?

Answer 13

T = test statistic (that population intercept/slope = 0 against two sided alternative), compared to t with n-2 degrees of freedom finds P = 0, i.e. intercept/slope is not 0

Question 14

What is S in regression analysis output?

Answer 14

Standard Error of the Regression (S) = average distance that values fall from regression line

Question 15

What is R^2?

Answer 15

• Determine the strength and significance of association
• coefficient of determination
• measures proportion of total variation explained, i.e.
• = explained variation / total variation = SSreg / SSy =(correlation coefficient)^2
• Will be between 0 and 1; a value close to 1 indicates most of the variation in y is explained by the regression equation

Question 16

What is important about R?

Answer 16

r = ± √r2

Question 17

What is Homoscedasticity?

Answer 17

If variation is constant (residuals show constant spread around zero), called homoscedastic

Question 18

What is Hetroscedasticity?

Answer 18

If variation is non-constant (residuals show varying spread around zero), called heteroscedastic

Question 19

What is true about Large Standardised Residuals?

Answer 19

Minitab flags “Large Standardised Residuals” R - should be about 5%, - indicates normality of residuals

Question 20

What must be true to make predictions from a regression analysis?

Answer 20

• High R-sq, small std error of estimate
• All assumptions appear valid
• Predictions should only be made for values inside the observed limits

Question 21

What does β1 represent in a multiple regression with 2 predictors?

Answer 21

β1 represents the expected change in Y when X1 is increased by one unit, but X2 is held constant or otherwise controlled

Question 22

What is meant by additive effects of multiple regression?

Answer 22

Combined effects of X1 and X2 are additive - if both X1 and X2 are increased by one unit, expected change in Y would be ( β1 + β2 )

Question 23

What must be true for us to find a Least Squares solution for a multiple regression?

Answer 23

• Number of predictors is less than number of observations

- Non of the independent variables are perfectly correlated with each other

Question 24

What is true of the coefficient of multiple determination?

Answer 24

• Will go up as we add more explanatory terms to the model whether they are important or not
• Often we use adjusted R-sq - compensates for adding more variables, so it lower than R-Sq when variables are not “important”
• So, if comparing models with differing numbers of predictors, use Adjusted R-Sq to compare how much variation in response is explained by model

Question 25

What are the rules of dummy variable regression?

Answer 25

• Can code any discrete variable with k categories into (k-1) distinct dummy variables
• Usually only used when variables have 2 (sometimes 3) categoreis/levels

Question 26

What is a polynomial regression?

Answer 26

• Y = β0 + β1X + β2X^2 + β3X^3 + … + ε
• Equivalent to fitting a multiple regression where
  
  • X1 = x
  • X2 = x^2
  • Xk = x^k

Question 27

What is completeness and interactions terms in polynomial regression?

Answer 27

• Called “complete’ if all lower order terms of x are significant
  
  • If only had x and x^3 would be incomplete, third order polynomial regression
• Interaction Term
  
  • This is needed if the level of X1 affects the relationship
    between X2 and Y
• e.g. Second order model with interaction
• Y = β0 + β1X1 + β2X1^2 + β3X2 + β4X2^2 + β5X1X2 + ε

Question 28

What is overparamaterisation?

Answer 28

• Polynomial regression
• Because we’re fitting so many predictors (parameters) to so few observations, the regression may fit to data too well
• Meaning that it might not predict the population accurately
• Model doesn’t generalise
• High r SQ

Question 29

What are the components of a time series?

Answer 29

• Long term trend
• Cyclical variation
• Seasonal variation
• Random variation

Question 30

What is long term trend?

Answer 30

• Also called secular trend
• Relatively smooth pattern or direction
• Can be linear or non-linear

Question 31

What is cyclical variation?

Answer 31

• Wave-like pattern describing long term trend apparent over a number of years - cyclical effect
• Recurrence period over 1 year (definition)
• e.g. Business cycles
• Rare to find cyclical patterns that are consistent and predictable

Question 32

What is seasonal variation?

Answer 32

• Cycles that occur over short repetitive calendar periods
• Duration less than one year (definition)
• “seasonal” may mean 4 seasons, or systematic patterns over a month/week/day
• e.g. restaurant demand features “seasonal” variation throughout the day, monthly traffic volume

Question 33

What is random variation?

Answer 33

• Irregular, unpredictable changes
• Not caused by other components (trend, cyclical, seasonal variation)
• Often referred to as “noise”
• Can mask the existence of other components
• Exists in all time series
• Goal of most time series analysis is to reduce impact of random variation on forecasting or interpretation