Coding Categorical Variables

Question 1

What are the 2 ways to code for a dichotomous predictor?

Answer 1

Dummy Coding & Effect Coding

Question 2

What are the differences between ways to code for dummy and effect coding?

Answer 2

Dummy code is 1 and 0 and Effect code is -1 and 1.

They serve different functions.

Question 3

What is 0 considered in dummy coding?

Is there a reference group in effect coding?

Answer 3

0 is the reference group.

There’s no reference group in effect coding.

Question 4

When we are looking at sex differences between GRE scores, what type of information do we need?

Answer 4

Male mean
Female mean
Grand mean

Question 5

How do we set up dummy code on r?

Answer 5

Instead of computing statements, we use contrast statements to set-up dummy codes.

Question 6

From the output in r, what will the estimate of the intercept always be when we dummy code?

What will the b1 coefficient be?

Answer 6

The estimate of the intercept will always be the mean of the reference group (0).

The b1 coefficient will be the difference between the reference group and 1.

ex) b0 = 585
b1 = -10

-10 is the difference between the mean of 585 and female score of 575.

Question 7

From the output in r, what will the estimate of the intercept always be when we effect code?

What will the b1 coefficient be?

Answer 7

The b0, or intercept will be the Grand mean of the Equal group sizes.

b1 coefficient is the difference bw Male Mean and Grand Mean

ex) b0 = 580
b1 = 5

Question 8

How do we do an equation for dummy codes?

How do we interpret the coefficients?

Answer 8

We will be plugging the 0’s and 1’s into the difference of means starting with the reference group.

ex) yhat = b0 + b1D1 + b2D2 + b3D3 + b4D4

b0 is the mean of the reference group

b1 – difference between mean of interest group and reference group mean… so on and so on.

Question 9

The intercept of effect coding gives us _________.

The coefficients are the difference between __________ and __________.

Answer 9

the Grand mean

the coefficients are the difference between the Grand Mean and the mean of the group we are interested in (first group).

Question 10

For equal n dummy coding, how many code vectors are required?

Answer 10

Multiple coded vectors, where coding vectors = # of categories - 1.

Question 11

How can we apply ANOVA type partitioning to Effect coding? Conceptually, this shows that ANOVA and regression are pretty much the same.

Answer 11

The structural model is similar in that Y = GM + unique + error.

Each subject’s score includes contribution of the overall score, treatment effect, and error.

Overall = grand mean
Treatment effect = the effect of 1 coefficient
Error = the residual

Question 12

What are examples of categorical groups?

Answer 12

Ethnic groups
gender
SES

Question 13

When would we use effect coding over dummy coding, and vice versa?

Answer 13

Effect coding will be used if we are looking at the effects of each group.

Dummy coding to look at how each group varies compares to our reference group.

Question 14

What type of mean is now added for unequal n?

Answer 14

The average mean, which is the difference of

Question 15

What’s the difference between grand mean and average mean (unequal n)?

Answer 15

The grand mean is the total number of scores we have divided by the total number of n.

The average mean divides the total mean scores by the number of mean scores.

Question 16

When the data are unbalanced (unequal n), how do the coding of vectors and interpretation of output from (1) dummy coded, (2) effect coded, and (3) orthogonally coded analyses differ (from the equal n case)?

Answer 16

Nothing changes for dummy coding, we interpret it the same as we did for equal n.

However, for effect coding, IF THE EFFECT CODE STAYS THE SAME (-1 and 1) Interpretations change; the intercept is no longer the grand mean, now it’s the unweighted Average Mean of the group. The coefficients will be the Difference of the Group mean and the Average mean.

Question 17

How do we restore our interpretation for effect coding for unequal n?

Answer 17

We restore the interpretation by introducing Weighted effect codes.

Question 18

If we want the intercept to get back to to the grand mean rather than the average mean, what would we do?

Answer 18

We have to use weighted effect codes.
We have to weigh our groups by n. So instead of using -1 for comparison group, we use values based on sample sizes of the 2 groups involved in the contrast.

ex)
ex) counting =8, intentional (comparison group) = 10.

so, -8/10 = -.8

These negative values are used in place of -1.

Question 19

How would we calculate the weight means for unequal effect code?

Answer 19

We take the -n of group we’re interested in OVER the n of the comparison group.

ex) counting =8, intentional (comparison group) = 10.

so, -8/10 = -.8

These negative values are used in place of -1.

Question 20

How do we interpret the output of the intercept and b-coefficients in R after running weighted effect codes?

How do we create an equation for the weighted effect codes?

What does the equation equal to?

Answer 20

The intercept is Grand mean.
The b1 coefficient is the difference between the group mean - the grand mean and so on.

The equation is as follows:
ŷ = GM + b1(W1) + b1(W2) + b2(W3); etc.

ex) ŷ = 10.20 + (-3.20)(-.8) + (-2.9)(-.7); etc…

GM = 10.20
b = -3.20 and -2.9
W = -.8 and -.7

The equation equals to the mean of the reference group (or Intentional group).

Question 21

How is weighted effect codes similar to ANOVA?

Answer 21

We get the same F information as if we ran an ANOVA.

The multiple R-squared here is the same as the eta-squared for ANOVA.

Question 22

Conceptually, what does it mean when the eta-squared goes up?

Answer 22

It means that the more predictors we add, the larger the eta-squared value meaning that the treatment might not be that effective - just that size of the treatment groups make the effect size bigger.

Question 23

How does weighted effect codes in R correct issues with ANOVA?

Answer 23

When there’s a significant ANOVA, we move on to multiple comparison tests.
The problem is that we inflate the Type 1 error.

However, the weighted effect codes produce significant effects of each coefficient without the inflation of the type 1 error.

Question 24

How do we verify and check orthogonality for more than 2 codes (4+ groups) for EQUAL n?

Why do we verify and check for orthogonality?

Answer 24

The sum of the multiplied contrasts (+ or -) need to equal zero. We need to make sure to check each possible pairs separately:

C11C21 +C11+C31+C11+C41
+C12C22+C12C32+C12C42; etc

We verify orthogonality because we need to check the independence of the variables since although there could be zero correlation (no linear relationship)- that doesn’t mean there’s no co-linearity.

Question 25

How many vectors do we need to represent a categorical variable for orthogonal coding?

Answer 25

o Orthogonal - A full set of orthogonal comparisons include # of levels – 1 comparison, exhausting available information

Question 26

What types of conditions must contrasts (+ or - signs) meet to be considered orthogonal?

Answer 26

o Independence – no linear relationship

o Zero correlation is necessary but not sufficient;
• zero correlation means no linear relationship, but we can still have zero correlation and have a relationship to a variable.

o Orthogonal vectors can be used to code a priori comparisons among group means

o A full set of orthogonal comparisons include # of levels – 1 comparison, exhausting available information
• Treeing-out on its side, but we make vectors.

Question 27

How do we interpret the output of the intercept and b-coefficients in R after running equal n Orthogonal codes?

What does the equation equal to?

Answer 27

• b0 – Intercept is Grand mean
• b1 – Each coefficient means are treed-out and divided over the range of codes we use (smallest and biggest).

EXAMPLE)
o b1 = .485 = (mean of counting, rhyming, adjective, and imagery groups – mean of intentional)/range of code = (9.575 – 12)/ (1-(-4))
o b2 = 1.275 = (mean of counting, rhyming, adjective – imagery groups)/range of code = (8.3-13.4)/ (1-(-3))
o b3 = ‐1.35 = (mean of Counting and Rhyming groups– mean of Adjective) / range of code = (6.95‐11)/(1‐(‐2))
o b4 = .05 = (mean of Counting – mean of Rhyming) / range of code = (7‐6.9)/(1‐(‐1))

The equation always equals the predicted values (group means).

Question 28

What’s a caveat of using equal n orthogonal coding?

Answer 28

Caveat is that we have to divide everything by the range of codes we used (the smallest to biggest; 1-4 for example).

Question 29

We use our orthogonal codes in 2 ways, what are they?

Answer 29

1. We create our coefficients by looking down the column.

2. To get back our group means, or our predictive values, we go across the row.

Question 30

What is the purpose of criterion scaling? How is it carried out? What portions of standard regression output need to be corrected?

Answer 30

o Criterion scaling creates a single coded vector (not 4 like before) to represent group membership since we can’t apply selection algorithms or all possible regressions with categorical predictors.
o Useful when there’s many levels of categorical predictors in an analysis and for model selection procedures
• If we have 2 categorical predictors and we want to see the interaction between age and rhyming, it’s awkward to do that with all of the vectors, so we use criterion scaling to look at the interaction first, THEN break it apart if necessary.

Question 31

How do we code things by criterion scaling?

Answer 31

We code the vectors in criterion scaling by using the average of each group and use it as the code.

So instead of 0 1, -1 1; etc… this time, we use the mean of the group as coding vectors.

Question 32

How do we code things in criterion scaling?

Answer 32

We code things by criterion scaling by using the average of each group and use it as the code.

This allows us to include categorical predictors with many categories into our selection algorithms.

Question 33

In the standard regression output, the following need to be corrected:

Answer 33

• df (using 1 vector to code the categorical predictor) fucks everything up including the F-test and t-test.
• We need to re-calculate the df, MS, and F-test by hand (using correct df information)

Question 34

How is criterion scaling useful?

Answer 34

o Criterion scaling is useful in predicting whether the categorical predictor itself is even worth including in our model.

• It’s useful for model selection and model building because the R2 for the model is appropriate which means that if we added another predictor, we could do an F-test of the R2∆ to see if my categorical predictor accounts for a significant proportion of variability in my outcome measure, over and above the other predictors in the model. If it does account for a significant proportion of variability, then we can remove the criterion scaled vector, and re-run a different coding scheme including all the vectors into my model.

Question 35

• How many vectors do you need to represent a categorical variable?

Answer 35

o # of categories – 1; so if we have 5 categories, we would have 4 vectors

o Orthogonal - A full set of orthogonal comparisons include # of levels – 1 comparison, exhausting available information

Question 36

Under what circumstances might dummy, effect, and orthogonal coding be especially useful?

Answer 36

o Dummy coding is useful for looking at how each group varies compared to the reference group.
o Effect coding is useful for looking at the effects of each group in general.
o Orthogonal coding is useful

Question 37

• How do you interpret the test of R2 in an analysis involving categorical predictor variables? Will this depend on the coding scheme?

Answer 37

o dummy code – the predicted value of the models gives us back the means of each variable (gender) – we are comparing significant differences of coefficients to the reference group.

o effect code – gives us the effect of one coefficient (the effect of counting on intentional; etc) – shows us if a coefficient is significantly different from the grand mean (similar to SS between in ANOVA).

• Caveat is that we won’t get the effect of the “reference” group.

Question 38

• Be able to interpret b’s from regression analyses (with equal n) using dummy coding of categorical variables.

Answer 38

o dummy code – reference group is the one we don’t care about (because we’re comparing this to zero, which we don’t care about – the other coefficients are being compared to the reference group, so significant test actually matters).

• b0 – is the mean of the reference group
• b1 – difference between mean of interest group (first group) and reference group mean.
• b2 – difference between mean of 2nd group and reference group mean.; until b4 (no b5 because we do not include the reference group).

Question 39

• Be able to interpret b’s from regression analyses (with equal n) using effect coding of categorical variables.

Answer 39

o effect code
• b0 – Intercept is always Grand mean
• b1 – Difference between the mean of the first one and Grand mean (effect of treatment 1)
• b2 – Difference between mean of 2nd group and Grand mean; so on… (effect of treatment 2; etc. Until b4 and no b5).

Question 40

• Be able to interpret b’s from regression analyses (with equal n) using orthogonal coding of categorical variables.

Answer 40

o orthogonal
• b0 – Intercept is Grand mean
• b1 – Each coefficient means are treed-out and divided over the range of codes we use (smallest and biggest).
• EXAMPLE)
o b1 = .485 = (mean of counting, rhyming, adjective, and imagery groups – mean of intentional)/range of code = (9.575 – 12)/ (1-(-4))
o b2 = 1.275 = (mean of counting, rhyming, adjective – imagery groups)/range of code = (8.3-13.4)/ (1-(-3))
o b3 = ‐1.35 = (mean of Counting and Rhyming groups– mean of Adjective) / range of code = (6.95‐11)/(1‐(‐2))
o b4 = .05 = (mean of Counting – mean of Rhyming) / range of code = (7‐6.9)/(1‐(‐1))

Question 41

• When the data are unbalanced (unequal n), how do the coding of vectors and interpretation of output from (1) dummy coded, (2) effect coded, and (3) orthogonally coded analyses differ (from the equal n case)?

Answer 41

o dummy code- Codes are identical to equal n – interpretations are unchanged.
o effect code- IF effect codes aren’t changed, interpretations change; the intercept is no longer the grand mean, now it’s the unweighted Average Mean of the group. The coefficients will be the Difference of the Group mean and the Average mean.
• This poses an issue because we aren’t really interested in the average mean, we’re interested in the effect of a coefficient against the grand mean. So we calculate using Weighted Effect codes (-ncurrent ÷ ncomparison; counting = 8, intentional (comparison group = 10); -8/10 = -.8. These negative values are used in place of -1.

Question 42

• Orthogonal vectors must meet what conditions?

Answer 42

Independence – no linear relationship
o Zero correlation is necessary but not sufficient;
• zero correlation means no linear relationship, but we can still have zero correlation and have a relationship to a variable.
o Orthogonal vectors can be used to code a priori comparisons among group means
o A full set of orthogonal comparisons include # of levels – 1 comparison, exhausting available information
• Treeing-out on its side, but we make vectors.