Lecture 8: Measument invariance, test fairness and norms

Question 1

What is measurement invariance (definition)?

Answer 1

interindividual differences in a (usually) continuous characteritic (intended construct) are recorded in the measured variables (items of a test) independently from any portential group membership.

Question 2

Differential Item Functioning (DIF) - definition

Answer 2

DIF is present if an item exhibits different psychometric porperties across groups, independetly from the question if there are real differences (in the construct) between the groups.

Question 3

If messurement invariance is teneable … (vertretbar)

Answer 3

• group differences can be interpreted as real
• indvidual items do not distort group differences in an uncontrolled way because of measurment error.

Question 4

Predictive invariance (definition)

Answer 4

is present if interindividual differences in a (usually) continuous predictor are related to criterion performance independently from any group memebership.

Question 5

Test bias (definition)

Answer 5

because of certain group membership, an individual’s chances of attaining a certain goal are no (only) related to the level of the measured characteristic.
- this included decreased and increased chances.

Question 6

What deos a non-existence of test bias imply?

Answer 6

that every individual’s chances of attaining a certain goal are the same and only dependent on their level of the measured characteristic but independent from any other variables, such as membership in a certain group.

Question 7

What can be causes of test bias?

Answer 7

• tests are usually developed and produced in the language of the majority group
• members of the majority group are more familiar with the task and item formats used in the tests.

Question 8

What is an easy and historical idea to identify test bias?

Answer 8

identical regression slopes + intercepts (meaning no test bias)

Question 9

How do you “build” a linear regression model?

Answer 9

1. Predictor
2. Predictor + group membership (usually dummy coded)
3. Predictor + Group + Interaction

Question 10

What are norms (definition)

Answer 10

are measurement units into which raw scores of different diagnostic measures can be transformed with the goal of making them comparable and interpretable in the context of a (preferably) representative reference group.

Question 11

What are important characteristic of norms?

Answer 11

1. Are needed if we want to compare test scores.
2. Are always tied to clearly described populations.
3. We can use group-specific norms in order to improve test fairness.

Question 12

Variability norms

Answer 12

allow individual test scores to be classified with regard to the distribution of scores observed from a representative reference group.

Question 13

How is a scores z-standardized?

Answer 13

z= x - x(mean)/ Sx

Question 14

What do standard scores always have?

Answer 14

• a mean of 0
• and a standard deviation of 1

Question 15

What are some commonly used scales?

Answer 15

1. IQ = 100 +15z
2. SV = 100 +10 z
3. T = 50 + 10z

Question 16

Percentile rank norms - what is special about them?

Answer 16

• accentuate differences in the measured variables in the middle area but level them out in extreme areas.

Question 17

What are different rough norms (Grobnormen)?

Answer 17

1. Stanine (Standard nine)
2. Sten (standard ten)

Question 18

When are reduced (rough) norms utilized?

Answer 18

• due to relatively low reliability of diagnostic tests, the need to employ large confidence intervals around the individual test scores
• then reduced (rough) norms may be utilized.

Question 19

What to do in the case of deviation from normal distribution?

Answer 19

• normalize raw scores
• conversion of raw scores into percentile ranks
• z-transformation of percentile ranks.

Question 20

In case of deviation from normal distribution - When is the z-transformation of percentile ranks applicable?

Answer 20

• if the deviation from normality is not too large
• and we can assume the measured construct is indeed distributed normally in the population.

Question 21

What are different samples types?

Answer 21

• regional (area is divided into subgroups, and from each subgroup sb. selected)
• quota sample (for example women and men are selected in the same amount)

Question 22

What is the case about sample size planning?

Answer 22

• the anticipated level of reliability
• the anticipated measurement accuracy
• utilization of the confidence interval for individual scores.

Question 23

What is the standard error of the mean?

Answer 23

a measure of the accuracy of the estimate of the population mean

Question 24

How is the standard error of the mean calculated?

Answer 24

by dividing the unbiased estimate of the standard deviation by the square root of the sample size.

Question 25

What do group-specific norms affect?

Answer 25

• can affect the reliability as well as validity of a test result.
• can improve or reduce test fairness
• operate similar to quotes

Question 26

Why are norms problematic?

Answer 26

participants are not treated equally.