Reliability and Validity

Question 1

What does Chronbach’s Alpha measure?

Answer 1

The uniformity or homogeneity of items.

Question 2

How does test length affect Chronbach’s Alpha?

Answer 2

More test items = Higher alpha level.

Question 3

What does test-retest reliability measure?

Answer 3

The stability of a test. The same test is given more than once and the scores are compared. The longer the interval between tests, the lower the score on the second test will be.

Question 4

What is alternate forms reliability?

Answer 4

Multiple forms of the same test are given. If there is AFR, the scores on different forms should be similar.

Question 5

What is interrater reliability?

Answer 5

The degree to which raters agree. If two people observe the same child, how close would their ratings of the child’s behavior be?

Question 6

What does validity measure?

Answer 6

Does the test measure what it is supposed to measure? No test is valid for all purposes. A test is only valid for a specific purpose. A valid test means that you can make inferences/conclusions based on the results.

Question 7

How do reliability and validity interact?

Answer 7

Reliability is necessary for validity, but it does not guarantee validity.

Question 8

What is construct validity?

Answer 8

The degree to which a test measures a purported psychological construct/trait

Question 9

What is convergent validity?

Answer 9

Convergent validity measures how well measures of the same domain in different formats correlate with each other.

Question 10

What is divergent validity?

Answer 10

Divergent validity measures how little measures of different domains correlate with each other .

Question 11

What is concurrent validity?

Answer 11

• Based on correlation of scores on two different measures given one right after the other
• If scores on both tests correlate -> good concurrent validity

Question 12

What is predictive validity?

Answer 12

• Based on correlation of scores on two measures with a time gap between administration
  e. g. reading test given at beginning of school year and then at the end of the school year
• Good predictive validity if scores are similar (e.g. high score on first test and second test)

Used when criterion measure is available in the future

Question 13

What does reliability measure?

Answer 13

• How precise is psychological measurement?

- If Katie took the WISC over and over, would she get the same score?

Question 14

What are measurement errors?

Answer 14

Different factors that can affect a test score. (Mood, affect, examiner error, etc.)

Question 15

What is correlation?

Answer 15

The degree to which two variables covary. Ranges from -1 to 1.

Question 16

How does the range affect correlation?

Answer 16

Lower range = lower correlation

Higher range = higher correlation

Question 17

What are ceiling effects?

Answer 17

The number of difficult items at higher levels of a test that can differentiate among participants who test well. More test items gives you more ability to differentiate between abilities at different levels.

Question 18

What are floor effects?

Answer 18

-The number of easy items at the lowest level of a test that can differentiate among children who show low ability
More items = more ability to differentiate ability at different levels

Question 19

What is effect size (cohen’s d)?

Answer 19

• Measures the degree/magnitude of a result

- Based on standard deviation units, independent of the sample size

Question 20

How does reliability relate to standard error?

Answer 20

If reliability is 1 (perfect), there is no error in measurement.