Week 3: Quality of Measurement Instruments

Question 1

reliability coefficient rule of thumb

Answer 1

questionnaire/test needs at least 0.7 for scientific purposes

Question 2

test-retest reliability

Answer 2

• for test, test two more times in same sample
• determine consistency in time

Question 3

inter-item reliability or internal consistency

Answer 3

for the whole test, determine to what extent all items of instrument covary

Question 4

rule of thumb for Cronbach’s alpha

Answer 4

< 0.6, insufficient
0.6 - 0.8 moderate fo good
> 0.8 good

Question 5

inter-rater reliability

Answer 5

rate phenomenon by at least two raters

association between ratings

Question 6

parallel-test reliability

Answer 6

• develop two equivalent (content, response format, statistics) tests but with different items

association between scores of the two tests

Question 7

replication

Answer 7

repeat the whole study/experiment

if effect is true/exists, results should be consistent

Question 8

to improve reliability

Answer 8

1. reliable instruments
2. standardize testing procedure
3. clear instructions
4. train raters using a protocol
5. protocolize data collection

Question 9

types of validity

Answer 9

1. face validity
2. construct validity
3. criterion validity
4. content validity

Question 10

face validity

Answer 10

does it look valid at first sight?

Question 11

construct validity

Answer 11

association with other tests that measure similar constructs

Question 12

criterion validity

Answer 12

does measure predict criterion as expected

• concurrent: predict criterion now
• predictive: predict criterion in the future

Question 13

content validity

Answer 13

does a test test the full domain?