Week 6 Reliability & Validity

Question 1

What are the key criteria for evaluating quantitative measurement?

Answer 1

1. Reliability
2. Validity

Question 2

Reliability

Answer 2

How consistently a data collection instrument measures the variable

Question 3

Validity

Answer 3

Degree to which an instrument measures what it is supposed to measure

Question 4

What are the 3 aspects of reliability?

Answer 4

1. Stability
2. Internal consistency
3. Equivalence

Question 5

Stability

Answer 5

Extent to which scores are similar on 2 separate administration of instrument, appropriate for fairly enduring characteristics
e.g personality, IQ test

Question 6

How is stability assessed?

Answer 6

Test retest reliability - same instrument is given 2x to same group, scores may not be identical but differences are small (reliability coefficient)
Range from 0-1 (0.7-0.8 acceptable)

Question 7

Internal consistency

Answer 7

Extent that all subparts of instrument measure same trait, appropriate for multi item instruments by administering instrument on one occasion

Question 8

How is internal consistency assessed?

Answer 8

Cronbach’s alpha (coefficient alpha) - indicates how well a group of items together measure the trait of interest
0.7-0.9 acceptable

Question 9

Equivalence

Answer 9

Concerns the degree to which 2 or more independent observers agree about the scoring of an instrument

Question 10

How is equivalence assessed?

Answer 10

Inter-rater reliability using Cohen’s Kappa (k) & intraclass correlation coefficient (ICC)

Question 11

What are the 2 aspects of validity?

Answer 11

1. Content validity
2. Criterion related validity

Question 12

Content validity

Answer 12

Appropriately & adequacy of instrument content

Question 13

How is content validity assessed?

Answer 13

Expert panel review rate items on a 4-point scale by using content validity index (CVI, only for 3/4) for evaluating each item’s relevance (0.9>)

Question 14

Criterion validity

Answer 14

Extent to which instrument corresponds to gold standard or another well established measure of target variable

Question 15

Types of criterion validity

Answer 15

1. Concurrent validity
2. Predictive validity

Question 16

Concurrent validity

Answer 16

Correlation of tested & gold standard instrument scores at the same time

Question 17

Predictive validity

Answer 17

Correlation of scores with of outcome
To predict responses in the future