Lecture 5: Reliability And Validity

Question 0

Face validity

Answer 0

• does the measure appear to measure what it claims to measure?
• assessed qualitatively or by expert panel and/or client input
• consider: has little scientific rigor, necessary but not sufficient condition

Question 1

The process of scale development

Answer 1

• the intuitive concept
• definition of construct
• operational definition
• measurement scale
• validity of measurement
• reliability of measurement

Question 2

Concurrent validity

Answer 2

• The new measure should correlate (imperfectly) with an established measure of the same construct
• needs to be theoretically related as well

Question 3

Convergent validity

Answer 3

• 2 new measures of same construct correlating

Question 4

Construct validity

Answer 4

• demonstrates that the measure being validated behaves as the construct would behave under varying conditions
• assessed through a triangulation of correlations

Question 5

Discriminant validity

Answer 5

• Groups that ought to differ with respect to the construct are found to do so. Related to predictive validity
• assessed via differences in mean scores by group and impaired t-test to assess statistical significance of difference

Question 6

Sources of unreliability: observer error

Answer 6

• technical errors or misjudgments by the individual rating participants
• Assessed: difficult, inter-eater reliability covers part of this problem

Question 7

Sources of unreliability: Environment changes

Answer 7

• changes in the environment may influence performance by the participant
• ideally test under consistent circumstances

Question 8

Sources of unreliability: participant changes

Answer 8

• Changes within the participant that alter the score

Question 9

Sources of unreliability: changes in the construct

Answer 9

• the construct being measured changes b/w testing occasions e.g. Pain
• measurements need to be made near enough in time to minimize risk of change in the underlying construct

Question 10

Test-retest reliability

Answer 10

• same scores from individuals across 2 points in time

- assessed by correlation and examining the magnitude and significance of change within person

Question 11

Parallel forms reliability

Answer 11

• similar to test-retest but with alternate versions of the test being used
• helps avoid scores changes due to exposure to test

Question 12

Internal reliability

Answer 12

• the extent to which the items that make up a measure are all measuring a consistent construct
• assessed by split half reliability (correlate scores calculate by randomly splitting the items into 2 halves) which can lead to different correlations depending on how the items are split
• Cronbach’s alpha: an average of all possible split half correlations