Week Four - Reliability & Validity Flashcards
In order for measures to be reliable..
Tools used for measurement must be consistent and accurate (precise and unbiased)
In order for measures to be valid..
Descriptions, relations and explanations (causation) must be truthful
Reliability =
Consistency
What are the three broad types of
consistency?
Consistency over time (test-retest reliability)
Consistency across items (internal consistency)
Consistency across researchers (inter-rater reliability)
What is consistency measured by?
Variability
Explain the inverse relationship between variability and reliability.
High variability = low reliability
Low variability = high reliability
Ameasure that is reliable is not automatically..
Valid
Variability occurs for what 2 reasons?
Causal relationship between an IV and DV (effect).
.Variability due to random factors (noise).
Short-term change may indicate..
Problems with reliability (and therefore validity)
Long-term change may reflect
More likely to reflect real changes/effects in
variable, not merely error
What is Test-retest Reliability?
Test-retest reliability is a measure of reliability obtained by administering the same test twice over a period of time to a group of individuals. The scores from Time 1 and Time 2 can then be correlated in order to evaluate the test for stability over time.
What is Internal consistency?
Consistency of responses across test items.
If participants are consistent in their responding to the questions scores on different items should be correlated.
How do we test internal consistency? (2)
Split-half reliability.
Cronbach’s alpha (α)
What is Interrater reliability?
Inter-Rater Reliability refers to statistical measurements that determine how similar the data collected by different raters are. A rater is someone who is scoring or measuring a performance, behavior, or skill in a human or animal.
Inter-rater reliability is tested using what?
Cohen’s kappa (κ)
What is Validity?
Validity is the extent to which measures represent what they are intended to represent
Three major forms of validity
Construct validity
Internal validity
External validity
What is Construct validity?
Extent to which manipulated variables (independent variables, IVs) and measured variables (dependent variables, DVs) represent what they are intended to.
- Are IQ tests a measure of intelligence?
Construct validity can be tested through what 2 forms of validity?
Convergent and Discriminant validity
Convergent Validity would be demonstrated by strong, positive correlations (≥ .8)
Discriminant Validity would be demonstrated by weak correlations among assumed unrelated measures (≈0).
What is Face Validity?
Do IVs and DVs appear to reflect or measure what they are intended to measure.
What is Internal Validity?
Extent to which causal statements about the relations among variables can be made.
What is External validity?
The extent to which the findings of study can be generalised beyond the study itself.
External validity can only be confirmed by
Replication
Internal Validity is dependent upon what?
Study design
Designs that effectively control extraneous variables have high what?
Internal validity
When does Externally invalid research occur?
When research conclusions are limited to original setting, sample and/or situation.
What does Convergent validity test?
whether constructs/measures that should be related are related.
What does discriminant validity test?
Tests whether constructs/measures that should be unrelated are unrelated.
What does predictive validity test?
Tests whether current performance on some measure predicts future performance on a different.
What does Concurrent validity test?
Test whether performance on a validated test correlates with performance on a new test.
What are the 7 threats to Internal validity?
- History
- Maturation
- Instrumentation decay
- Statistical regression
- Selection
- Testing
- Mortality
