Week 10 Reliability and validity Flashcards
What is reliability?
The consistency of measurements, or of an individual’s performance on a test; or the absence of measurement error
What are the two types of error?
Systematic and random
What is systematic error?
Consistent error which biases the true score
What is random error?
Unpredictable error which biases the true score
What are the types of reliability?
Relative and absolute
What is relative reliability?
The degree to which data maintain their position in a sample with repeated measurements
What is absolute reliability?
The degree to which repeated measurements vary for individuals
What are the tests for reliability?
Retest reliability
Inter-rater reliability
Internal consistency
How does retest reliability test reliability?
Test stability in measurements across two tests
How does Inter-rater reliability test reliability?
Correlate the scores with data obtained from different researchers
How does internal consistency test reliability?
Reliability across different parts of a measurement instrument (questionnaire sub-scale)
Does correlation infer agreement and provide an example?
No
A perfect positive correlation exists between an expensive
laboratory body fat analyser & a cheap ‘home use’ analyser but they calculate different BF%
What are the measures of absolute reliability?
Technical error of measurement
Standard error of measurement
Coefficient of variation
Limits of agreement
What is validity?
The extent to which a test or instrument measures what i is supposed to measure
What are the types of validity of measurements?
Face validity
Content
Construct validity
Criterion (concurrent and predictive validity)
What are the types of validity in a study?
Internal
External
What is face validity?
The method of data collection obviously involves the factor being measured (timing gates in a 100m sprint)
What is content validity?
Evidence that the content on a test correspond to the content it was designed to cover
What is construct validity?
Assess the extent to which an instrument accurately measures hypothetical constructs
What is convergent validity?
Scores on a instrument are related to those on a similar instrument
What is discriminant validity?
Scores on a instrument are not related to those on a from an instrument which assess a different construct
What is criterion-related validity?
Evidence that scores form an instrument correspond with or predict concurrent external measures conceptually related to the measured construct
What is concurrent validity?
Scores collected at roughly the same time
What is predictive validity?
Criterion instruments completed at a later date (testing a group of subject for a certain construct, and then comparing them with the results obtained at a later date)
What is commonly used to assess criterion related and construct validity?
Person’s correlation coefficient
What is internal validity?
Refers to the ability to attribute changes in the dependent variable to the manipulation of the independent variable.
How is internal validity effected?
Depends on the robustness of methods
What is external validity?
Refers to the ability to generalise the results of a study to other setting and other individuals
What are the threats to external validity?
Reactive or interactive effects of testing
Interaction of selection bias and the treatment
Reactive effects of experimental arrangements
Multiple-treatment interference
