Validity & Reliability Flashcards
1
Q
Precision
A
- Consistency, reliability, homogeneity of the data
- calibration of apparatus
- consistency of participant
- environmental factors (eg. Humidity, wind)
-measures of variability: standard deviation, coefficient of variation (SD/mean)x100
- measures of concordance: correlation coefficient (consistency of results of paired measurement
- coefficient of correlation is an index of concordance
2
Q
Accuracy
A
- based on precision of measurement
- good accuracy if averaged values = standard (standards rare in biological sciences)
- accuracy=sensitivity+specificity
3
Q
Reliability
A
Consistency of the measures. Precision
4
Q
Validity
A
does a variable represent what it is intended to represent
5
Q
Nuisance variables
A
- systematic error/bias
- accuracy influenced by systematic error
- extraneous/confound variables are source of bias
- types of biases and solutions:
- observer/experimenter: blinding procedures
- subject/participant: blinding/unobtrusive measures
- apparatus: calibration
- random error
- precision of measurements (and therefore consistency and reliability) influenced by random error (unpredictable fluctuations)
- due to random fluctuations in participants, experimental conditions, method of measurement etc
- main sources:
- observer reliability
- participant reliability
- instrument reliability
6
Q
Reliability/consistency
A
- consistent results over repeated measurements
- refers to precision of measurements
- test-retest reliability: stability of test scores over time
- alternative (parallel) forms: eg: use of two lists in a memory test; recognition/recall example with tests
- internal consistency: how consistent is the measure across items intended to measure the same concept
- inter-rater reliability
- intra-rater reliability
7
Q
Validity
A
- does it measure what its intended to measure
- construct: wrong independent variables are identified
- statistical conclusion: random error and wrong selection of statistical tests, low power, violation of statistical assumptions
- internal validity
- external validity
8
Q
Face validity
A
- how well the test appears to measure what it is designed to measure
- it is a plausible measurement of the variable we want to estimate
- eg: common sense definition of stress
9
Q
Content validity
A
- how adequately the measure addresses the representativeness of the measured event or phenomenon as a whole
- represents the whole content
- expert opinion can determine this type of validity
- eg: you lack content validity if you want to measure stress, but only take behavioural measures and no physiological measures
10
Q
Construct validity
A
- a measure of how well a test and operational definition assess some underlying (theoretical) construct or variable
- depends heavily on the operational definitions
- eg: “stress” the measurement procedure and the variable it measures are in agreement
- an assay of glucocorticoids suggesting high levels of cortisol is associated with high stressful situations
11
Q
Criterion validity
A
- the ability of a measure to assess or predict an outcome or criterion
- performance measures
- types of criterion-related validity:
- concurrent validity
- convergent/divergent validity
- discriminant validity
- predictive validity
12
Q
Concurrent validity
A
- a measure of how well a test estimates a criterion/performance in relation to another concurrent phenomenon or group of subjects at the same point in time
- a new test is validated as it concurs with an older, better established one
- eg: measuring endocrinological stress at the level of the HPAC axis: validating ELISA with RIA for a new species
- the two tests should measure and determine the same thing
13
Q
Convergent/divergent validity
A
- two or more methods of measurement converge/diverge upon one another
- strong relationship between scores are found
- can be established by correlation
- eg: pupil dilation and prosodic pitch in highly stressed individuals converge as measures.
- blood pressure and cholesterol levels are associated with heart disease
- same results from 2 different measures
14
Q
Discriminant validity
A
- the methods of measurement diverge upon one another and the divergence is expected
- a measure of stress should not be expected to be highly correlated with a measure/construct of empathy
- good stress is NOT distress… may present the same but they are not the same
- must be able to distinguish measures for each
15
Q
Predictive validity
A
- a measure of how well a test predicts a phenomenon on a time criterion
- eg: pre/post
- measures future states
