Chapter 4.2: Reliability Flashcards
Refers to stability or consistency of the measurement
Reliability
An index of reliability, a proportion that indicates the ratio
between the true score variance on a test and the total variance
Reliability Coefficient
○ Refers to the proportion of the total variance attributed to true
variance
○ The greater the proportion of the total variance attributed to
true variance, the more reliable the test
Reliability
Goals of Reliability
○ Estimate errors in psychological measurement
○ Devise techniques to improve testing so errors are reduced
All of the factors associated with the process of measuring
some variable, other than the variable being measured
Measurement Error
Categories of Errors:
➢ Source of error in measuring a targeted variable caused by
unpredictable fluctuations and inconsistencies of other
variables in the measurement process
➢ Primarily influences the reliability of the measurement
Random Error
Categories of Errors:
➢Source of error in measuring a variable that is typically
constant or proportionate to what is presumed to be the
true value of the variable being measured
➢Primarily influences the validity of the measurement
Systematic error
Sources of Error Variance:
The extent to which the score is affected by the content
sampled in the test and by the way the content is
sampled (Item sampling or Content sampling)
Test Construction
Sources of Error Variance:
– Test environment: Room temperature, level of lighting, amount of ventilation
and noise
– Test taker Variables: Pressing emotional variables, physical discomfort, lack of
sleep, and effect of drugs or medications
– Examiner-related variables: Examiner’s physical appearance and demeanor
Test Administration
Sources of Error Variance:
– NOT ALL test can be scored by computer
– Scorers and scoring system are potential source of error
variance
– If subjectivity is involved in scoring, then the rater can be a
source of error variance
– Subjectivity in scoring can even enter in behavioral
assessment
Test Scoring and Interpretation
Reliability Estimates:
– Using the same instrument to measure the same thing as
two points in time
– The results of evaluation is called test-retest reliability
– 1 group; 2 different administration
– Measure something that is relatively stable over time
such as personality
Test-Retest Reliability Estimates/ Time Sampling
Method
interval between testing is 6
months
Coefficient of Stability
most appropriate in reaction time
and perceptual judgment
Test-Retest Reliability
Reliability Estimates:
Each form of the test, the means and the variances of
observed test scores are equal
Parallel Forms
Reliability Estimates:
– Simply different versions of a test that have been
constructed so as to be parallel
– Applicable on relatively stable traits
Alternate Forms
