Chapter 5 Cath Flashcards
Give a basic Definition of Reliability in Psychometrics
Consistency in Measurement. (Not necessarily good or bad).
How does Context influence reliability?
A test may be reliable in one context, but not another
What is a Reliability Coefficient?
A reliability coefficient is an index of reliability, a proportion that indicates the ratio between the true score variance on a test and the total variance.
How can we represent the concept of reliability in an equation?
X = T + E
Where X is the score
T is the true score
E is error
What is the Variance?
Variance is the Standard Deviation Squared and can be broken down into components: total variance, true variance & error variance.
What is True Variance?
Variance from true difference is true variance. True differences are assumed to be stable, to yeild consistent scores on repeated administrations of the same test as well as on equivalent forms of test
What is Error Variance?
Error Variance is Variance from irrelevant, random sources.. BEcause error variance may increase or decrease a test score by varying amounts, consistency of the test score, and thus reliability, can be affected.
How does a systematic source of error influence variability?
A Systematic source of error would not effect consistency, it does not change the variability of the distribution or affect reliability.
List the possible sources of Error Variance
The possible sources of error variance are: Test construction, Test Administration, Test Scoring & Interpretation, Other sources of error also exist.
How can test construction influence error variance?
Test construction influences the extent to which a testtaker’s score is affected by the content sampled on the test and by the way in which the item is constructed.
What is item sampling or content sampling in relation to error variance?
Item sampling or Content Sampling refers to the variation among items within a test as well as to variation among items between tests.
What is one of the key challenges faced by test developers in relation to test construction?
A challenge in test development is to maximise the proportion of the total variance that is true variance and to minimise the proportion of the total variance that is error variance.
How can Test Administration influence Error Variance?
Sources of error variance that occur during test administration may influence the testtaker’s attention or motivation.
What are some examples of factors that can have untoward influence during test administration?
Factors that can influence test administration include: Test Environment (i.e. room temperature, lighting, ventilation noise), Testtaker variables (emotional problems, physical discomfort, drug/medication effects, lack of sleep); Examiner-related variables (physical appearance & demeanor, presence or absence of examiner, departure from the prescribed procedure, providing non--verbal cues in oral exams).
How can Test Scoring & Interpretation become a source of error variance?
If subjectivity is involved in scoring (e.g. in the case of nonobjective-type personality tests, tests of creativity, essay tests), then the scorer can be a score of error variance.
