Chapter 5: Measurement Concepts Flashcards
A Likert scale is an example of which type of scale of measurement?
Ordinal
Which correlation is the strongest? -.87 \+.86 \+.30 .00
-.87
The correlation of scores from one test administration with another administration
of the same test at a later date would be an example of which type of reliability?
Test-retest
A test developer creates a new instrument to measure depression. He correlates this with an existing assessment that measures self-esteem. The test developer hopes not to find a high correlation to support the new instrument’s validity. Which type of validity is this?
Discriminant
T/F: In order for a test to be valid, it must be reliable.
True
What is a nominal scale?
Naming or classifying only
Does not possess magnitude, equal internals or an absolute zero
What is an ordinal scale?
Order or rank of nominal categories Magnitude only
What is an interval scale?
Equal distance among ranks or intervals; magnitude, equal intervals
What is a ratio scale?
Possesses magnitude, equal intervals, absolute zero
What are the key concepts related to reliability?
Measurement error
Correlation
Coefficient of determination
What is reliability?
How consistently a test measures a variable
The extent to which it eliminates chance and other extraneous factors in its results
Explain Measurement Error
Positive or negative bias within an observed score
True score can never be known
Individual error, test error, testing condition error
What is the formula to show how error impacts scores?
observed score (X)=true score (T) + error score (e)
What is correlation?
The degree to which two sets of measures are related
What is reliability in regard to correlation?
Reliability is the relationship of one score with another actual or hypothetical test and an estimate of the test scores, not the test
Does correlation indicate causation?
No
What are the types of reliability?
Test-retest
Alternate-forms/parallel-forms
Internal consistency (split-half reliability, interitem consistency)
Interrater
What is test-retest reliability?
Same individuals, two administrations
What is alternate-forms reliability?
Consistency of scores of individuals within the same group on two alternate by equivalent forms of the same test
What is internal consistency reliability?
Consistency of responses to the different items or parts of a test during a single test and administration
What are two common measures of internal consistency?
Split-half reliability
Interitem consistency
What is interrater reliability?
The degree of agreement between two or more independent judges
How is interrater reliability calculated?
By dividing the number of agreements that an event occurred by the number of possible agreements
What is validity?
The extent to which meaningful and appropriate inferences can be made
What are the two types of invalidity?
Construct underrepresentation
Construct irrelevant variance
What are the types of validity?
Face validity (not evidence of validity)
Content validity
Criterion-related validity (concurrent/predictive)
Construct validity (convergent/discriminant)
Treatment validity
What is content validity?
Representativeness of items from a “population” of items
What is criterion-related validity?
Calculated by comparing test scores with performance on a criterion measure.
Can be represented or not by a test score
What is concurrent validity?
Test scores and criterion performance scores are collected at the same time
What is predictive validity?
Criterion performance scores of interest are collected at a later time
What are the base rates of predictive validity?
The proportion of people in a population who represent the particular characteristic or behavior that is being predicted
What is incremental validity?
The extent to which a particular assessment adds to the accuracy of predictions obtained from other tests or other less extensive methods of assessment
What are the predictive validity concepts?
False negative- predict falsely
False positive- fail to predict
Sensitivity- the accuracy of a cutoff score in detecting those people who belong in a particular category
Specificity- the accuracy of a cutoff score in excluding those without that condition
What is construct validity?
Explaining the psychological meaning of the variable (construct) measured by the test- factor analysis
What is convergent validity?
Test correlates positively and strongly with tools it should
What is discriminant validity?
Test correlates negatively and strongly with tools it should
What are the steps in assessment development?
- Test items are written according to the objectives and purpose of the test
- Items are then checked for cultural bias and other concerns
- Items are then piloted
- Tests of reliability and validity are done
- A scoring system is developed
- Norms are established
Gender, race, political affiliation, mode in a data distribution, or presence or absence of a diagnostic criterion are examples of what scale?
Nominal Scale
Degree of job satisfaction, university national rankings, and median values or percentile ranks of a data distribution are examples of what scale?
Ordinal Scale
Temperature, checklist of behaviors, and standard deviation are examples of what scale?
Interval Scale
What is the simplest scale?
Nominal Scale
What is the most advanced and precise measurement scale?
Ratio Scale
Time and height are examples of what scale?
Ratio Scale
Can no measurement error exist or possible?
No
What can be calculated to provide more information about the shared variance between two variables?
Coefficient of determination
