Chapter_6_Developing a Measurement Strategy Flashcards
Week 5
Manifest Variables
Variables that can be directly observed
- the physical characteristics of people and objects, behavior, physiological responses, answers to questions
Hypothetical Constructs
Cannot be directly observed
Manifest Variables & Hypothetical Constructs Relationship
Manifest variables as OPERATIONAL DEFINITIONS of hypothetical constructs
- Hypothetical construct CAUSES the presence and strength of the manifest variable
- REFLECT, albeit imperfectly
- INFER the strength of the hypothetical construct (such as a person’s level of hostility) from the strength of the manifest variable
Measurement Error
The difference between true scores and observed scores
- Random error
- Systematic error
Reliability
Degree of consistency
- Reliable = low random error
Validity
Degree of accuracy
- Valid = low systematic error (and should not have high random error)
- High reliability does not guarantee high validity
Random Error
Fluctuates each time a measurement is made
- leads to instability of measurement and lower reliability estimates
- can result from sources
Systematic (Nonrandom) Error
is present every time a measurement is made.
- As a result, the observed score is stable, but inaccurate as an indicator of the true score
- can result from sources or a test
Three Types of Reliability
- Test-Retest Reliability
- Alternative Forms Reliability
- Internal Consistency
Test-Retest Reliability
- Consistency of a measure over time by administering the same test twice to the same individuals
- Assesses stability over time; susceptible to changes in the variable being measured
- Correlation
Alternative Forms Reliability
- Consistency between different versions of the same test that are designed to be equivalent
- Ensures equivalent test versions yield similar results; accounts for test practice effects
- Correlation
Internal Consistency
- Consistency among items within a measure by examining how well items correlate with one another.
- Focuses on item homogeneity within a single measure; does not assess stability over time
- Cronbach’s alpha or split-half correlation
Cronbach’s alpha
A function of the MEAN CORRELATION of all the items with one another and can be interpreted like a correlation coefficient.
- Because they assess the degree to which responses to the items on a measure are similar
Limitations of Cronbach’s alpha
- A high alpha value does not mean that a measure is unidimensional
- Item synonyms -> artificially high
- Items increases -> alpha increases
- Alpha assumes tau-equivalence (i.e., equal factor loadings).
Classical Test Theory (CTT)
Focuses on reliability of overall test scores
- each item measure one trait
Item Response Theory (IRT)
Focuses on the reliability of individual items on a test
- items are thought of as falling at different positions along the underlying [trait] continuum
- does not assume that all test items are equally reliable.
6 Important Characteristics of Validity
- is inferred from available EVIDENCE (not directly measured)
- depends on many DIFFERENT types of evidence
- is expressed by DEGREE (high, moderate, low)
- NOT “valid” or “invalid” - is SPECIFIC to a particular use
- refers to the INFERENCE drawn from test scores, not the test itself
- is a UNITARY concept
4 Categories of Validity according to Standards
- Internal structure
- Test content
- Relations to other measures
- Response processes
Traditional Typology of Validity
- Content-related
- Construct-related
- Criterion-related
Process to validate a newly developed psychological measurement
- Definition of construct
- item generation
- content and structural validity
- reliability - Relationship of the construct with others
- hypothesis regarding the relationships
- select criterion measures
Modes of Measurement
- Self-report measures
- Cognitive / Affective / Behavioral / Kinesthetic - Behavioral measures
- Operational definition / nonverbal cues - Physiological measures
- neural activity / ANS activity / hormone levels
- operational definition of psychological states - Implicit measures
Strengths of Self-report Measures
- The most direct way to obtain some kinds of information
- can be easy to collect
Limitations of Self-report Measures
- Accuracy
- Factors influencing behaviors or cognitive processes
- not be Willing to
- Social desirability
- Verbal skills
Strengths of Behavioral Measures
- unaware of observation
- performed automatically
