Ch. 5 - Measurement Concepts Flashcards
What is the primary criteria that researchers use to evaluate the quality of their own and others’ research?
Reliability and validity
Reliability
- Refers to the CONSISTENCY or stability of a measure of behaviour
- Increasing the reliability of a measure reduces uncertainty or error associated with that measure
True score
-Is the person’s real score on the variable
Measurement error
-The degree to which a measurement deviates from the true score value
Pearson product-moment correlation coefficient
- Symbolized as “r”
- The Pearson correlation coefficient can range from 0.00 to +1.00 and 0.00 to -1.00.
- A correlation of 0 tells us that the two variables are not related at all
- The closer a correlation is to either +1 or -1, the stronger is the relationship
- The positive and negative signs provide information about the direction of the relationship
Test-retest reliability
- Is assessed by giving many people the same measure twice
- A reliability coefficient determined by the correlation between scores on a measure given at one time with scores on the same measure given at a later time
Alternate forms reliability
- Is sometimes used to avoid artificially high correlation coefficients if people taking the test a second time, remember it from the first time
- Alternate forms reliability involves administering two different forms of the same test to the same people at two points in time
Internal consistency reliability
- Assesses how well a certain set of items relate to each other
- One common indicator of internal consistency is a value called Cronbach’s alpha
Cronbach’s alpha
-An indicator of internal consistency reliability assessed by examining the average correlation of each item (question) in a measure with every other question
Interrater reliability
- Is the extent to which raters agree in their observations
- A commonly used indicator of interrater reliability is called Cohen’s kappa
Construct validity
-The degree to which a measurement device accurately measures the theoretical construct it is designed to measure
Indicators of construct validity?
- Face validity
- Content validity
- Predictive validity
- Concurrent validity
- Convergent validity
- Discriminant validity
Face validity
- The content of the measure appears to reflect the construct being measured
- Face validity is not sufficient to conclude that a measure has construct validity
Content validity
- Is based on comparing the content of the measure with the theoretical definition of the construct
- The content of the measure captures all the necessary aspects of the construct and nothing more
Predictive validity
-The construct validity is assessed by examining the ability of the measure to predict a future behaviour
Concurrent validity
-Is assessed by research that examines the relationship between the measure and a criterion behaviour at the same time (ie. concurrently)
Convergent validity
-Is the extent to which scores on the target measure in a question are related to scores on other measures of the same construct or similar constructs
Discriminant validity
-When the measure is not related to variables with which it should not be related, discriminant validity is demonstrated
Reactivity
- A problem of measurement in which the measure changes the behaviour being observed
- Awareness of being measured changes an individual’s behaviour
What are the 4 measurement scales that a variables levels can be conceptualized in terms of?
- Nominal
- Ordinal
- Interval
- Ratio
Nominal scale
- Categories with no numeric scales or quantitative properties
- Are sometimes called “categorical variables”
- ex. males/females, experimental condition/control condition, major of undergraduates
- Even if you were to assign numbers to the different categories, the numbers would be meaningless, except for identification
- Impossible to define any quantitative values and/or differences between/across categories
Ordinal scale
- Rank ordering
- Numeric values have limited meaning
- Instead of having categories that are simply different, as in a nominal scale, the categories can be ordered from first to last
- ex. 2,3,and 4 star restaurants, birth order
- No particular value is attached to the intervals between the numbers used in ordinal scales
Interval scale
- Numeric properties are literal
- Assume equal interval between values
- ex. intelligence, temperature
- No true zero, the zero on any interval scale is only an arbitrary reference point
Ratio scale
- Does have an absolute zero point that indicates absence of variable measured
- Assume equal interval between values
- ex. reaction time, age, frequencies of behaviour
- Can form ratios, ex. someone responds twice as fast as another person
