Lecture 4 - Concepts, Operationalization, and Measurement Flashcards
Validity & 4 Types
Validity: Are statements about a given relationship true (valid) or false (invalid)?
4 Types:
Conclusion Validity:
* Are 2 variables statistically related
- Lack of conclusion validity = Bias
- Highly related to sample size (aka power)
Internal Validity:
* Extent to which you can draw conclusions about casual effects of one variable on another
- Is there an error? Confounding variable?
External Validity:
* Extent to which researchers findings are applicable to other populations and settings
- aka Generalizability
Construct Validity:
* Extent to which an observed relationship between variables represents the causal process
Ex. Are patient death rates indicative of a doctor’s skill?
Statistical Power
How likely we can find a statistically significant difference
* Bigger sample size = more power
Importance of Validity
Key measure of how informative a study is
* i.e. how biased are conclusions
Relative importance of each validity type depends on study purpose
Conceptions vs. Concept/Construct
Conceptions: Mental images we have of something
Concept/Construct: Words, phrases, or symbols that represent these mental images in communication
Ex. If conception is “outdoors”, then concept would be things like “grass” and “sky”
Conceptualization
Conceptualization: Process of specifying precisely what we mean when we use particular terms
3 components:
* Construct
* Dimensions and Indicators
* Conceptual Definition
Dimensions:
* Specifiable aspects of a construct (sub categories)
- Can be 1 dimension or multiple
Indicators:
* Indicate the presence or absence of a dimension
Ex. If a dimension is Degree of Harm, indicator would be a scale from 1 to 10
Conceptual Definition:
* Working definition assigned to a construct
* Allows researchers to agree for study
* Does not directly produce observations
Conceptualization Process:
* Conception - Mental images
* Concept/Construct - Words, symbols to represent conception
* Dimension - Specific aspects of a concept
* Indicator - Specifies presence or absence of indicators
* Conceptual definition - working definition of a concept/construct
Example of Conceptualization process:
* Conception - Anger
* Concept/Construct- Low Self control
* Dimension - Immediate gratification, lack of concern
* Indicator - Present = Yes, Not present = No
* Conceptual definition - Low self-control is a
personality trait comprised of a tendency for a lack of tolerance of frustrations, preference for activities providing immediate and exciting gratification, and a
tendency to disregard others’ well-being
Operationalization
Operationalization: Process of specifying the operations necessary for measuring constructs
* Specifying the variables to represent the construct (identifying attributes or variables)
Conceptual Hypothesis vs. Operational Hypothesis
Conceptual Hypothesis:
* Student effort -> Academic Performance
Operational Hypothesis:
* # of hours studied -> Exam grade
* # of times participated in class -> Course Grade
Operational Definitions
- What we will observe
- How we will observe it
- What interpretations we will make
Variable Identification for Low self-control example
Example of Conceptualization process:
* Conception - Anger
* Concept/Construct- Low Self control
* Dimension - Immediate gratification, lack of concern
* Indicator - Present = Yes, Not present = No
* Conceptual definition - Low self-control is a
personality trait comprised of a tendency for a lack of tolerance of frustrations, preference for activities providing immediate and exciting gratification, and a
tendency to disregard others’ well-being
Variables:
* No concerns for future
* Risk-taking
* Problems controlling behavior
Attributes must be exhaustive
Must be able to classify every observation in terms of 1 of the attributes
Exhaustive: All possible cases or values of a variable must be accounted for
Ex. of non-exhaustive variable:
Is your family income:
* Less than 25k
* Between 25k and 75k
* Between 75k and 100k
The problem:
* Non-exhaustive because it does not have an option for a family income over 100k
Attributes must be mutually exclusive
Must be able to classify every observation in only 1 of the attributes
Ex. of Non-mutually exclusive variable:
How old are you?
* Under 18
* 18-30
* 30-45
* 45 or above
Problem:
* If you are 30, you belong in 2 of the categories. Therefore, it is not mutually exclusive
4 Levels of Measurement
Nominal Measures
* Named categories with no clear organization or hierarchy
Ex. Place of birth, Gender identity, Ethnicity, etc.
Ordinal Measures
* Attributes are rank ordered on a continuum, but no specified numerical difference between values
Ex. Scale ranging from strongly disagree to strongly agree
Interval Measures
* ordered categories with equal difference between values
- arbitrary zero point (e.g. temperature, IQ)
Ratio Measures
* Ordered categories with equal distance between values and true zero point
- Zero point represents an absence
- e.g. age, # of crimes, time spent in prison
Example: Operational definitions for age using all 4 levels of measurement
Nominal: age categorized into non-ordered groups
* “Child”, “Teen”, “Adult”, “Senior”
Ordinal: Age is categized into ordered groups, but unequal numerical differences
* Age ranges:
* 0-18
* 19-35
* 36-60
* 61+
Interval: Age is measured with equal intervals but no true zero point
* Born in 1985
* Born in 1990
* Born in 1995
* Born in 2000
Ratio: Age is measured in years with a meaningful zero point
* 0 years old
* 25 years old
* 50 years old
* 75 years old
Easy way of thinking of 4 Levels of Measurement
- Nominal - Categorical, No Order
- Ordinal - Categorical, Ordered, Unequal Intervals
- Interval - Numerical, Ordered, Equal intervals, No true zero
- Ratio - Numerical, Ordered, Equal Intervals, True Zero Exists
Key difference:
- If you can name it, but not rank it, then nominal
- If u can put in order, but difference is unequal, then Ordinal
- If there’s no true zero, it’s interval
- If “twice as much makes sense”, it’s ratio
Practice examples
📅 Birth years of students in a class (e.g., 1995, 2000, 2005)
🎬 Movie genres (Action, Comedy, Horror, Drama)
🏆 Olympic medal types (Gold, Silver, Bronze)
🌡️ Temperature in Fahrenheit (e.g., 32°F, 100°F)
🏅 Runners’ finishing positions in a race (1st place, 2nd place, 3rd place)
💰 Monthly salary in dollars (e.g., $3,000, $5,000)
⏳ Time of day on a 12-hour clock (e.g., 3:00 PM, 10:00 AM)
📏 Height in centimeters (e.g., 150 cm, 180 cm)
🎓 Education level (High school, Bachelor’s, Master’s, PhD)
🚗 Car brands (Toyota, Ford, Honda, BMW)
Answer key:
- Interval (Years are evenly spaced, but “year 0” is arbitrary)
- Nominal (Categories with no order)
- Ordinal (Ranks exist, but the difference between ranks isn’t equal)
- Interval (Equal intervals, but no true zero—0°F doesn’t mean “no temperature”)
- Ordinal (Ordered, but the gaps aren’t necessarily equal)
- Ratio (True zero exists, and “twice as much” makes sense)
- Interval (Equal intervals, but 0:00 AM doesn’t mean “no time”)
- Ratio (Height has a true zero, and “twice as tall” makes sense)
- Ordinal (Ordered, but differences between levels aren’t equal)
- Nominal (Just labels, no ranking)
Measurement Quality
2 standards of quality:
* Validity
* Reliability
Validity: Are statements about a given relationship true (valid) or false (invalid)?
Reliability: If a measurement technique is applied repeatedly, will it yield the same result each time?
Reliability Methods
Test-retest reliability:
* Making some measurement more than once
- If info is some, but responses vary, then unreliable
Inter-rater reliability:
* Compare coders to reduce inconsistencies
- Adjust coding if not the same
Coders: individuals who analyze and categorize qualitative or quantitative data based on a set of coding rules or guidelines.
Factors to Increase Reliabilty
- More questions
- Clear Instructions (Ex. drugs might have a different meaning depending on who you ask - Cocaine vs. Alcohol)
- No Distractions
Validity Testing
Face validity
* Does the measure appear appropriate?
Construct Validity
* Are the measures and constructs logically related?
Criterion Validity
* Are scores on measures comparable to external, established measure?
Content Validity
* How well does a measure cover the range of meanings in the construct?
Multiple measures
* Are scores on measures comparable with additional measures of the same construct?
