3: Psychological Measurement Flashcards
Construct
A concept which cannot be directly observed and or which there is not one single reference (abstract, hypothetical)
- ie. personality traits, emotional states, attitudes, abilities
- often represent tendencies to feel or behave certain ways or involve internal processes
Conceptual Definitions
Describe the behaviours and internal processes that make up a construct, along with its relation to other variables.
- conc. defn. for one construct may vary for one construct
Operational Definition
Definition of a variable in terms of precisely how it is to be measured; can fall into one of three broad categories:
- self-report measures
- behavioural measures
- physiological measures
Reducing Impact of Inconsistency
- Measure more than once and use average
- Measure using more than one method
- Use the best method possible to maximize reliability
Reliability
Refers to the consistency of a measure.
Three Types:
1. Over time; test-retest (testing a group at one time, then retesting the same group at a later time)
2. Across items; internal consistency (consistency of people’s responses across the items on a multiple-item measure)
3. Across researchers; inter-rater reliability
Measurement
The assignment of scores to individuals so that the scores represent some characteristic of the individuals.
- psychological measurement is often referred to as psychometrics
Self-report Measures
Participants report on their own thoughts, feelings, and actions.
-ie. Rosenberg Self-Esteem Scale
Behavioural Measures
Some other aspect of participants’ behaviour is observed and recorded.
ie. Bandura’s bobo doll expiriment (aggression measured based on the amount of specifically defined acts of violence the children imposed on the doll in a 20-minute period)
Physiological Measures
Involve resording any of a wide variety of physiological processes, including heart rate, blood pressure, galvanic skin response, hormone levels, blood flow to brain, etc.
Converging Operations
Various operational definitions are coming together on the same construct.
- allows for more useful general conclusions to be drawn (ie. stress is negatively correlated with immune system functioning vs. people’s scores on the Perceived Stress Scale are negatively correlated with their white blood counts)
Interval Level
Assigning scores using numerical scales in which intervals have the same interpretation throughout.
ie. difference between 30 and 40 degrees is equal to that between 80 and 90
Issue: do not have a true, unarbitrary zero point
Ratio Level
Assigning scores in a way that there is a true zero point representing a complete absence of quantity.
ie. height measured in meters; kelvin scale (as opposed to celsius)
Nominal Level
Used for categorical variables; involves assigning scores which are category labels.
ie. marital status
- do not imply order among responses
Levels of Measurement
Four types of information that can be communicated by a set of scores and their appropriate cooresponding statistical procedures.
1. Nominal
2. Ordinal
3. Interval
4. Ratio
Ordinal Level
Assigning scores representing the rank order of individuals.
ie. assigning numerical scores 1-5 representing states from very dissatisfied to very satisfied
Test-retest Reliability
When researchers measure a construct that they assume to be consistent across time, then the scores they obtain should also be consistent across time.
-test-retest coorelation or .80 + is indicative of good reliability
Internal Consistency
The consistency of people’s responses across the items on a multiple-item measure.
ie. people who agree that they are a person of worth should tend to agree that they have a number of good qualities
Split-half Correlation
A score that is derived by splitting the items into two sets and examining the relationship between the two sets of scores in order to assess the internal consistency of a measure.
- split-half correlation of +.80 + is generally considered good internal consistency
Cronbach’s α
The mean of all possible split-half correlations for a set of items; measures internal consistency among items in a measure.
Inter-rater Reliability
The extent to which different observers are consistent in their judgments.
- assessed using Cronbach’s α when the judgments are quantitative or an analogous statistic called Cohen’s κ (the Greek letter kappa) when they are categorical
Validity
The extent to which the scores from a measure represent the variable they are intended to.
1. Face
2. Content
3. Criterion
Face Validity
The extent to which a measurement method appears, on superficial examination, to measure the construct of interest.
Content Validity
The extent to which a measure reflects all aspects of the construct of interest.
ie. if a researcher conceptually defines test anxiety as involving both sympathetic nervous system activation (leading to nervous feelings) and negative thoughts, then his measure of test anxiety should include items about both nervous feelings and negative thoughts
Criterion Validity
The extent to which people’s scores on a measure are correlated with other variables (known as criteria) that one would expect them to be correlated with.
Criterion
A variable that theoretically should be correlated with the construct being measured (plural: criteria).
ie. one would expect test anxiety scores to be negatively correlated with exam performance and course grades and positively correlated with general anxiety and with blood pressure during an exam.
Concurrent Validiity
A form of criterion validity, where the criterion is measured at the same time (concurrently) as the construct.
ie. validity of results obtained from one test (e.g., self-report of job performance) as assessed by comparison with a separate but related measurement (e.g., supervisor rating of job performance) collected at the same point in time.
Predictive Validity
A form of validity whereby the criterion is measured at some point in the future (after the construct has been measured), to determine that the construct “predicts” the criterion.
ie. predictive validity of a test designed to predict the onset of a disease would be strong if high test scores were associated with individuals who later developed that disease
Convergent Validity
A form of criterion validity whereby new measures are correlated with existing established measures of the same construct.
- does measure correlate with what it should?
Discriminant Validity
The extent to which scores on a measure of a construct are not correlated with measures of other, conceptually distinct, constructs and thus discriminate between them.
ie. If a measure of self-esteem were highly correlated with a measure of mood, it could be argued that the new measure is not really measuring self-esteem; it is measuring mood instead.
Types of Criterion Validity
- Concurrent
- Predictive:
- Convergent:
- Discriminant:
Developing Measure
- clarifying construct and dimensions
- literature review
- focus group/ qualitative study/ existing data
- conceptual definitions/ appropriate labels
- content validity - develop items (questions) to represent each dimesion
Qualities of a Good Item
- reflects construct of interest
- unidimensional
- clear and unambiguous
Item Formats
- Dichotomous (YN, T/F, (in)correct)
- Likert-type (5-7 balanced responses: strongly (dis)agree)
- Visual analog (line, dial)
- Select from list (mutually exclusive and exhaustive)
Making Items
- include useful redundency; increases reliability
- test drive with experts on relevance, clarity and conciseness, aspects missing as well as participants
Simple Random Sampling
Each member of a population has an equal & independent chance of being sampled (e.g., names in hat)
Systematic Sampling (probability sample)
Random start, then researchers select members of the population at a regular interval.
(ie. selecting every 15th person on a list)
risk: ordered list
Stratified Random Sampling (probability sample)
Random sampling within specified subgroups/strata based on characteristics that they share, then ramdonly sampling each group using another method.
Cluster (/multistage) Sampling (probability sample)
Dividing a population into clusters, then randomly selecting some of these clusters
Probabiity Sampling Issues
-Sample Representativeness: cannot assume results generalize to a population
-Response Rate
Non-probability Sampling Methods
-convenience (availability)
-purposive (focus on particular subsets): snowball (participants suggest others) or quota (set proportions of particular groups)
