PSYC - Ch 3 + 6.2 Flashcards
Why measure?
comparison
classification
prediction
program evaluation
decision-making
diagnosis
Variables can be
Directly observable or inferred states
Constructs
A hypothetical mechanism that helps explain and predict behaviour.
External stimuli -> Construct -> External behaviours
Presumed unobservable internal mechanisms that account for externally observed behaviour.
Rewards -> Motivation -> Performance
Exam -> Anxiety -> Affects behaviour
The construct cannot be studied directly, but the stimuli and behaviours which are influenced by the construct can be studied.
Change in environment leads to observable behaviours
ie: anxiety, self-esteem, motivation, aggression, intelligence.
Brings from Abstract to Concrete
Unobservable (stress) to observable (HR, persipration,)
A variable that is a hypothetical entity created from theory and speculation. They are influenced by external stimuli and influence external behaviour
Step 3 of the Research Process
- determining a method for defining and measuring the variables that are being studied
Operational definition
Precise description of what you will measure, how you will measure it, and when you will measure it (testing protocol). Defines how you will measure the construct. (Ie. sores on an anxiety test)
Defines the “operations” allowing us to link unobservable with observable
Converts abstract to concrete/measured
Needs to be - clear + precise - allows for replication
EX: the operational definition of anxiety could be in terms of a test score, withdrawal from a situation, or activation of the sympathetic nervous system.
ODs must be VALID and RELIABLE
Validity
Accuracy of the measure
Will the test measure what it is supposed to measure?
Reliability
Consistency of the measure over repeated applications in same conditions. (Reproducability)
Measurement always varries a little but should be as little as possible.
(Dots close to each other or close to the line on a line graph)
Types of Validity - in regards to a measurement
Face validity
Predictive validity
Concurrent validity
Construct validity
Convergent & divergent
Types of Validity - in regards to the study as a whole
Internal validity
External validity
Face Validity
Extent to which a measurement appears to be a plausible measure of the variable at first glance.
Does it make sense? Would others agree?
Ex: Measuring anxiety through observation at a party or through # of toes.
Predictive Validity
The strength of the relationship between 2 variables (correlation)
Can you use one to predict the other?
Intelligence - GPA
Neuroticism - Stress
Concurrent Validity
Extent to which a measure relates to existing measures of the same thing (construct).
The new (yours) and old test (IQ test) have to produce similar results/have a high correlation or else the measure may have low validity
Construct Validity
Extent to which scores obtained from a measure behave exactly the same as the variable itself (e.g., aggression & temperature).
The extent to which your test or measure accurately assesses what it’s supposed to
It is based on many studies that use the same measurement procedure. It grows as each new study contributes more evidence (same measurement with similar results).
Use multiple measures in your study, to help establish construct validity.
Convergent & Divergent Validity
The scores of some of the measure will converge - measuring similar or related constructs
The scores of some of the measures will diverge - measuring unrelated constructs
Ex: Aggression is children - is it high energy or aggression
Measure of aggression should converge (correlate highly).
Behavioral measure of energy and the above measures should diverge (have a low correlation).
Internal Validity
Can you safely say the changes in X have caused the changes in Y.
Depends on the control of other variables/confounding variables.
4 week yoga retreat decreases stress. Was it the yoga? The lack of work? The sun and beach?
External Validity
Extent to which your results can generalize to other settings and populations. (Generalizable)
If effects remain with different groups = good external validity
Types of Reliability
Test-retest reliability
Inter-rater reliability
Split-half reliability
Test-Retest Reliability
Repeat same measurement twice with the same individuals and calculate the correlation between scores - reliability coefficient
If the scores from Time 1 and Time 2 have a high correlation you have test-retest reliability.
Inter-Rater Reliability
Compare the scores from the two raters and calculate correlation.
If results from rater 1 are similar to rater 2 - you have inter-rater reliability.
Split-Half Reliability
Typically, for clinical scales and questionnaires.
Take the scores from half the items and correlate them with the scores
from the remaining half - they should correlate highly.
Good internal consistency if high correlation.
Measurement Error
Observed scores may not be a true reflection of the variable or construct being measured - There is always some degree of error present.
Observed score = measurement + error
Sources of Error
The participant - mood, motivation, fatigue, health, memory, knowledge, ability
The instrumentation / apparatus - sensitivity, clarity of instructions, appropriateness, length, vocabulary
The testing environment. - comfort (heat), presence of others (social facilitation), distractions (noise, interruptions)
Scoring guidelines - clarity, complexity, experience requires, individual differences
Reducing Error
Try to minimize the effects of possible confounds.
STANDARDIZATION
Standardization
Participants - what will your inclusion and exclusion criteria will be?
ie: age, gender, educ level, health, ethnicity
Test protocol - no variation, must be consistent
ie: instructions, treatment of participants, order of tests
Environment - best environment conducive to testing and replicate is as best possible
ie: time of day/year, temperature, noise level
Scoring procedure - be as objective as possible, marking criteria should be as clear as possible for both participants and rater.
ie: allow practice rounds for both rater and participants
Reliability coefficient
It reflects the degree to which the measurement is free of error variance.
You can never completely eliminate error, therefore you must account for it.
Reliability coefficient - ratio of true score variance to observed score variance
r = Score true/Score observed
= (score observed - score error)/score observed
Relationship Between Validity & Reliability
A measurement procedure must be reliable (consistent) in order to be valid.
A measurement procedure can be reliable & not necessarily valid.
Can be reliable but not valid.
Cannot be valid if not reliable.
Types of Measurement
Quantitative
Qualitative
Quantitative Measures
Assigning a numerical value to a variable
Nominal Scale
Ordinal Scale
Interval Scale
Ratio Scale
Nominal Scale
Used fo categorization
No inherent order
Gender, Education
Ordinal Scale
Used for ranking
Has order
Height - short/tall
Strength - weak/strong
Interval scale
Has order + magnitude + equal intervals between values but NO TRUE ZERO
Temperature
Ratio Scale
Has order + magnitude + equal intervals between values WITH A TRUE ZERO
Distance, length, reaction time, HR
Modalities of Measurement
Self-report - direct, subjective, social desirability bias (not honest)
Physiological - objective, invasive, costly, time consuming
Behavioural - interpretation, clusters better
Multiple measures
Good - more confidence in the validity
Bad - can require complex statistical procedures & interpretation can be challenging.
Sensitivity & Range Effects
Range effect - measurement is not sensitive enough to detect a difference.
Ceiling effect - clustering of scores at high end of scale; little possibility of increases in scores.
Floor effect - clustering of scores at low end of scale; little possibility of decreases in scores.
Artifacts
An external factor introduced to the study accidentally that may influence or distort measurements.
Experimenter bias
Measurements influenced by experimenter’s expectations regarding the outcome of the study.
Participant reactivity
Participants changing their natural behaviour because they are participating in a research study.
Good subject role: support the hypothesis.
Negativistic subject role: acts contrary to the hypothesis (sabotage).
Apprehensive subject role: act/answer in socially desirable manner (fake good).
Faithful subject role: follow instructions to the letter (highly desirable).
*You want these subjects
How to limit Experimenter Bias
Standardize or automate the experiment
Single-blind - the researcher is not aware of the expected results.
Double-blind - neither the participant nor the researcher know the expected results.
How to limit Participant Reactivity
Disguise or conceal the measurement process (low face validity, deception, blind)
Reassure participants (performance/responses are confidential and anonymous)
Demand Characteristics
Any cues or feature of the experiment that:
- suggest the purpose and hypothesis of the study.
- influence the participants to respond or behave in a certain way.
Can lead to reactivity, whereby participants modify their natural behavior knowing they are in a study.
