Midterm #2 - Qualitative Research Design Flashcards
Main philosophical worldview that uses qualitative research designs
Post-positivism
What is Post-positivism?
Belief that there is 1 reality, 1 objective truth that is waiting to be discovered through research
Key defining features of post-positivism
DETERMINISM - Causes determine effects (use of experiments)
Reductionism = reduce ideas to small, discrete,
testable variables (makes hypotheses and
research questions)
Empirical observation and measurement = knowledge based on observation and measurement
Theory testing = theories are tested, supported, refined
Difference between Determinism and Causality
Determinism is a philosophical doctrine, and causality is a principle under it
Determinism
Doctrine that assumes every even has causes (nothing happens on it’s own)
- Makes a causal claim
- Post-positivist relation = we find and test for the cause
- no such thing as an “Accident”, implies no cause (injury prevention doesn’t like this, wants to anticipate and prevent)
Causality
Principle that everything has a cause
- Effects can have multiple causes
What do we need to claim causation?
1) Covariation
2) Isolation of causal variable
3) Effects come after causes (order)
4) Manipulating the cause will manipulate the effect
Covariation
Statistical correlation
correlation =/= causation !
Isolation of causal variable
Only x -> y. No z!
Cause is not because of a third variable
z a.k.a extraneous/control variable
3 Goals of Science
Describe - Descriptive (what) goals only cover CORRELATION
Explain - explanatory (how) goals
Predict - predictive (why) goals
- Both include causation
3 Types of claims
Causal (affects, leads to, causes, leads to, etc.)
Association (associated , relates, linked, correlates, etc.)
Frequency (counts, x # of, x% of, x-y/day, just giving the ‘what’, etc.)
Frequency claim
Describes rates or degrees of ONE measured variable
Association claim
At least 2 measured variables
- Correlated somehow (+, -, or 0)
- No manipulation or interference (takes away #4 of causal claim requirements)
- Make predictions based on strength of association (except for 0), stronger closer to 1
Causal claim
Variable x (expected to) changes variable y
- Independent and dependent variables
- Usually experimental design
Mediator variable
variable z that relates/bridges x and y
- Mediation model: X → Z → Y
- If you take Z away, no relationship
NOT the same as a “third variable”
- Interventions operate via mediators, so you NEED to identify it
Moderator variable
Changes sign/strength of the IV’s affect on DV
- Statistical interaction on a graph
- Effect modifiers = creates/enhances/modifies relationship
Confounder variable
Variable that changes relationship between IV and DV because it’s related to both
- Not part of the causal chain
- Source of difference for both variables (ex. population size in church-murder example)
Validity of qual. research designs is composed of
Internal and external validity
Validity of qual. measures are composed of
Content and statistical validity
Internal validity (IV)
Confidence in our results, and that any Δ in outcome was because of the treatment (also addressing threats to validity
- NOT because of a 3rd variable
Threats related to experimental procedure (IV)
Testing: participants just get better at the test, measures skill instead of their true reactions
Instrument Accuracy: Measures must be valid and reliable (working, no calibration error, no misuse, same collection technique)
Threats Related to Treatment or
Manipulation (IV)
Diffusion of treatment: Participants in different groups (esp. control group) talk to each other and adopt treatments (results will be the same b/w both groups)
Halo effect: Researcher bias towards their hypothesis (often not aware, double-blinding fixes this)
Threats Related to the Participants (IV)
Maturation: growing, learning, puberty, natural changes (fix by having a control group also experiencing maturation to see if that’s the only difference)
History: Events other than the treatment impacting study (mental issues, trauma) (fix by having traumatized people in experimental and control groups)
Regression: Extreme scores tend to regress back to the mean (best jump ever, and the next few tests you’re vertical goes down to the mean, this can impact baseline/pretests and skew results)
Threats Related to the Participants (IV)
Selection Bias: No random assignment (even volunteers who come may introduce selection bias)
Experimental Drop-Out (Mortality): People leaving, researchers compare drop-outs to people who complete to see any predictors (ex. drug trials, something about intervention isn’t working)
Placebo Effect: Participants react how they think they’re supposed to (ex. common in drug trials, people think they’re in control)
Hawthorne Effect: Participants may act favourably b/c they’re being watched (fix through control group and double-blind)
Ways to minimize threats to IV
- Control group
- Random Assignment
- Validated Measures (validity of measures)
- Double-blind (participants and researchers don’t know who is receiving which treatment
- Identifying potential confounding factors
External Validity (EV)
How well we can generalize the findings to real world
- Concerns how sample was chosen
Threats to EV
Selection x Treatment Interaction: characteristics of participants (kin students) make treatment (exercise intervention) only work for them (more likely to stick to it), further research needed to generalize
Setting x Treatment Interaction: highly controlled lab environments aren’t well translatable to real world (more “real world” procedures/research needed)
History x Treatment Interaction: timing of intervention is important, the past or future may be different (age, time period) (replicate research in different times with new info + tech to see if it’s still valid) (ex. Korea boat tragedy)
Generalizability
How much findings in study (internal validity) can be generalized to real world (external validity)
Normally high IV = low EV
4 Kinds of Qualitative Research Designs
1) True Experimental Design/Experimental
Design
2) Quasi-Experimental Design
3) Pre-Experimental Design
4) Non-Experimental Design
True Experimental Design
- Identified cause-and-effect relationship (causality)
- Highly controlled lab setting (ensures IV, but less EV)
- At least 1 experimental/treatment/intervention group (EG)
- At least 1 control group (CG)
- Random assignment to remove bias (RA)
RA+CG+manipulates independent variable=true experiment
Two subtypes of True Experimental Design
1) Randomized Control Trial (RCT):
- Pre and post test design - assumes pre test doesn’t impact results (since both groups do it) and both groups are equivalent and randomized
- Post test design only
2) Solomon Four-Group Design - examines effects of pre-test on outcome
What does a pre and post test RCT design look like?
1 independent variable:
R O1 T O2
R O1 O2
> 1 independent variable:
R O1 TX1Ty1 O2
R O1 TX1Ty2 O2
R O1 TX2Ty1 O2
R O1 TX2Ty2 O2
R=randomize
O1=pre test
T=intervention
O2=post test
What does a post-test RCT design look like?
R T O1
R O1
- Best kind of test, but rare b/c difficult
- Shows that a pre-test IS NOT required for a true experiment
What does a Solomon Four-Group Design look like?
R O1 T O2
R O1 O2
R T O2
R O2
Quasi-Experimental Design
Approximate conditions of a true experiment in a setting where we can’t control/manipulate every relevant variable
- No Random Assignment (lowed IV b/c groups aren’t equivalent, less accurate replicas)
- Usually higher EV than true experiment (b/c normally done in a natural environment, better generalization, out in the real world)
- Independent variable still introduced/manipulated
- Dependent variable still measured
Normally done b/c of lack of money, resources, physical condition (ex. pregnancy)
What does a Quasi-Experimental Design look like?
Non-equivalent Control Design (b/c no random assignment R)
O1 T O2
O1 O2
Pre-Experimental Research Design
- No control, but sometimes ‘comparison’ group (non-equivalent group identified during or after intervention)
- Independent variable still introduced and manipulated
- Dependent variable still measured
- Less IV and EV
- No Random assignment
This can be used as a lead-up to actual experiment, seeing if intervention can actually be handled, if it’s very hard to get participants, etc.
Non-Experimental Research Design
- No random assignment (b/c no groups to randomize to)
- No manipulation of IV
Not doing anything, just observing and comparing - Can use pre-existing groups (compared to understand potential effects of differences on dependent variable)
- Frequency claims, correlational, longitudinal designs, descriptive, cross-sectional (compare existing groups at one time like active vs. inactive)
Used in epidemiological research, population health knowledge
Sample vs. Population
A sample is a group of participants in a study that represent a population
A population is a well-defined collection of individuals of interest with similar characteristics (who we want to generalize findings to)
Random vs. Randomize
Random = no pattern or regularity
Randomize = to select subjects without detectible biases or patterns (allows generalization to population)
Probability vs. Chance
Probability = quantifiable likelihood that something will happen
Chance = random, no idea if something will happen
Law of Large Numbers
If we repeat random events again repeatedly, their fraction of success gets closer to the average/expected value (ex. flipping a coin is 50/50, rolling dice is 1 in 6)
3 Steps in the Sampling Process
1) Unit of analysis: population of interest, groups to be investigated
2) Define study population: know the nature of the population the sample is coming from
3) Establish sampling frame: complete list (ex. UAlberta students, phone directories, census lists, etc.)
2 Types of samples
Probability Sampling: random selection, probability of an individual being selected is known
Non-Probability Sampling: probability of an individual being selected is NOT known (constructivists use this)
Types of probability sampling
Random (simple random, systematic)
Stratified
Cluster
2 types of Random Sampling
Simple Random: everyone has an equal + random chance of being chosen, IV, eliminates bias
Systematic: pick a random starting point on a list, pick every nth subject
Stratified Sampling
Concerned about underrepresented groups
- Divide samples into subgroups, and randomly select from the subgroups proportional to population/sampling frame
- Ensure small groups aren’t overlooked
Cluster Sampling
Focused on where people cluster, helpful when population is spread out/hard to access
- sampling frame focuses on clusters, select a few clusters, randomly select within those clusters
ex. soccer teams
4 Types of Non-probability Sampling
1) Convenience - most accessible/most willing
2) Quota - population divided into subgroups based on census data, selected based on reflected proportion in population (like stratified but no random assignment, still bias)
3) Purposive - Focus on a limited population/small geographic area, participants have experience with phenomena of interest
4) Snowball - Identify 1 or more subjects, and ask them to find more, useful for difficult to reach populations
Non-response rate/bias
Error due to non participation or non response (ex. who answers the phone)
Sampling error
Inaccuracies in generalizations of a population because sample doesn’t represent the population
- SD of sample, sample size
Types of quantitative data and levels of measurement
Discrete data: takes on a particular numerical or categorical value (Nominal, ordinal)
Continuous data: any value/range (Interval, ratio)
Difference between levels of measurement
Nominal - values hold no meaning, no numeric value
Ordinal - Groups are different to a degree (we don’t know what degree, but still a vague order)
Interval - Differences between groups to a specified degree, but no true zero point
Ratio - True zero point
Concept vs. Construct
Concept is an idea, can be concrete or abstract
Construct is something we invent to tap into concept
