Week 4 Flashcards
Why do people conduct experiments? i.e. what is the key aspect of why experiments are conducted?
The aspect of CONTROL
- increased internal validity of the study - i.e. able to generate cause and effect relationships
What are the types of quantitative research?
Descriptive
Exploratory
Causal
Research type: Used to gain new insights, discover new ideas and increase knowledge about a phenomenon (conducted when little is known about a topic)
Exploratory (quantitative) research
Research type in which: Structured observations or survey (or both) are used to describe a phenomenon, situation, group or characteristic.
Descriptive (quantitative) research
Research type in which the research assesses cause and effect relationships, statistically.
Causal (quantitative) research
What is the most common purpose of conducting quantitative research, especially through experiments?
Establish a causal relationship between IV and DV.
What are the conditions required to make a causal statement?
1 - X precedes Y
2 - X and Y are correlated
3 - No extraneous variables impacting outcome
Although we NEVER PROVE something, we can do this.
Gather evidence that supports our claim
Testing a theory
Deductive
Directs the research process - framework of who, what, when, where and how - telling the audience who is involved, when, where, how, etc.
Research Design
What is the main purpose of research design?
Answers research questions
What are the purposes of research design?
Answer research questions Level of control Understand biases Analytical techniques/analysis Interpretation
Experimental design:
- What is the aim?
- Comparison between _______ and _______ groups
- to get cause and effect relationships, we need at least two groups, and we need to _______ them
cause and effect relationship - aim
control and treatment
compare (control and treatment groups))
Experimental design - definition:
- There is an ________
- There is an _______ and a _______ group
- There is _____ _______ to groups
- _________ of conditions exists across groups (Except the treatment/intervention)
- intervention
- experimental (treatment); control
- Random assignment
- Constancy
Technique to ensure that experimental and control groups are as similar as possible (other than the intervention)
Random assignment
Random assignment is related to _______ validity - statistically speaking, it should ensure the two groups have similar attributes
internal
Randomization:
- Random _______ to groups (_______validity) - equal extraneous variables in both groups
- Random _________ from population to sample (________ validity) equals extraneous variables in the sample that are true for the population
- Assignment
2. Selection
Measurable characteristic that varies among subjects
Variable
What are the different types of variables?
IV
DV
Extraneous variable or covariate
Name the variable based on the definition
A - outcome or presumed effect
B - Interventions or presumed cause
C - Alternative or other possible causes
A - DV
B - IV
B - EV
What is the purpose of control (concept of control)?
1 - decrease error and the influence of unwanted extraneous variables
2 - Increase probability that study findings are an accurate reflection of the IV-DV relationship
What are the different types of settings?
Natural setting
Partially controlled
Highly controlled
Name the setting type based on the description.
A - Artificial environment for the sole purpose of doing research
B - Uncontrolled, real-life situation
C - Manipulated or modified in some way
A - Highly controlled
B - Natural setting
C - Partially controlled
Match the research design to the setting.
A - Experiment
B - Quasi-experimental
C - Observational
A - highly controlled
B - partially controlled
C - natural (some exceptions)
___________ has the highest level of manipulation of all research methods
Experimentation
Describe the validity type based on the description.
A - Are changes in the outcome (DV) due to a change in the exposure (IV)?
B - Are the changes generalizable to other populations/settings?
C - The goal of which is to be generalizable to other populations
D - Changes we are observing are the result of the intervention alone - causality
A - Internal
B - external
C - external
D - internal
When doing experiments, generalizability is often low. But, researcher have one assumption to alleviate this, what is it?
Assume that are commonly shared human characteristics - a form of universal evidence from which to generate generalizability
Potential, common types of extraneous variables.
Threats to internal validity
What are some threats to internal validity?
Alternative explanations Rival hypothesis(ses) History, maturation, testing, instrumentation, selection, attrition/mortality
If participants characteristics raise possible alternative explanations
Selection bias
Inter-rater differences, failure of instrument
Instrumentation (threat to internal validity)
IF an event occurs simultaneously with the intervention and affects the outcome
history (threat to internal validity)
Natural changes which affect the outcome of the study
Maturation (threat to internal validity)
Act of being tested once affects outcome of next test
Testing (threat to internal validity)
For the following descriptions, name the threat to internal validity
A - People become sensitive - change their behaviour as a result of being tested
B - Getting hungry and changing your behaviour
C - Battery in the watch dies
D - if one group has something drastically different from the other
E - some people drop out of the study or die
A - testing B - Maturation C - Instrumentation D - selection bias E - Mortality/attrition
Differential loss from groups
Mortality/attrition
What are the different threats to external validity?
selection effects
Hawthorne effects
Measurement effects
If a study sample does not represent the population of interest
Selection effects
Natural reactions to being studied - people in both groups know what is occuring and either change their behaviour or communicate with each other, or both
Hawthorne effects
Actual act of being tested affects outcome - sensitivity
Measurement effects
Describe the level of control for threats to validity for the following research designs:
A - experimental
B - Quasi-experimental
C - Non-experimental
A - control most threats
B - control some threats
C - may control some threats
For the following flow-chart, describe what research design is chosen:
- Are participants randomly assigned?
a) Yes - A
b) No –> continue to 2 - Is there a control group
a) Yes - B
b) No - C
A - experimental
B - quasi-experimental
C - non-experimental
Describe the typical flowchart of an experimental design, starting from the eligible and consenting sample and ending with the outcome.
Eligible and consenting sample –> random assignment to treatment or control groups –> Manipulation –> outcome –> compare outcome (analyses)
Threats to validity occur in what part of the experimental design flowchart?
Selection effects occur before placing individuals into experimental or control group
other threats occur from the groups –> outcome stage
What are the strengths of the experimental design?
Establish causality/causal direction
Control
What are the limitations of experimental design?
May be difficult to implement
Generalizability/external validity may be low
Not ethical for some conditions
What is the major difference between experimental and quasi-experimental design?
No random assignment
What additional threat to validity does the quasi-experimental design have compared to the experimental design?
selection effects
What are the strengths of the quasi-experimental design?
Establish causality/causal direction
Control
What are the limitations of the quasi-experimental design?
Cannot make clear cause and effect statements
May be difficult to implement
Generalizability (External validity) may be low
Not ethical for some conditions
If we delete the randomizatoin step from the classic experimental design, what do we have?
Quasi-experimental design called non-equivalent control
If we delete the control group from the classic experimental design, hat do we have?
Quasi-experimental design called one group (pre-test - post-test)
What is the difference between a non-experimental design and experimental design?
No random assignment (may have random selection however)
No manipulation
The analysis of a non-experimental design may be?
Univariate, bi-variate, multivariate
What are the strengths of the cross sectional non-experimental design?
Fast
less expensive
large number of participants
large number of variables
What are the limitations of the cross-sectional non-experimental design?
No causal certainty
threat to validity - selection bias
What are the strengths of the non-experimental prospective (longitudinal study) design?
timeline established (exposure before outcome) large number of participants Large number of variables
What are the limitations of the non-experimental prospective (longitudinal study) design?
VERY expensive, long-term follow-up needed
large loss to follow-up possible
Threats to validity
- selection bias
- history (many people change exposure over time)
- instrumentation (tests may change over time - e.g. breast cancer screening)
- testing (if people are aware of outcomes - may change behaviours)
What are the strengths of the non-experimental retrospective research design?
Fewer participants - very specific
Large number of variables
What are the limitations of the non-experimental retrospective research design?
Very difficult to find adequate control group
threats to validity
- selection bias
- recall bias (testing bias)
