Experimental design - definition: 1. There is an ________ 2. There is an _______ and a _______ group 3. There is _____ _______ to groups 4. _________ of conditions exists across groups (Except the treatment/intervention)

1. intervention 2. experimental (treatment); control 3. Random assignment 4. Constancy

Week 4 Flashcards by Daniel Picard

Why do people conduct experiments? i.e. what is the key aspect of why experiments are conducted?

The aspect of CONTROL

- increased internal validity of the study - i.e. able to generate cause and effect relationships

How well did you know this?

Not at all

Perfectly

What are the types of quantitative research?

Descriptive
Exploratory
Causal

How well did you know this?

Not at all

Perfectly

Research type: Used to gain new insights, discover new ideas and increase knowledge about a phenomenon (conducted when little is known about a topic)

Exploratory (quantitative) research

How well did you know this?

Not at all

Perfectly

Research type in which: Structured observations or survey (or both) are used to describe a phenomenon, situation, group or characteristic.

Descriptive (quantitative) research

How well did you know this?

Not at all

Perfectly

Research type in which the research assesses cause and effect relationships, statistically.

Causal (quantitative) research

How well did you know this?

Not at all

Perfectly

What is the most common purpose of conducting quantitative research, especially through experiments?

Establish a causal relationship between IV and DV.

How well did you know this?

Not at all

Perfectly

What are the conditions required to make a causal statement?

1 - X precedes Y
2 - X and Y are correlated
3 - No extraneous variables impacting outcome

How well did you know this?

Not at all

Perfectly

Although we NEVER PROVE something, we can do this.

Gather evidence that supports our claim

How well did you know this?

Not at all

Perfectly

Testing a theory

Deductive

How well did you know this?

Not at all

Perfectly

Directs the research process - framework of who, what, when, where and how - telling the audience who is involved, when, where, how, etc.

Research Design

How well did you know this?

Not at all

Perfectly

What is the main purpose of research design?

Answers research questions

How well did you know this?

Not at all

Perfectly

What are the purposes of research design?

Answer research questions
Level of control
Understand biases
Analytical techniques/analysis
Interpretation

How well did you know this?

Not at all

Perfectly

Experimental design:

What is the aim?
Comparison between _______ and _______ groups
to get cause and effect relationships, we need at least two groups, and we need to _______ them

cause and effect relationship - aim
control and treatment
compare (control and treatment groups))

How well did you know this?

Not at all

Perfectly

Experimental design - definition:

There is an ________
There is an _______ and a _______ group
There is _____ _______ to groups
_________ of conditions exists across groups (Except the treatment/intervention)

intervention
experimental (treatment); control
Random assignment
Constancy

How well did you know this?

Not at all

Perfectly

Technique to ensure that experimental and control groups are as similar as possible (other than the intervention)

Random assignment

How well did you know this?

Not at all

Perfectly

Random assignment is related to _______ validity - statistically speaking, it should ensure the two groups have similar attributes

internal

How well did you know this?

Not at all

Perfectly

Randomization:

Random _______ to groups (_______validity) - equal extraneous variables in both groups
Random _________ from population to sample (________ validity) equals extraneous variables in the sample that are true for the population

Assignment

2. Selection

How well did you know this?

Not at all

Perfectly

Measurable characteristic that varies among subjects

Variable

How well did you know this?

Not at all

Perfectly

What are the different types of variables?

IV
DV
Extraneous variable or covariate

How well did you know this?

Not at all

Perfectly

Name the variable based on the definition
A - outcome or presumed effect
B - Interventions or presumed cause
C - Alternative or other possible causes

A - DV
B - IV
B - EV

How well did you know this?

Not at all

Perfectly

What is the purpose of control (concept of control)?

1 - decrease error and the influence of unwanted extraneous variables

2 - Increase probability that study findings are an accurate reflection of the IV-DV relationship

How well did you know this?

Not at all

Perfectly

What are the different types of settings?

Natural setting
Partially controlled
Highly controlled

How well did you know this?

Not at all

Perfectly

Name the setting type based on the description.

A - Artificial environment for the sole purpose of doing research
B - Uncontrolled, real-life situation
C - Manipulated or modified in some way

A - Highly controlled
B - Natural setting
C - Partially controlled

How well did you know this?

Not at all

Perfectly

Match the research design to the setting.
A - Experiment
B - Quasi-experimental
C - Observational

A - highly controlled
B - partially controlled
C - natural (some exceptions)

How well did you know this?

Not at all

Perfectly

___________ has the highest level of manipulation of all research methods

Experimentation

Describe the validity type based on the description. A - Are changes in the outcome (DV) due to a change in the exposure (IV)? B - Are the changes generalizable to other populations/settings? C - The goal of which is to be generalizable to other populations D - Changes we are observing are the result of the intervention alone - causality

A - Internal B - external C - external D - internal

When doing experiments, generalizability is often low. But, researcher have one assumption to alleviate this, what is it?

Assume that are commonly shared human characteristics - a form of universal evidence from which to generate generalizability

Potential, common types of extraneous variables.

Threats to internal validity

What are some threats to internal validity?

``` Alternative explanations Rival hypothesis(ses) History, maturation, testing, instrumentation, selection, attrition/mortality ```

If participants characteristics raise possible alternative explanations

Selection bias

Inter-rater differences, failure of instrument

Instrumentation (threat to internal validity)

IF an event occurs simultaneously with the intervention and affects the outcome

history (threat to internal validity)

Natural changes which affect the outcome of the study

Maturation (threat to internal validity)

Act of being tested once affects outcome of next test

Testing (threat to internal validity)

For the following descriptions, name the threat to internal validity A - People become sensitive - change their behaviour as a result of being tested B - Getting hungry and changing your behaviour C - Battery in the watch dies D - if one group has something drastically different from the other E - some people drop out of the study or die

``` A - testing B - Maturation C - Instrumentation D - selection bias E - Mortality/attrition ```

Differential loss from groups

Mortality/attrition

What are the different threats to external validity?

selection effects Hawthorne effects Measurement effects

If a study sample does not represent the population of interest

Selection effects

Natural reactions to being studied - people in both groups know what is occuring and either change their behaviour or communicate with each other, or both

Hawthorne effects

Actual act of being tested affects outcome - sensitivity

Measurement effects

Describe the level of control for threats to validity for the following research designs: A - experimental B - Quasi-experimental C - Non-experimental

A - control most threats B - control some threats C - may control some threats

For the following flow-chart, describe what research design is chosen: 1. Are participants randomly assigned? a) Yes - A b) No --> continue to 2 2. Is there a control group a) Yes - B b) No - C

A - experimental B - quasi-experimental C - non-experimental

Describe the typical flowchart of an experimental design, starting from the eligible and consenting sample and ending with the outcome.

Eligible and consenting sample --> random assignment to treatment or control groups --> Manipulation --> outcome --> compare outcome (analyses)

Threats to validity occur in what part of the experimental design flowchart?

Selection effects occur before placing individuals into experimental or control group other threats occur from the groups --> outcome stage

What are the strengths of the experimental design?

Establish causality/causal direction | Control

What are the limitations of experimental design?

May be difficult to implement Generalizability/external validity may be low Not ethical for some conditions

What is the major difference between experimental and quasi-experimental design?

No random assignment

What additional threat to validity does the quasi-experimental design have compared to the experimental design?

selection effects

What are the strengths of the quasi-experimental design?

Establish causality/causal direction | Control

What are the limitations of the quasi-experimental design?

Cannot make clear cause and effect statements May be difficult to implement Generalizability (External validity) may be low Not ethical for some conditions

If we delete the randomizatoin step from the classic experimental design, what do we have?

Quasi-experimental design called non-equivalent control

If we delete the control group from the classic experimental design, hat do we have?

Quasi-experimental design called one group (pre-test - post-test)

What is the difference between a non-experimental design and experimental design?

No random assignment (may have random selection however) | No manipulation

The analysis of a non-experimental design may be?

Univariate, bi-variate, multivariate

What are the strengths of the cross sectional non-experimental design?

Fast less expensive large number of participants large number of variables

What are the limitations of the cross-sectional non-experimental design?

No causal certainty | threat to validity - selection bias

What are the strengths of the non-experimental prospective (longitudinal study) design?

``` timeline established (exposure before outcome) large number of participants Large number of variables ```

What are the limitations of the non-experimental prospective (longitudinal study) design?

VERY expensive, long-term follow-up needed large loss to follow-up possible Threats to validity - selection bias - history (many people change exposure over time) - instrumentation (tests may change over time - e.g. breast cancer screening) - testing (if people are aware of outcomes - may change behaviours)

What are the strengths of the non-experimental retrospective research design?

Fewer participants - very specific | Large number of variables

What are the limitations of the non-experimental retrospective research design?

Very difficult to find adequate control group threats to validity - selection bias - recall bias (testing bias)