experimental independent random group design/issues Flashcards
goals of scientific method
description: describe events and relationships between variables using nomothetic approach and quantitative analysis
prediction: correlational relationships allow prediction of behaviours/events, but do not allow inferences on causes of these relationships
explanation: understand the cause of the phenomena when 3 conditions are met: covariation, timeorder relationship, elimination of alternatives
application: apply knowledge and research to improve lives
construct validity
frequency claims
how well have you measured the variable in question
construct validity
association
how well have you measured each of the 2 variables in the association
construct validity
causal claims
how well have you measured or manipulated the variables of the study
statistical validity
frequency
how large is the margin of error
statistical validity
association
how strong is the association (effect size) is it statistically significant
statistical validity
causal claims
is there a difference between groups and how large is it (what is the effect size) is the difference between groups statistically significant
internal validity
frequency
not relevant
internal validity
association
not asserting causality, but should ask if researchers control for other variables when they analyse the relationship
internal validity
causal claims
only experimental design can establish causation. was it an experiment, does it control for alternate explanations by limiting confounds and by random assignment to groups. does it avoid several internal validity threats
external validity
frequency
how representative is the sample, was it a sample of the pop of interest
external validity
association
how representative is the sample, to what other settings of problems might the association be generalised, to what pop settings and times can we generalise this association claim?
external validity
causal
how rep is the sample, do the manipulations in the experiment generalise to other situations/settings to what pop/setting/time can we generalise causal claim
Logic of Experimental Research
Manipulate an independent variable (IV) in an experiment to observe the effect on behaviour. The effect on behaviour is assessed by the dependent variable (DV)
An IV must have at least two levels/conditions
Treatment and control (or comparison)
Identifying the IV and DV
IV exists as levels. The values are categorical.
To state the IV caused the observed changes in the DV (effect)
3 conditions need to be satisfied
+Covariation
+Time-order relationship
+Elimination of plausible alternative causes
When the 3 conditions are met, the experiment is said to have internal validity.
+Covariation – observe a relationship between IV and DV
+Time-order relationship – manipulate an IV and then observe a subsequent difference is behaviour
+Experimental control allows researchers to make causal inference because it eliminates plausible causes
Control is the essential ingredient of experiments
Gained through (1) manipulation, (2) holding conditions constant , and (3) balancing
Experimental Manipulation
Not just observing to collect data or responding to a survey to collect data
Involves exposing different people or groups of people to different experimental conditions (sometimes groups do not know which condition they are in) and comparing the outcomes of the groups
To manipulate: the researcher actively changes the condition of the different groups to test the effect/outcome from each group, and
To draw conclusions of causality
Holding Conditions Constant
A control technique to avoid confoundings
Holding conditions/factors/variables constant means
they do not change (variables are the same for all groups)
they cannot possibly covary with the IV
They are held constant because they are thought to influence the behaviour we are studying , hence they are referred to as plausible alternative causes.
There is the possibility that there may be confounding factors that were not anticipated
Balancing
Comparable groups in terms of individual differences, e.g., personality, intelligence, etc.
Individual differences cannot be held constant
Apply balancing technique to establish equivalent groups
Balancing = averaging individual differences across conditions. How is this achieved?
Random assignment of participants to conditions
Assigning Group Members
How should the participants be assigned so that confounding variables (e.g., individual differences) are counterbalanced or distributed across the experimental conditions?
How many to assign to each group? Sample size?
A priori power analysis (e.g., using Gpower - more about this in Week 4)
Independent Groups Design – Randomised Groups
- initial sample
- randomised assignment to one of two or more groups
- independent variable manipulation
- dependent variable measured
Block Randomisation
A common procedure to carry out random assignment
Random order the conditions
For example: 3 conditions:
Imagery (I); Rehearsal (R), and Control (C)
Assign a number to each condition:
I = 1, R = 2 and C = 3
Use a random number table (SZZ 9th ed. Pg 443/10th ed. Pg 438) or randomise the numbers in excel
Using random number table: 1-5-6-6-4-1-0-4-9-3-2-0-4-9-3-2-8-3-9-1-9-1-1-3-2-1-9-9-9-5-9-5-1-6-8-1-6-5-2-2-7-1-9-5-4-8-2-2-3-4-6-7-5-1-2-2-9-2-3
If we want to have 4 participants for each condition, we would need to have 4 blocks consisting of the 3 randomised conditions.
You have Participant A, F, G, and K for Imagery.
You have Participant C, E, J, and L for Rehearsal.
You have Participant B, D, H, and M for Control
Internal Validity
The degree to which a researcher draws accurate conclusions about the effects of the IV = when an experiment is internally valid, the researcher can confidently conclude that differences were due to the IV.
Nothing differ systematically other than the IV, i.e. no confounding has occurred. Confounding is a fatal flaw
Threats to Internal Validity
The Pepsi Challenge in 1975:
A taste test in which people were asked to taste two cola beverages and indicate which they preferred. As it was originally designed, glasses of Pepsi were marked with a letter ‘M’ and glasses of Coca-Cola were marked with a ‘Q’. People seemed to prefer Pepsi to Coca-Cola in these tests, but a confound was present. What was it?
Intact Groups (not individuals), e.g., psychology students
Subject Loss or Attrition
Mechanical subject loss
Selective subject loss (a more serious matter)
Subjects are lost differentially across the conditions of the experiment
When some characteristic of the subject is responsible for the loss
When this subject characteristic is related to the DV used to assess the outcome of the study
Placebo Control and Double-Blind Experiments (not common in psychology
External Validity
The degree to which the results obtained in one study can be replicated or generalised to other samples, research settings, and procedures
Refers to the generalisability of the results
Internal validity and external validity are inversely related to some extent
The more tightly the experimenter controls the experimental setting, the more internally valid the results but the lower the external validity
Is the goal of experimental research to generalise to the real world
