Chapter 3: Research methods and study design Flashcards
What are the steps to forming a good experimental design?
- Select the population
- Operationalize variables
- Divide into groups
- Random sampling
- Random assignment
- Measurement
- Test the hypothesis
What are the objectives in the “select the population” step?
- Determine the population of interest
- Consider what group will be pragmatic to sample
What are the common flaws in the “select the population” step?
- Population is too restrictive
- Sampling all individuals of interest is not practical
What are the objectives in the operationalize variables step?
- Determine the independent and dependent variable
- Specify exactly what is meant by each variable
- Make sure the dependent variable can be measured quantitatively within the parameter of the study
What are the objectives in the divide into groups step?
- Carefully select experimental and control groups
- Homogenize the two groups
- Isolate the treatment by controlling for potential extraneous variable
What is the flaw in the operationalized variable step?
- Insufficient rigor in the description
- Manipulation of the independent variable presents practical problems
What are the flaws in the divide into groups step?
- Control group does not resemble treatment along important variable
- Experiments is not double-blind
- Participants can guess the experiment, allowing a placebo effect to occur
What is the objective in the random sampling step?
- Make sure all members of the population are represented
2.. Ideally each member has an equal chance of being selected - Meeting these criteria is often not possible for practical reasons
What are the flaws in the random sampling step?
- Sampling is not truly random
- Sample does not represent the population of interest
What are the objectives of the random assignment step?
- Individuals who have been sampled are equally likely to be assigned to treatment or control
- Consider matching along potential extraneous variable which have been pre-selected
What are the flaws in the random assignment step?
- Groups are not properly matched
- Assignment is not perfectly random
What is the objective of the measurement step?
- Make sure measurements are standardized
- Make sure instruments are reliable
What are the flaws in the measurement step?
- Tools are not precise enough to pick up a result
- Instruments used for measurements are not reliable
What are the objectives in the test the hypothesis step?
- Use statistics to check for a significant difference
- Assign a pre-established threshold at which the null hypothesis will be rejected
What is the flaw in the test the hypothesis step?
- Small sample size leads to insufficient power
- Researchers do not set thresholds in advance and make after the fact conclusion that lead to logical fallacies
Impression management
Participants adapt their responses based on social norms or perceived researcher expectations, self-fulling prophecy, a methodology is not double-blind, Hawthrone effect
Confounding variables
Extraneous variable not accounted for in the study, another variable offers an alternative explanation for results, lack of a useful control
Lack of reliability
Measurement tools do not measure what they purport to, lack consistency
Sampling bias
Selection criteria are not random, the population used for the sample does not meet conditions for a statistical test (e.g., the population is not normally disturbed)
Attrition effects
Participants fatigue, participants drop out of the study
Demand characteristics
Participants interpret what the experiment is about and subconsciously respond in ways that are consistent with the hypothesis
Experiment doesn’t reflect real world
Laboratory setups that don’t translate to the real world, lack of generalizability
Selection criteria
Too restrictive of inclusion/exclusion criteria for participants (i.e., sample is not representative)
Situational effects
Presence of laboratory conditions changes outcome (e.g., pre-test and post-test, presence of experimenter, claustrophobia in an MRI machine)