Chapter 3: Research methods and study design Flashcards
What are the steps to forming a good experimental design?
- Select the population
- Operationalize variables
- Divide into groups
- Random sampling
- Random assignment
- Measurement
- Test the hypothesis
What are the objectives in the “select the population” step?
- Determine the population of interest
- Consider what group will be pragmatic to sample
What are the common flaws in the “select the population” step?
- Population is too restrictive
- Sampling all individuals of interest is not practical
What are the objectives in the operationalize variables step?
- Determine the independent and dependent variable
- Specify exactly what is meant by each variable
- Make sure the dependent variable can be measured quantitatively within the parameter of the study
What are the objectives in the divide into groups step?
- Carefully select experimental and control groups
- Homogenize the two groups
- Isolate the treatment by controlling for potential extraneous variable
What is the flaw in the operationalized variable step?
- Insufficient rigor in the description
- Manipulation of the independent variable presents practical problems
What are the flaws in the divide into groups step?
- Control group does not resemble treatment along important variable
- Experiments is not double-blind
- Participants can guess the experiment, allowing a placebo effect to occur
What is the objective in the random sampling step?
- Make sure all members of the population are represented
2.. Ideally each member has an equal chance of being selected - Meeting these criteria is often not possible for practical reasons
What are the flaws in the random sampling step?
- Sampling is not truly random
- Sample does not represent the population of interest
What are the objectives of the random assignment step?
- Individuals who have been sampled are equally likely to be assigned to treatment or control
- Consider matching along potential extraneous variable which have been pre-selected
What are the flaws in the random assignment step?
- Groups are not properly matched
- Assignment is not perfectly random
What is the objective of the measurement step?
- Make sure measurements are standardized
- Make sure instruments are reliable
What are the flaws in the measurement step?
- Tools are not precise enough to pick up a result
- Instruments used for measurements are not reliable
What are the objectives in the test the hypothesis step?
- Use statistics to check for a significant difference
- Assign a pre-established threshold at which the null hypothesis will be rejected
What is the flaw in the test the hypothesis step?
- Small sample size leads to insufficient power
- Researchers do not set thresholds in advance and make after the fact conclusion that lead to logical fallacies
Impression management
Participants adapt their responses based on social norms or perceived researcher expectations, self-fulling prophecy, a methodology is not double-blind, Hawthrone effect
Confounding variables
Extraneous variable not accounted for in the study, another variable offers an alternative explanation for results, lack of a useful control
Lack of reliability
Measurement tools do not measure what they purport to, lack consistency
Sampling bias
Selection criteria are not random, the population used for the sample does not meet conditions for a statistical test (e.g., the population is not normally disturbed)
Attrition effects
Participants fatigue, participants drop out of the study
Demand characteristics
Participants interpret what the experiment is about and subconsciously respond in ways that are consistent with the hypothesis
Experiment doesn’t reflect real world
Laboratory setups that don’t translate to the real world, lack of generalizability
Selection criteria
Too restrictive of inclusion/exclusion criteria for participants (i.e., sample is not representative)
Situational effects
Presence of laboratory conditions changes outcome (e.g., pre-test and post-test, presence of experimenter, claustrophobia in an MRI machine)
Lack of statistical power
Sample groups have high variability, sample size too small
What are the non-experimental designs?
- Correlational
- Ethnographic
- Twin
- Longitudinal
- Case
- Phenomenological
- Survey
- Archival
- Biographical
- Observational
Correlational
Measures the quantitive relationships between two variables
Ethnographic
Deep lengthy qualitative analysis of culture and its characteristic
Twin
Analysis of heritability through measuring characteristic of twins
Longitudinal
Long-term analysis that intermittently measures the evolution of some behavior or characteristic
Case
Deep analysis of a single case or example
Phenomenological
Self-observation of phenomenon by researcher or small group of participants
Survey
Use of a series of questions to allow participants to self-report behaviors or tendencies
Archival
Analysis of historical records for insight into a phenomenon
Biographical
Exploration of the events and circumstances of an individual’s life
Observational
Broad category that includes any research in which experiments do not manipulate the situation or results
What are the strengths and weaknesses of correlational study?
Strengths:
1. Great preliminary technique
2. Usually easy to conduct
Weaknesses:
1. Does not establish causality
2. May not pick up a nonlinear relationship
What are the strengths and weaknesses of Ethnographic study?
Strength:
1. Provides detailed analysis and comprehensive evaluation
Weaknesses:
1. Researcher’s presence may affect an individual’s behavior
2. Heavily dependent on the researcher conducting the study, difficult to replicate,, and objectivity may not be compromised
What are the strengths and weaknesses of Twin study?
Strengths:
1. Offers insight into how nature and nurture might interact to lead to various characteristics
Weaknesses:
1. Difficult to find participants who meet the criteria
2. Difficult to analyze the complex variable involved and how they interact
What are the strengths and weaknesses of Longitduinal study?
Strengths:
1. Scientist can understand how trait of interest changes over time
Weaknesses:
1. Logistically demanding expensive and difficult to implement
2. High attrition rate
What are the strengths and weaknesses of Case study?
Strength:
1. Offers comprehensive details about the single case
Weaknesses:
1. Results may not be generalizable
2. Does not offer points of reference or comparison
What are the strengths and weaknesses of Phenonmenological study?
Strengths:
1. Introspection can provide insight into behaviors and occurrences that are difficult to measure
Weaknesses:
1. Lacks objectivity due to results coming from self-analysis
2. Difficult to generalize results to other circumstances or individuals
What are the strengths and weaknesses of Survey study?
Strengths:
1. Easy to adminster
2. Can provide quantitative data that can be compared to large participants pools
Weaknesses:
1. Self-reporting creates limitations in objectivity
What are the strengths and weaknesses of Archival study?
Strengths:
1. Provide insight into events from the past that are unique from every-day behavior
Weaknesses:
1. Quality of analysis subject to the quality and integrity of records
2. Difficult to conduct follow-ups
3. Data are unlikely to be comprehensive, leaving ambiguity and unanswered questions
What are the strengths and weaknesses of Biographical study?
Strengths:
1. Comprehensive knowledge of all the details of an individual’s life
Weaknesses:
1. Limitations in objectivity
2. Diffculuts to generalizing observations
What are the strengths and weaknesses of Observational study?
Strengths:
1. Naturalistic observation of circumstances as they are
2. Difficult to tease out the complex interplay of many variables
The ________ method is the primary method used by researchers to establish causality
Experimental
The ________ variable is the variable manipulated by the researcher
Independent
The _______ variable is the variable that is measured to test for an effect
Dependent
An _________ definition is a precise definition of the meaning of research variables
Operational
____________ indicates numerical, whereas ________ indicates descriptive or categorical
Quantitative, Qualitative
A ________ group is the group researcher use as a point of comparison, it doesn’t receive the treatment
Control
An ______________ group is the group that does receive the treatment in a study
Experimental
Extraneous variables
Variables other than the research variable that could potentially explain a result in an experiment
A sample within research should reflect the ______________
Population
Individuals who are sampled from the population should be _______ assigned to the control and experimental group
Randomly
In _____________, researchers assign the null hypothesis, the hypothesis that any effects are due to chance and there is no experimental result
hypothesis testing
Type 1 error
Known as false positive, rejecting the null hypothesis when in fact it is true
Type 2 error
Known as false negative, accepting the null hypothesis when in fact it is false
Internal validity
How well done a study is or how well it allows researchers to draw conclusions from measured effects
External validity
How well a research results apply to the population at large and real-life scenarios
_____________ methodologies, don’t allows researchers to draw conclusions about causality
Non-experimental
