Research: Experimental Method Flashcards
Qualitative research:
It involves collecting and analyzing non-numerical data, such as words, images, or observations, to uncover patterns, themes, and insights.
Interviews: Open-ended questions to learn about people’s perspectives in depth.
Focus groups: Talk about a specific topic, and learn about their opinions, viewpoints, and attitudes about a topic.
Observation: Quietly watching people and take notes on people’s behaviours, interactions, and surroundings.
Ethnography: Spending time with people, joining their activities and conversations to understand their values and way of life.
Quantitative research:
Quantify and analyze numerical data to test hypotheses, identify patterns, and establish relationships between variables.
Quantitative Research Methods: Surveys:
Standardized questions to a large sample of participants to collect data on attitudes, behaviors, or demographics.
Surveys often use closed-ended questions with response options on a Likert scale or other rating scales. They use numerical scales to measure responses.
Quantitative Research Methods: Experiments:
Manipulating one or more variables to observe their effects on other variables under controlled conditions.
Helps them figure out cause-and-effect relationships.
Ex: Researchers change something (like giving a medicine) to see what happens (like if the patient gets better).
See if the medicine caused any changes.
Quantitative Research Methods: Longitudinal Study:
Researchers collect data from the same group of people over a long period of time. They do this to see how things change or stay the same over time, using statistical methods to analyze trends.
Ex: Imagine tracking a group of students’ grades over many years. In a longitudinal study.
Correlation:
Two variables share some kind of relationship.
Causation:
One variable causes something to happen in another variable.
One variables CAUSES the other.
Correlation: One-way causality:
Variable X is the cause of variable Y.
« Reverse causality » if Y is the cause of X
Correlation: Two-way causality
Both variables may be the cause of
each other. X could be causing Y, and Y could be causing X.
For example, watching TV might be causing pain, while having pain might be causing one to watch TV.
Correlation: Confound:
A third variable may be responsible for the correlation.
Z that’s causing there to be a correlation between 2 variables.
Confound = Z.
Correlation: Spurious correlation:
A mathematical relationship in
which two events or variables have no causal connection.
Experimental studies:
Used to investigate cause-and-effect relationships between variables. In an experimental study, the researcher manipulates an independent variables to observe their effects on the dependent variable, while controlling for other factors that could influence the results.
Random assignment (make sure there isn’t a confound [a factor that every participation has] that could be “creating” the correlation]), control groups, manipulated of variables.
Experimental Studies: Independent Variables
The variable that the research manipulates/ changes (assumed to effect the dependent variable).
Experimental Studies: Dependent Variable
The variable that the researchers observes to assess the effects of the independent variable.
Assumed to be influenced by changes in the independent variable.
Experimental Studies: Control Group/ Comparison Group:
Group that is treated identically to the experimental group, except they are not exposed to the manipulation of the independent variable.
The control group allows researchers to account for extraneous variables and assess the specific effects of the independent variable.
Experimental Studies: Experimental Group:
The experimental group is the group of participants who are exposed to the manipulation or treatment of the independent variable. This group allows researchers to assess the effects of the independent variable on the dependent variable.
Experimental Studies: Random Assignment:
Random assignment involves randomly assigning participants to either the experimental group or the control group.
Reduces chance of confound, and bias.
Experimental Studies: Controlled Conditions:
This often involves standardizing procedures, using consistent measurement instruments, and controlling environmental factors.
Hypothesis:
The expected relationship between the independent and dependent variables.
Self-selected sample:
People who chose to be affected by the manipulated independent variable. There could be lots of confounds involved.
The Balanced Placebo Design:
Used in studies investigating the effect of drugs or treatments, to split apart the effects of the actual drug on the body and the placebo effect.
Group 1: Receives the drug and is told so.
Group 2: Receives the drug and isn’t told so.
Group 3: Receives the placebo and told that it’s the real drug.
Group 4: Receives the placebo and told its the placebo.
Can see what effects come from the drug and which effects comes from the mind itself (placebo).
Between-subjects (between-groups) study design:
Different people test each conditions (each person is exposed to a condition)
Participants are exposed to their respective conditions, and researchers measure the outcome (attitudes, behaviours, etc) of each group, they then compare outcomes of each group to see the effects of the independent variable (manipulated conditions) on the dependent variable (measured outcome).
Often used for cause-and-effect relationships (the one used for the project).
Within-subjects (repeated-measures) study design:
Participants test all conditions, where conditions are presented in a random order to them.
Each participant is their own control group (because they stay the same = their environment, while being exposed to different levels of the independent variable).
Hypothesis guessing:
Participants guess the study’s purpose/ hypothesis and change their behaviour/ response to align with their perceived study expectations.
Contamination:
One level of the independent variable can influence participants’ perception to the other level.
Within-subjects study design: PROS and CONS
PROS:
- Lower cost (need fewer participants = lower costs elsewhere, compensation, recruitment).
- Smaller sample size required.
- Noise control (each participant is their own control group).
CONS:
- Hypothesis guessing: After being exposed to a few conditions, participants might start guessing the hypothesis/ purpose of the study.
- Contamination: Response of one level of the independent variable could influence the next one’s response.
Between-subjects study design: PROS and CONS:
PROS:
- Avoid contamination: In between-subjects designs, each participant is only exposed to one condition or treatment. This helps prevent contamination, where exposure to one condition influences responses to another condition.
- Minimizes hypothesis guessing: Participants are less likely to guess the study’s hypotheses or purpose as they are only exposed to one condition.
CONS:
- Larger sample size required: This is because each group of participants represents a separate condition or treatment, and researchers need enough participants in each group to ensure statistical power and reliability.
- More noise if randomization fails: Random assignment of participants to different conditions helps control for individual differences between groups. However, if randomization fails to eliminate these differences, there may be more “noise” or variability in the data. If randomization fails to eliminate individual differences between groups.
Internal validity:
Extent that the outcomes in the study is caused by the independent variable.
“Are the findings due to the independent variable?” (= “did I establish causality?”)
- True randomization.
- Large sample sizes.
External validity:
Can I generalize the results to other populations or contexts?
Large population.
Convenience samples:
Sample was created through convenience (just having undergraduate students cause we have more connections with them).
WEIRD Samples:
White, Educated, Industrialized, Rich and Democratic. Does it represent other demographics?
Ecological Validity:
Does the study mimick what would happen in real life (realism of the experimental conditions)? Do the findings reflect what would happen in every day life due to the experimental conditions?
Different environment = different behaviours and decision-making process.
Represent real-life behaviours.
Simple “A/B” Experiments:
A method used in marketing, web design, and product development to compare two or more versions of something (such as a webpage, email, advertisement, or product feature) to determine which one performs better.
Randomly assigned to a variation (A or B) -> performance measured -> analyzed which one is better -> better one is implemented.
Measured by: higher conversion rates, increased sales, or improved user engagement.
Complex AI-based optimization:
Constant and automatic
experimentations on visitors in
order to find the best version for
each individual visitor (complete
personalization).
A/B Testing: Testable Features (X’s):
What can be changed on the webpage/ variations on the webpage:
- Endless creative features
- Animations (rich media)
- Placement on webpage
- Target audience
A/B Testing: Measurable Outcomes (Y’s):
- Click through
- Likes / Comments
- Target site visits (w/o click through)
- Time on site
- Sales or subscriptions
- Intermediate steps in purchase process
Ethical considerations for research:
- Obtain informed consent:
By ensuring that the were provided enough information, ensuring that they fully understand the study and able to ask questions, ability to decline or withdraw (voluntariness). - Protect vulnerable populations: Children, prisoners, people with impaired cognitive capacity.
Moderator:
Variable that affects the direction and/or strength of the relation between independent and dependent variables.
