Research: Experimental Method

Question 1

Qualitative research:

Answer 1

It involves collecting and analyzing non-numerical data, such as words, images, or observations, to uncover patterns, themes, and insights.

Interviews: Open-ended questions to learn about people’s perspectives in depth.

Focus groups: Talk about a specific topic, and learn about their opinions, viewpoints, and attitudes about a topic.

Observation: Quietly watching people and take notes on people’s behaviours, interactions, and surroundings.

Ethnography: Spending time with people, joining their activities and conversations to understand their values and way of life.

Question 2

Quantitative research:

Answer 2

Quantify and analyze numerical data to test hypotheses, identify patterns, and establish relationships between variables.

Question 3

Quantitative Research Methods: Surveys:

Answer 3

Standardized questions to a large sample of participants to collect data on attitudes, behaviors, or demographics.

Surveys often use closed-ended questions with response options on a Likert scale or other rating scales. They use numerical scales to measure responses.

Question 4

Quantitative Research Methods: Experiments:

Answer 4

Manipulating one or more variables to observe their effects on other variables under controlled conditions.

Helps them figure out cause-and-effect relationships.

Ex: Researchers change something (like giving a medicine) to see what happens (like if the patient gets better).

See if the medicine caused any changes.

Question 5

Quantitative Research Methods: Longitudinal Study:

Answer 5

Researchers collect data from the same group of people over a long period of time. They do this to see how things change or stay the same over time, using statistical methods to analyze trends.

Ex: Imagine tracking a group of students’ grades over many years. In a longitudinal study.

Question 6

Correlation:

Answer 6

Two variables share some kind of relationship.

Question 7

Causation:

Answer 7

One variable causes something to happen in another variable.

One variables CAUSES the other.

Question 8

Correlation: One-way causality:

Answer 8

Variable X is the cause of variable Y.

« Reverse causality » if Y is the cause of X

Question 9

Correlation: Two-way causality

Answer 9

Both variables may be the cause of
each other. X could be causing Y, and Y could be causing X.

For example, watching TV might be causing pain, while having pain might be causing one to watch TV.

Question 10

Correlation: Confound:

Answer 10

A third variable may be responsible for the correlation.
Z that’s causing there to be a correlation between 2 variables.

Confound = Z.

Question 11

Correlation: Spurious correlation:

Answer 11

A mathematical relationship in
which two events or variables have no causal connection.

Question 12

Experimental studies:

Answer 12

Used to investigate cause-and-effect relationships between variables. In an experimental study, the researcher manipulates an independent variables to observe their effects on the dependent variable, while controlling for other factors that could influence the results.

Random assignment (make sure there isn’t a confound [a factor that every participation has] that could be “creating” the correlation]), control groups, manipulated of variables.

Question 13

Experimental Studies: Independent Variables

Answer 13

The variable that the research manipulates/ changes (assumed to effect the dependent variable).

Question 14

Experimental Studies: Dependent Variable

Answer 14

The variable that the researchers observes to assess the effects of the independent variable.

Assumed to be influenced by changes in the independent variable.

Question 15

Experimental Studies: Control Group/ Comparison Group:

Answer 15

Group that is treated identically to the experimental group, except they are not exposed to the manipulation of the independent variable.

The control group allows researchers to account for extraneous variables and assess the specific effects of the independent variable.

Question 16

Experimental Studies: Experimental Group:

Answer 16

The experimental group is the group of participants who are exposed to the manipulation or treatment of the independent variable. This group allows researchers to assess the effects of the independent variable on the dependent variable.

Question 17

Experimental Studies: Random Assignment:

Answer 17

Random assignment involves randomly assigning participants to either the experimental group or the control group.

Reduces chance of confound, and bias.

Question 18

Experimental Studies: Controlled Conditions:

Answer 18

This often involves standardizing procedures, using consistent measurement instruments, and controlling environmental factors.

Question 19

Hypothesis:

Answer 19

The expected relationship between the independent and dependent variables.

Question 20

Self-selected sample:

Answer 20

People who chose to be affected by the manipulated independent variable. There could be lots of confounds involved.

Question 21

The Balanced Placebo Design:

Answer 21

Used in studies investigating the effect of drugs or treatments, to split apart the effects of the actual drug on the body and the placebo effect.

Group 1: Receives the drug and is told so.
Group 2: Receives the drug and isn’t told so.
Group 3: Receives the placebo and told that it’s the real drug.
Group 4: Receives the placebo and told its the placebo.

Can see what effects come from the drug and which effects comes from the mind itself (placebo).

Question 22

Between-subjects (between-groups) study design:

Answer 22

Different people test each conditions (each person is exposed to a condition)

Participants are exposed to their respective conditions, and researchers measure the outcome (attitudes, behaviours, etc) of each group, they then compare outcomes of each group to see the effects of the independent variable (manipulated conditions) on the dependent variable (measured outcome).

Often used for cause-and-effect relationships (the one used for the project).

Question 23

Within-subjects (repeated-measures) study design:

Answer 23

Participants test all conditions, where conditions are presented in a random order to them.

Each participant is their own control group (because they stay the same = their environment, while being exposed to different levels of the independent variable).

Question 24

Hypothesis guessing:

Answer 24

Participants guess the study’s purpose/ hypothesis and change their behaviour/ response to align with their perceived study expectations.

Question 25

Contamination:

Answer 25

One level of the independent variable can influence participants’ perception to the other level.

Question 26

Within-subjects study design: PROS and CONS

Answer 26

PROS:
- Lower cost (need fewer participants = lower costs elsewhere, compensation, recruitment).
- Smaller sample size required.
- Noise control (each participant is their own control group).

CONS:
- Hypothesis guessing: After being exposed to a few conditions, participants might start guessing the hypothesis/ purpose of the study.
- Contamination: Response of one level of the independent variable could influence the next one’s response.

Question 27

Between-subjects study design: PROS and CONS:

Answer 27

PROS:
- Avoid contamination: In between-subjects designs, each participant is only exposed to one condition or treatment. This helps prevent contamination, where exposure to one condition influences responses to another condition.
- Minimizes hypothesis guessing: Participants are less likely to guess the study’s hypotheses or purpose as they are only exposed to one condition.

CONS:
- Larger sample size required: This is because each group of participants represents a separate condition or treatment, and researchers need enough participants in each group to ensure statistical power and reliability.
- More noise if randomization fails: Random assignment of participants to different conditions helps control for individual differences between groups. However, if randomization fails to eliminate these differences, there may be more “noise” or variability in the data. If randomization fails to eliminate individual differences between groups.

Question 28

Internal validity:

Answer 28

Extent that the outcomes in the study is caused by the independent variable.

“Are the findings due to the independent variable?” (= “did I establish causality?”)

• True randomization.
• Large sample sizes.

Question 29

External validity:

Answer 29

Can I generalize the results to other populations or contexts?

Large population.

Question 30

Convenience samples:

Answer 30

Sample was created through convenience (just having undergraduate students cause we have more connections with them).

Question 31

WEIRD Samples:

Answer 31

White, Educated, Industrialized, Rich and Democratic. Does it represent other demographics?

Question 32

Ecological Validity:

Answer 32

Does the study mimick what would happen in real life (realism of the experimental conditions)? Do the findings reflect what would happen in every day life due to the experimental conditions?

Different environment = different behaviours and decision-making process.

Represent real-life behaviours.

Question 33

Simple “A/B” Experiments:

Answer 33

A method used in marketing, web design, and product development to compare two or more versions of something (such as a webpage, email, advertisement, or product feature) to determine which one performs better.

Randomly assigned to a variation (A or B) -> performance measured -> analyzed which one is better -> better one is implemented.

Measured by: higher conversion rates, increased sales, or improved user engagement.

Question 34

Complex AI-based optimization:

Answer 34

Constant and automatic
experimentations on visitors in
order to find the best version for
each individual visitor (complete
personalization).

Question 35

A/B Testing: Testable Features (X’s):

Answer 35

What can be changed on the webpage/ variations on the webpage:

• Endless creative features
• Animations (rich media)
• Placement on webpage
• Target audience

Question 36

A/B Testing: Measurable Outcomes (Y’s):

Answer 36

• Click through
• Likes / Comments
• Target site visits (w/o click through)
• Time on site
• Sales or subscriptions
• Intermediate steps in purchase process

Question 37

Ethical considerations for research:

Answer 37

• Obtain informed consent:
  By ensuring that the were provided enough information, ensuring that they fully understand the study and able to ask questions, ability to decline or withdraw (voluntariness).
• Protect vulnerable populations: Children, prisoners, people with impaired cognitive capacity.

Question 38

Moderator:

Answer 38

Variable that affects the direction and/or strength of the relation between independent and dependent variables.