10 Experiment Design Flashcards
What are the stages of preparing an experiment?
- Identifying your objective
- Formulate a hypothesis
- Choose the variables
- Choose the experimental design
- Choose the tasks
- Recruit participants
- Run the experiment
- Perform statistical tests on the data
- Analyze and interpret the results
What is an independent variable?
- Characteristics that are changed to produce different conditions
- Manipulated by the researcher
- Example: In a study examining the impact of different study techniques on exam scores, the independent variable might be the study technique (e.g., flashcards, summarization, or practice tests).
Changeable characteristics manipulated by the researcher.
Think “what I change.”
Example: In a study on study techniques and exam scores, it’s the method of study (flashcards, summarization).
What is an dependent variable?
- Used to test the hypothesis
- What is being measured?
- Outcome driven by the independent variable
- Example: Using the study technique example, the dependent variable would be the exam scores. The scores are expected to vary based on the different study techniques.
What’s being measured or tested.
Outcome influenced by the independent variable.
Think “what I measure.”
Example: In the study on study techniques, it’s the exam scores, varying with study methods.
What is an Control variable?
Kept constant to ensure experiment validity.
Think “what I keep the same.”
Example: In a plant growth experiment, sunlight, water, and soil type are controlled to isolate fertilizer effects.
What is an Confounding variable?
Can affect the experiment outcome.
Not the dependent variable.
Think “what might mess things up.”
Example: In a drug study, if age influences both drug and outcome, age becomes a confounding variable.
Give an example of an experiment where you use a ratio data type to measure a dependent variable. What statistical test should you use? Why?
An example is measuring participants’ weight change before and after an exercise program. Use a paired t-test as weight data is continuous and ratio, ensuring the same participants are compared before and after, making it suitable for paired data analysis.
For this hypothesis: “There is no difference in the time required to locate an item in an online store between novice users and experienced users”, is it appropriate to employ a within-group experimental design? Why?
No, it is not appropriate for a within-group experimental design because it compares two distinct groups (novice and experienced users). A between-groups design, comparing separate groups, is more suitable to test differences between these two user categories.
What is within-subjects design? + and -?
Within-Subjects looks at how the same individuals respond to different conditions.
Imagine you have a group of people, and you make each person try different things or conditions.
You’re comparing how each person does when they experience different situations.
It’s like giving one person a taste of different flavors and seeing which one they like the most.
+ Need fewer participants
- Carryover effects, participation in one condition may effect performance in other conditions
- Fatigue effects, tired and fatigued after taking part in the first condition, negatively affects the result of condition
What is between-subjects design? + and -?
Between-Subjects compares different groups to see which one does better in a specific condition.
Now, picture having two separate groups of people, and each group only tries one thing or condition.
You’re comparing how one entire group performs compared to another group.
It’s like having one group taste only chocolate ice cream and another group taste only vanilla, and then deciding which group overall liked their ice cream more.
+ lowers the chances of participants having carryover effects
- Need many participants
what is Internal Validity
Question: Is the experiment solid?
Definition: Does the experiment accurately show the relationship between what you’re changing (independent variable) and what you’re measuring (dependent variable), without interference from other factors?
Example: If you’re testing a new teaching method’s impact on student performance, internal validity is at risk if other things like changes in the classroom aren’t controlled.
what is External Validity
Question: Does the result apply to different situations or people?
Definition: How well do the findings of a study apply to various scenarios, groups, or times?
Example: If a study is done with college students, external validity is compromised if you assume the results apply to everyone of all ages and backgrounds.
what is Construct Validity
Question: Does the result match what we know theoretically?
Definition: Does the experiment truly measure what it’s supposed to measure according to theory?
Example: In an intelligence study, construct validity is good if the test measures intelligence and not other unrelated factors.
what is Content Validity
Question: Does the test cover all it’s supposed to measure?
Definition: How well does a test or survey represent everything it’s meant to measure?
Example: For a high school math test, content validity is good if it covers a representative sample of high school math topics.
what is Ecological Validity
Question: Does the result apply to real life?
Example: Testing a blind-friendly interface with normal users isn’t ecologically valid. For stress studies, if it’s done in a lab, you need to check if the stress findings also apply to real-world situations like work or home.
what is parametric data?
type, distribution assumption, tests
Parametric is for numbers with assumptions.
Type: Interval or ratio (think numbers with meaningful intervals).
Distribution Assumption: Assumes a specific distribution, usually normal.
Tests: Parametric tests are powerful but need data to meet assumptions.
If your response time data looks like this:
Group A: 10, 12, 11, 13, 10
Group B: 15, 14, 16, 13, 15
This data is numerical and has a clear, meaningful order. If the data meets the assumptions (like being normally distributed), you might use a parametric test like the t-test.
what is non-parametric data?
type, distribution assumption, tests
Non-Parametric is for categories without assumptions.
Type: Ordinal or nominal (think categories or rankings).
Distribution Assumption: No specific distribution assumption.
Tests: Non-parametric tests are robust and work well even when assumptions are violated or with non-numerical data.
If your response time data looks like this:
Group A: Ranked as 1st, 2nd, 3rd, 1st, 2nd
Group B: Ranked as 3rd, 1st, 2nd, 1st, 3rd
Or if your data is not normally distributed.
This data is ordinal (ranking) or not meeting normal distribution assumptions. In this case, you might choose a non-parametric test like the Mann-Whitney U test (for independent samples) or Wilcoxon signed-rank test (for paired samples).
If the p-value is less than your chosen significance level (commonly 0.05), you ___
If the p-value is less than your chosen significance level (commonly 0.05), you reject the null hypothesis.
If the p-value is greater than your significance level, you ___
If the p-value is greater than your significance level, you fail to reject the null hypothesis.
