Experimental Design Flashcards
What are the different experimental variables?
Independent Variable,
Dependent Variable,
Control Variable and
Confounding Variable
What is an independent variable?
An independent variable is the variable that the researcher manipulates or changes to observe the effect on the dependent variable. It is the presumed cause.
What is a dependent variable?
A dependent variable is the variable that is measured in an experiment. It is what is affected by the independent variable during an experiment. It is the presumed effect.
What is a control variable?
Control variables are variables that are kept constant to ensure that any changes in the dependent variable are due to the manipulation of the independent variable, rather than some extraneous factor.
What is a confounding variable?
A confounding variable is an extraneous factor (not IV) that is not the focus of the study but can affect the results (DV). This variation can cause the researcher to inaccurately attribute the effect on the dependent variable to the independent variable, rather than to the confounding variable.
What is the difference between within-subjects and between-subjects experimental design?
Firstly, one difference is in terms of participants required. Within-subject would require lesser participants as they use the same participants throughout the experiments, whereas Between-subject would require more participants as different groups of participants will be used to test different conditions of the independent variables.
Secondly, the level of variability in the data differs between the two designs. Within-subject design typically exhibits less variability in terms of individual differences because the same participants are exposed to all conditions. This means that individual characteristics that could affect outcomes are constant across all treatments. Therefore, any change in the dependent variable is more likely to be attributed to the change in independent variable, rather than individual differences.
Thirdly, the results collected from Within Subject Design are subject to Order Effects, which may be Practice Effects, Fatigue Effects or Carryover effects. This means that the results collected may not be accurate. Between Subject Design on the other hand, is not subject to Order effects since each group of participants only test one condition, once.
Lastly, there is also difference in terms of time spent per participant. For Within Subject Design, the total time spent per participant is longer as they have to go through all the conditions. On the other hand, for Between Subject Design, the total time spent per participant is significantly lesser as they only have to go through one condition.
What are order effects?
Order effects in experimental designs refer to changes in participants’ responses caused by the order in which experimental conditions are experienced rather than the content of those conditions themselves. These effects can arise when participants are subjected to multiple treatments or conditions one after the other. There are several types of order effects:
Practice Effects (or Learning Effects): These occur when participants get better at a task over time simply because they have done it more than once. For example, if a participant has to solve puzzles in two conditions, they might perform better in the second condition just because they’ve had practice from the first, not necessarily because the second condition is easier or more effective.
Fatigue Effects: Participants may perform worse over time because they become tired or bored. In a long experiment with multiple tasks, a participant’s performance might decline simply due to exhaustion or loss of interest, which can be mistaken for an effect of the experimental conditions.
Carryover Effects: These happen when the effect of one condition carries over to affect performance in subsequent conditions. For example, if a participant drinks caffeine in one condition and then is tested on memory in the next, the caffeine may boost their memory performance, making it hard to tell if any change is due to the experimental manipulation or the lingering effects of caffeine.
Researchers try to control for order effects through counterbalancing (varying the order of conditions for different participants) or by implementing washout periods between conditions to allow any carryover effects to dissipate. In between-subjects designs, where each participant experiences only one condition, order effects are not a concern because there’s no sequence of treatments for each individual.
What are the advantages and disadvantages of Within Subject Design?
Advantages:
Subject to less variability in results collected due to lesser individual differences.
Lesser participants required
Disadvantages:
Subject to order effects including practice effects, fatigue effects, carryover effects, which may skew the results.
What are the advantages and disadvantages of Between Subject Design?
Advantages:
Results are not subjected to order effects as groups of participants only go through one condition of the independent variable
Multiple independent variables can be tested simultaneously, saving time.
Each participant only needs to be tested once so easier to implement this method of testing
Disadvantages:
More participants required
Higher individual variability and more individual differences which will skew the accuracy of the results
What is the ladder of validity?
It is a conceptual framework used in the field of research design, particularly within social sciences to evaluate the validity of an experiment or study. It is a metaphorical ladder with different rungs representing varying levels of evidence strength or validity. As one moves up the ladder, the level of validity and confidence in the causality of the findings increases. This framework is often used to assess the credibility of research findings and to guide the design of more robust experiments
What are the stages of the validity ladder?
Content Validity
Construct Validity
Internal Validity
External Validity
Ecological Validity
What is content validity?
This is the base of the ladder, ensuring that the test or measurement actually assesses the variable of interest. If your experiment is intended to measure the effectiveness of a new interface, content validity would ensure that your measurements truly reflect the interface’s usability and not something else
- Does the result reflect a variable of interest?
- Is the result interesting or a known fact?
What is construct validity?
This rung concerns whether the results align with theoretical concepts. It assesses if the constructs being measured truly represent what they are intended to, and if the outcomes of the experiment can be explained using these theoretical constructs. It is critical in ensuring that the experiment is not just measuring something, but measuring the right thing to say.
What is internal validity
Moving up the ladder, internal validity addresses whether the experimental setup allows you to make a causal inference. It asks if the changes in the dependent variable can confidently be attributed to the manipulation of the independent variable, without interference from other potential confounding factors.
What is external validity?
Higher up the ladder, external validity deals with the generalisability of the experiment’s results. It asks if the findings from your sample and experimental conditions can be applied to other contexts, populations, and times. A high external validity means the research findings can be broadly applied.
