Week 8: Experimental and Quasi-Experimental Research Methods Flashcards
What is the experimental research strategy (aka experimental method)?
One variable is manipulated while another variable is observed and measured. To establish a cause-and-effect relationship between the two variables, an experiment attempts to control all other variables to prevent them from influencing the results.
What is an experiment?
Attempts to show that changing the value of one variable causes changes to occur in the second variable.
What are the two characteristics in the experimental method that differentiate experiments from other types of research studies?
Manipulation and Control
What is manipulation?
The researcher manipulates one variable by changing its value from one level to another. In the Polman et al. (2008) experiment examining the effect of violence in video games (Figure 1.5), the researchers manipulate the amount of violence by giving one group of boys a violent game to play and giving the other group a nonviolent game. A second variable is observed (measured) to determine whether the manipulation causes changes to occur.
What is control?
The researcher must exercise control over the research situation to ensure that other, extraneous variables do not influence the relationship being examined.
What are the two general categories of variables that researchers must consider?
Participant Variables and Environmental Variables
What are the three basic techniques researchers typically use to control other variables?
- Random assignment (equal chance of being assigned to a group) 2. Matching (e.g ensuring every group holds exactly 60% females and 40% males) 3. Holding them constant (e.g just using 10 year olds in the study)
True or false: in an experiment BOTH variables are measured?
FALSE. Only one is measured - the Dependent variable. The Independent variable is manipulated by the researcher.
What is the difference between a correlation study and an experimental study regarding variables?
In a correlation study, BOTH variables are measured (we obtain data consisting of two separate scores), where as in an experimental study, only the dependent variable is measured.
What is the control condition?
Individuals in a control condition do not receive the experimental treatment. Instead, they either receive no treatment or they receive a neutral, placebo treat-ment. The purpose of a control condition is to provide a baseline for comparison with the experimental condition
What is the experimental condition?
Individuals in the experimental condition do receive the experimental treatment.
The independent variable has to have at least ___ values (groups or conditions) to be considered a “variable”?
TWO
What is a “between-subjects” design?
An experiment in which different subjects are assigned to each group.
The control group is the group that serves as the ____?
Baseline, or “standard” condition
The experimental group is the group that receives some level of the ____?
Independent Variable
We describe the two groups in an experiment as the ______ and _____ groups?
Experimental and Control
True or false: an experiment may involve the use of two experimental groups and no control group
TRUE.
How do we control who is in the study?
By (ideally) random sampling, to gather a sample that is representative of the population about whom we are trying to generalize. Further, use random assignment of subjects to two conditions (to make groups equivalent)
What are the two major problems with successfully conducting an experiment PROVING there is a causal relationship between smoking and cancer?
Firstly, it is completely unethical to assign participants to the experimental group by which they smoke for a period of time Secondly, in order for the experiment to be successful we would have to control many other aspects of their life for the period of time needed to conduct the study.
What is the posttest-only control group design?
An experimental design in which the dependent variable is measured after the manipulation of the independent variable.
What is a pretest/posttest control group design?
An experimental design in which the dependent variable is measured both before and after manipulation of the independent variable.
How many subjects per condition is considered adequate?
20 to 30 subjects per condition
What are the disadvantages of multiple testings?
There is more opportunity for an experimenter to influence the subjects.
What is a Solomon four-group design?
A design with four groups that is a combination of the posttest only control group design and the pretest/posttest control group design. By merging the two designs, the disadvantages of each individual design are minimized, and the advantages are maximised.
If a researcher fails to control for something, then the study is open to ____?
Confounds
What are confounds?
An uncontrolled extraneous variable or flaw in an experiment.
What are the problems with confounds in an experiment?
If a study is confounded, it is impossible to say whether changes in the dependent variable were caused by the independent variable or by the uncontrolled extraneous variable.
Why is maximising control with human subjects so difficult for psychologists?
Because humans have many dimensions - personality, intelligence, rearing issues. There is great variability, so psychologists need to be concerned about preexisting differences.
To avoid extraneous variables, we need to maximise the ____ validity of a study?
Internal
What is the “internal validity”?
The extent to which the results of an experiment can be attributed to the manipulation of the independent variable rather than to some confounding variable.
What are the threats to internal validity?
Nonequivalent Control Group (when random sampling and random assignment are not used - may lead to important differences between the subjects assigned to the experimental and control groups) History effect Maturation Testing effect Regression to the mean Instrumentation effect Mortality (attrition) Diffusion of treatment Experimenter Effect
Confounds are most problematic to _____ designs?
Nonexperimental
What is the history effect?
Changes in the dependent variable may be due to outside events that take place during the course of the study. To minimise, use an equivalent control group.
What is the maturation effect?
Changes in the dependent variable may be due to subjects maturing (growing older) during the course of the study. To minimise, use an equivalent control group.
What is the testing effect?
Changes in the dependent variable may be due to participants being tested repeatedly and getting either better or worse because of these repeated testings. To minimise, use an equivalent control group.
What is the regression to the mean?
Subjects who are selected for a study because they are extreme (either high or low) on some variable may regress toward the mean and be less extreme at a later testing. To minimise, use an equivalent group of subjects with extreme scores.
What is the instrumentation effect?
Changes in the dependent variable may be due to changes in the measuring device, either human or machine. (particularly problematic when humans are the measurement device - ie qualitative research - perhaps human is fatigued) To minimise, use an equivalent control group.
What is mortality (attrition)?
Differential attrition or dropout in the experimental and control groups may lead to inequality between the groups. To minimise, monitor for differential loss of subjects in experimental and control groups.
What is diffusion of treatment?
Changes in the behaviors or responses of participants may be due to information they have received from others participating in the study. e.g students talking to other students about the study, effecting the study To minimize, test subjects all at once or as close together in time as possible.
What is the experimenter effect?
A threat to internal validity in which the experimenter, consciously or unconsciously, affects the results of the study. AKA experimenter bias or expectancy effects To minimise, use a double-blind or single-blind procedure.
How do researchers choose to combat experimenter effects?
By conducting blind experiments.
What is a single-blind experiment?
An experimental procedure in which either the subjects or the experimenters are blind to the manipulation being made. The experimenter being blind in a s ingle-blind experiment would help to combat experimenter effects.
What is a double-blind experiment?
An experimental procedure in which neither the experimenter nor the subject knows the condition to which each subject has been assigned; both parties are blind to the manipulation.
What is subject effect?
A threat to internal validity in which the subject, consciously or unconsciously, affects the results of the study.
What are demand characteristics?
A type of subject effect in which subjects try to guess what characteristics the experimenter is in effect “demanding.”
What is the Hawthorne effect?
A type of reactivity in which subjects improve or change an aspect of their behaviour because they know that they are being studied, rather than in response to the experimental manipulation.
What is the placebo group?
A group or condition in which subjects believe they are receiving treatment but are not.
What is placebo?
An inert substance that subjects believe is a treatment.
What are floor and ceiling effects?
When researchers have chosen measures for the dependent variable that are NOT sensitive enough to detect differences between groups. To minimise: ensure that the measuring instrument is reliable and valid before beginning the study.
What is the floor effect?
A limitation of the measuring instrument that decreases its capability to differentiate between scores at the bottom of the scale.
What is the ceiling effect?
A limitation of the measuring instrument that decreases its capability to differentiate between scores at the top of the scale.
What test can be used to determine if the measurement scale is sensitive enough?
A pretest. A pretest should show subjects receiving difference scores on the dependent measure. If subject are scoring about the same (either very low or very high), then a floor and ceiling effect may be present.
While internal validity is important for a study to be meaningful, external validity is also important. What is external validity?
The extent to which the results of an experiment can be generalized
How can generalisation to the population being studied be achieved?
By randomly sampling subjects
What is the college sophomore problem?
An external validity problem that results from using mainly college sophomores as subjects in research studies. (especially in psychology research) Conclusions are based on studies of young people with late adolescent mentality who are still developing self-identities and attitudes.
Why do we need to exercise some caution when generalising from the laboratory setting to the real world?
Because of the artificiality criticism. The labratory is not the real world!
How can we address the artificiality criticism and generalisation issues? (three ways)
By using conceptual replication, and systematic replication.
What is “exact replication”?
Repeating a study in exactly the same manner - using the same means of manipulating and measuring the variables as in the original study.
What is “conceptual replication”?
A study based on another study that uses different methods, a different manipulation, or a different measure.
What is “systematic replication”?
A study that varies from an original study in one systematic way—for example, by using a different number or type of subjects, a different setting, or more levels of the independent variable. Systematically changes one thing at a time and observes the effect, if any, on the results.
What is the difference between between-subjects and within subjects designs?
Between subjects are where the subjects in each condition are different (diff subjects assigned to the control and experimental condition) Within subjects (aka correlated group designs) are where the subjects in the experimental and control group are related (e.g same 20 participants act as control and experiment condition)
What are correlated-groups design?
An experimental design in which the subjects in the experiment a land control groups are related in some way
What is a within-subjects design? (aka repeated measures)
A type of correlated-groups design in which the same subjects are used in each condition
What is the most important benefit of a within-subjects design?
Within-subjects design increase statistical power, because the same individuals participate in multiple conditions, with individual differences being minimized.
What is variability due to individual differences?
When in a between-subjects design, the difference in performance between two groups is due to the fact individuals in one group are different from individuals in the other group.
In a within-subjects design, where does the variability between the two conditions (groups) come from?
The manipulation of the independent variable because the same participants produce both groups of scores. The differences between groups cannot be CAUSED by individual differences because the scores in both conditions come from the SAME person.
True or false: there can be variability due to individual differences in within-subjects design?
FALSE. There cannot be. The scores in both conditions come from the same person.
What are the advantages and weaknesses in within-subjects design?
Advantages: -Statistical Power -Takes less time -More cost effective Disadvantages: -Open to the most confounds -Inter validity is a concern -Order effects are problematic -Multiple testing - practice and fatigue effects are common
What are “order effect” problems?
A problem for within-subjects designs in which the order of the conditions has an effect on the dependent variable
What is “counterbalancing”?
A mechanism for controlling order effects either by including all orders of treatment presentation or by randomly determining the order for each subject.
How can testing effects in a within-subjects design be resolved?
Through counterbalancing
What is a “latin square”?
A counter-balancing technique to control for order effects without using all possible orders. EG design with four conditions would use four orders, not 24.
What is the “carry over” effect?
A type of testing effect often present in within-subjects designs in which subjects “carry” something with them from one condition to another.
What are “demand characteristics?”
The information participants infer about what the researcher wants.
Where possible, psychologists will choose which type of design?
Within-subjects
What is the correlated-groups “matched-subjects” design?
A type of correlated-groups design in which subjects are matched between conditions on variable(s) that the researcher believes is (are) relevant to the study
Why do we not used a within-subjects design when it is so similar to a matched-subjects design?
Because of carryover effects. Participating in one condition changes the subjects to such an extent that cannot participate in the second condition. (eg drug research)
What is one of the biggest weaknesses of a matched-subject design?
Mortality and matching itself. It can be difficult to find a participant who matches exactly another participant on a specific variable.
What is a nonmanipulated independent variable (aka subject variable)?
The independent variable in a quasi-experimental design in which subjects are not randomly assigned to conditions but rather come to the study as members of each condition.
What are subject variables?
Characteristic of a participant that cannot be changed
If a study is designed to assess differences in individuals on a subject variable, what type of study is it by default?
A quasi-experiment because it uses a nonmanipulated IV as subjects are not randomly assigned to conditions.
One type of quasi-experimental design is the single-group posttest-only design. What is this?
The simplest quasi-experimental design in which a single group of participants is given a treatment and then tested. No comparison group.
One type of quasi-experimental design is the single-group prettest/posttest design. What is this?
A design in which a single group of subjects takes a pretest, receives some treatment, and then takes a posttest measure. So measurements are taking twice and it is an improvement over the single-group posttest only. No comparison group.
One type of quasi-experimental design is the single-group time-series design. What is this?
A design in which a single group of subjects is measured repeatedly before and after a treatment.
One type of quasi-experimental design is the nonequivalent control group posttest-only design. What is this?
A design in which at least two nonequivalent groups are given a treatment and then a posttest measure. Similar to the simplest quasi-experimental design except there is a comparison group!
One type of quasi-experimental design is the nonequivalent control group pretest/posttest design. What is this?
A design in which at least two nonequivalent groups are given a pretest, then a treatment, and then a posttest measure.
What is a multiple-group time-series design?
A design in which a series of measures are taken on two or more groups both before and after a treatment. This is an extension of the nonequivalent control group pretest/posttest design.
What are developmental designs?
A type of quasi-experimental design where age is used as a nonmanipulated independent variable.
What are the two basic developmental designs?
Cross-sectional designs and longitudinal designs
What is a cross-sectional design?
A type of developmental design in which subjects of different ages are studied at the same time.
What is a cohort?
A group of individuals born at about the same time.
What is the cohort effect?
A generational effect in a study that occurs when the era in which individuals are born affects how they respond in the study.
What are the advantages/disadvantages to cross-sectional design?
Pros- quick data collection Cons- things like the cohort-effect influencing the DV, not the age itself.
What is a longitudinal design?
A type of developmental design in which the same subjects are studied repeatedly over time as they age.
What are the advantages/disadvantages of longitudinal designs?
Adv: Elimination of cohort effects as same subjects studies over period of time Diadv: Expensive and more time consuming than cross-sectional. Attrition problems.
What are sequential designs?
A developmental design that is a combination of the cross-sectional and longitudinal designs
What are single-case designs?
A design in which only one participant is used.
What is a small-n design?
A design in which only a few subjects are studied.
What is an advantage to the single-case design over group designs?
They are better at demonstrating a reliable effect of an independent variable. . Also error variance is reduced. Don’t need to look at group mean to conduct statistical analyses.
