Readings Flashcards
Difference between quasi-experimental and experimental design
Although the independent variable is manipulated, participants are not randomly assigned to conditions or orders of conditions. Because the independent variable is manipulated before the dependent variable is measured, quasi-experimental research eliminates the directionality problem (temporal precedence). But because participants are not randomly assigned— making it likely that there are other differences between conditions—quasi-experimental research does not eliminate the problem of confounding variables. In terms of internal validity, therefore, quasi-experiments are generally somewhere between correlational studies and true experiments.
Non-equivalent Groups Design
between-subjects experiments are randomly assigned to conditions, the resulting groups are likely to be quite similar. In fact, researchers consider them to be equivalent. When participants are not randomly assigned to conditions, however, the resulting groups are likely to be dissimilar in some ways. For this reason, researchers consider them to be non-equivalent. A non-equivalent groups design, then, is a between-subjects design in which participants have not been randomly assigned to conditions.
when people are not randomly assigned to conditions, they are not equivalent and can have important differences between them.
For example, in a teaching non-equivalent group study confounding variables can include teachers’ styles, and even the classroom environments, might be very different and might cause different levels of achievement or motivation among the students.
Researchers do their best to make the groups as similar as possible. for example, sampling from the same school, choosing two classes with similar academic scores, classes with same-sex teachers and same teaching styles. These steps increase the internal validity of the study because it would eliminate some of the most important confounding variables. But without true random assignment of the students to conditions, there remains the possibility of other important confounding variables that the researcher was not able to control.
Pretest-Posttest Design
scores are measured before and after intervention but since the order of conditions is not counterbalanced it is typically not possible for a participant to be tested in the treatment condition first and then in an “untreated” control condition.
If the average posttest score is better than the average pretest score, then it makes sense to conclude that the treatment might be responsible for the improvement. Unfortunately, one often cannot conclude this with a high degree of certainty because there may be other explanations for why the posttest scores are better. One category of alternative explanations goes under the name of history. Other things might have happened between the pretest and the posttest. Perhaps an anti-drug program aired on television and many of the students watched it, or perhaps a celebrity died of a drug overdose and many of the students heard about it. Another category of alternative explanations goes under the name of maturation. Participants might have changed between the pretest and the posttest in ways that they were going to anyway because they are growing and learning. If it were a yearlong program, participants might become less impulsive or better reasoners and this might be responsible for the change.
Another alternative explanation for a change in the dependent variable in a pretest-posttest design is regression to the mean. This refers to the statistical fact that an individual who scores extremely on a variable on one occasion will tend to score less extremely on the next occasion. For example, a cricketer with a long-term batting average of 15 who scores 150 in an innings will almost certainly score lower in her next innings. Her score will “regress” toward her mean (average) score of 15. Regression to the mean can be a problem when participants are selected for further study because of their extreme scores. Imagine, for example, that only students who scored especially low on a test of fractions are given a special training program and then retested. Regression to the mean all but guarantees that their scores will be higher even if the training program has no effect. A closely related concept—and an extremely important one in psychological research—is spontaneous remission. This is the tendency for many medical and psychological problems to improve over time without any form of treatment. The common cold is a good example. If one were to measure symptom severity in 100 common cold sufferers today, give them a bowl of chicken soup every day, and then measure their symptom severity again in a week, they would probably be much improved. This does not mean that the chicken soup was responsible for the improvement, however, because they would have been much improved without any treatment at all. The same is true of many psychological problems. A group of severely depressed people today is likely to be less depressed on average in 6 months. In reviewing the results of several studies of treatments for depression, researchers Michael Posternak and Ivan Miller found that participants in wait-list control conditions improved an average of 10 to 15% before they received any treatment at all (Posternak & Miller, 2001)[2]. Thus one must generally be very cautious about inferring causality from pretest-posttest designs.
Interrupted Time Series Design
A variant of the pretest-posttest design is the interrupted time-series design. A time series is a set of measurements taken at intervals over a period of time. For example, a manufacturing company might measure its workers’ productivity each week for a year. In an interrupted time series-design, a time series like this one is “interrupted” by a treatment. In one classic example, the treatment was the reduction of the work shifts in a factory from 10 hours to 8 hours (Cook & Campbell, 1979)[1]. Because productivity increased rather quickly after the shortening of the work shifts, and because it remained elevated for many months afterwards, the researcher concluded that the shortening of the shifts caused the increase in productivity. Notice that the interrupted time-series design is like a pretest- posttest design in that it includes measurements of the dependent variable both before and after the treatment. It is unlike the pretest- posttest design, however, in that it includes multiple pretest and posttest measurements.
This figure also illustrates an advantage of the interrupted time-series design over a simpler pretest-posttest design. If there had been only one measurement of absences before the treatment at Week 7 and one afterwards at Week 8, then it would have looked as though the treatment were responsible for the reduction. The multiple measurements both before and after the treatment suggest that the reduction between Weeks 7 and 8 is nothing more than normal week-to-week variation.
Combination Designs
A type of quasi-experimental design that is generally better than either the non-equivalent groups design or the pretest-posttest design is one that combines elements of both. There is a treatment group that is given a pretest, receives a treatment, and then is given a posttest. But at the same time there is a control group that is given a pretest, does not receive the treatment, and then is given a posttest. The question, then, is not simply whether participants who receive the treatment improve but whether they improve more than participants who do not receive the treatment.
Imagine, for example, that students in one school are given a pretest on their attitudes toward drugs, then are exposed to an anti- drug program, and finally are given a posttest. Students in a similar school are given the pretest, not exposed to an anti-drug program, and finally are given a posttest. Again, if students in the treatment condition become more negative toward drugs, this change in attitude could be an effect of the treatment, but it could also be a matter of history or maturation. If it really is an effect of the treatment, then students in the treatment condition should become more negative than students in the control condition. But if it is a matter of history (e.g., news of a celebrity drug overdose) or maturation (e.g., improved reasoning), then students in the two conditions would be likely to show similar amounts of change. This type of design does not completely eliminate the possibility of confounding variables, however. Something could occur at one of the schools but not the other (e.g., a student drug overdose), so students at the first school would be affected by it while students at the other school would not.
Finally, if participants in this kind of design are randomly assigned to conditions, it becomes a true experiment rather than a quasi experiment.
Key Takeaways
Quasi-experimental research involves the manipulation of an independent variable without the random assignment of participants to conditions or orders of conditions. Among the important types are non-equivalent groups designs, pretest- posttest, and interrupted time-series designs.
Quasi-experimental research eliminates the directionality problem because it involves the manipulation of the independent variable. It does not eliminate the problem of confounding variables, however, because it does not involve random assignment to conditions. For these reasons, quasi-experimental research is generally higher in internal validity than correlational studies but lower than true experiments.
ABAB Reversal Design
Positive reinforcement of good behaviour and ignoring negative behaviour (differential reinforcement) is a good ABAB reversal design which reduces disruptive behaviour
single subject design
what is it?
what is it not?
Focuses on individuals, with a small n
Single-subject research is a type of quantitative research that involves studying in detail the behaviour of each of a small number of participants. Note that the term single-subject does not mean that only one participant is studied; it is more typical for there to be somewhere between two and 10 participants. (This is why single-subject research designs are sometimes called small-n designs, where n is the statistical symbol for the sample size.)
What is it not?
Before continuing, it is important to distinguish single-subject research from two other approaches, both of which involve studying in detail a small number of participants. One is qualitative research, which focuses on understanding people’s subjective experience by collecting relatively unstructured data (e.g., detailed interviews) and analysing those data using narrative rather than quantitative techniques. Single-subject research, in contrast, focuses on understanding objective behaviour through experimental manipulation and control, collecting highly structured data, and analysing those data quantitatively.
It is also important to distinguish single-subject research from case studies. A case study is a detailed description of an individual, which can include both qualitative and quantitative analyses. (Case studies that include only qualitative analyses can be considered a type of qualitative research.) i.e., little albert study or Sigmund Freuds work. Case studies can be useful for suggesting new research questions and for illustrating general principles. They can also help researchers understand rare phenomena, such as the effects of damage to a specific part of the human brain. As a general rule, however, case studies cannot substitute for carefully designed group or single-subject research studies. One reason is that case studies usually do not allow researchers to determine whether specific events are causally related, or even related at all. For example, if a patient is described in a case study as having been sexually abused as a child and then as having developed an eating disorder as a teenager, there is no way to determine whether these two events had anything to do with each other. A second reason is that an individual case can always be unusual in some way and therefore be unrepresentative of people more generally. Thus case studies have serious problems with both internal and external validity.
Assumptions of Single-Subject Research
Again, single-subject research involves studying a small number of participants and focusing intensively on the behaviour of each one.
There are several important assumptions underlying single-subject research:
- It is important to focus intensively on the
behaviour of individual participants:
• Group research can hide individual
differences and generate results that do
not represent the behaviour of any
individual
• Single-subject designs can identify
individual differences - Sometimes it is the behaviour of a
particular individual that is primarily of
interest:
• For example, changing the behaviour of
a disruptive student.
• Using single-subject design provides
more direct and effective information
than group-means analysis when we
are interested in a specific behaviour in
a participant.
3. It is important to discover causal
relationships through the manipulation of
an independent variable, the careful
measurement of a dependent variable,
and the control of extraneous variables.
• Single-subject designs are experimental
designs with good internal validity.
• Dependant variable is measures
multiple times (baseline-intervention-
baseline; effect is introduced and
removed to provide strong evidence
that behaviour change is due to IV and
not extraneous variables).
- It is important to study strong effects that
have social importance.
• Minimizing or removing harmful
behaviours (i.e., treatments)
• Social validity (real-world applications)
Who Uses Single-Subject Research?
• Herman Ebbinghaus’s research on memory
and Ivan Pavlov’s research on classical
conditioning are other early examples.
• B. F. Skinner clarified many of the
assumptions underlying single-subject
research and refined many of its
techniques.
• This work was carried out primarily using
non-human subjects—mostly rats and
pigeons. This approach, which Skinner
called the experimental analysis of
behaviour—remains an important subfield
of psychology and continues to rely almost
exclusively on single-subject research.
• Many researchers were interested in using
this approach to conduct applied research
primarily with humans—a subfield now
called applied behaviour analysis (Baer,
Wolf, & Risley, 1968)[5]. Applied behaviour
analysis plays an especially important role
in contemporary research on
developmental disabilities, education,
organisational behaviour, and health,
among many other areas.
• Most contemporary single-subject research
is conducted from the behavioural
perspective, it can in principle be used to
address questions framed in terms of any
theoretical perspective.
Key Take-Aways
Single Subjects
Single-subject research—which involves
testing a small number of participants and
focusing intensively on the behaviour of
each individual—is an important alternative
to group research in psychology.
Single-subject studies must be
distinguished from case studies, in which
an individual case is described in detail.
Case studies can be useful for generating
new research questions, for studying rare
phenomena, and for illustrating general
principles. However, they cannot substitute
for carefully controlled experimental or
correlational studies because they are low
in internal and external validity.
Single-subject research has been around
since the beginning of the field of
psychology. Today it is most strongly
associated with the behavioural theoretical
perspective, but it can in principle be used
to study behaviour from any perspective.
General Features of Single-Subject Designs
Common Features in Single Subject Designs:
1. First, the dependent variable (represented
on the y-axis of the graph) is measured
repeatedly over time (represented by the
x-axis) at regular intervals.
- Second, the study is divided into distinct
phases, and the participant is tested under
one condition per phase. The conditions
are often designated by capital letters: A,
B, C, and so on. Thus Figure 10.2
represents a design in which the
participant was tested first in one
condition (A), then tested in another
condition (B), and finally retested in the
original condition (A). (This is called a
reversal design and will be discussed in
more detail shortly.) - Another important aspect of single-subject
research is that the change from one
condition to the next does not usually
occur after a fixed amount of time or
number of observations. Instead, it
depends on the participant’s behaviour.
Specifically, the researcher waits until the
participant’s behaviour in one condition
becomes fairly consistent from
observation to observation before
changing conditions. This is sometimes
referred to as the steady state strategy
(Sidman, 1960)[1]. The idea is that when
the dependent variable has reached a
steady state, then any change across
conditions will be relatively easy to detect.
Recall that we encountered this same
principle when discussing experimental
research more generally. The effect of an
independent variable is easier to detect
when the “noise” in the data is minimised.
Reversal Designs
The most basic single-subject research design is the reversal design, also called the ABA design. During the first phase, A, a baseline is established for the dependent variable. This is the level of responding before any treatment is introduced, and therefore the baseline phase is a kind of control condition. When steady state responding is reached, phase B begins as the researcher introduces the treatment. There may be a period of adjustment to the treatment during which the behaviour of interest becomes more variable and begins to increase or decrease. Again, the researcher waits until that dependent variable reaches a steady state so that it is clear whether and how much it has changed. Finally, the researcher removes the treatment and again waits until the dependent variable reaches a steady state. This basic reversal design can also be extended with the reintroduction of the treatment (ABAB), another return to baseline (ABABA), and so on.
Why is the reversal—the removal of the treatment—considered to be necessary in this type of design? Why use an ABA design, for example, rather than a simpler AB design? Notice that an AB design is essentially an interrupted time-series design applied to an individual participant. Recall that one problem with that design is that if the dependent variable changes after the treatment is introduced, it is not always clear that the treatment was responsible for the change. It is possible that something else changed at around the same time and that this extraneous variable is responsible for the change in the dependent variable. But if the dependent variable changes with the introduction of the treatment and then changes back with the removal of the treatment (assuming that the treatment does not create a permanent effect), it is much clearer that the treatment (and removal of the treatment) is the cause. In other words, the reversal greatly increases the internal validity of the study.
There are close relatives of the basic reversal design that allow for the evaluation of more than one treatment. In a multiple- treatment reversal design, a baseline phase is followed by separate phases in which different treatments are introduced. For example, a researcher might establish a baseline of studying behaviour for a disruptive student (A), then introduce a treatment involving positive attention from the teacher (B), and then switch to a treatment involving mild punishment for not studying (C). The participant could then be returned to a baseline phase before reintroducing each treatment—perhaps in the reverse order as a way of controlling for carry-over effects. This particular multiple-treatment reversal design could also be referred to as an ABCACB design.
In an alternating treatments design, two or more treatments are alternated relatively quickly on a regular schedule. For example, positive attention for studying could be used one day and mild punishment for not studying the next, and so on. Or one treatment could be implemented in the morning and another in the afternoon. The alternating treatments design can be a quick and effective way of comparing treatments, but only when the treatments are fast acting.
Multiple-Baseline Designs
There are two potential problems with the reversal design—both of which have to do with the removal of the treatment. One is that if a treatment is working, it may be unethical to remove it. For example, if a treatment seemed to reduce the incidence of self-injury in child with a developmental disability, it would be unethical to remove that treatment just to show that the incidence of self-injury increases. The second problem is that the dependent variable may not return to baseline when the treatment is removed. For example, when positive attention for studying is removed, a student might continue to study at an increased rate. This could mean that the positive attention had a lasting effect on the student’s studying, which of course would be good. But it could also mean that the positive attention was not really the cause of the increased studying in the first place. Perhaps something else happened at about the same time as the treatment—for example, the student’s parents might have started rewarding him for good grades.
One solution to these problems is to use a multiple-baseline design, which is represented in Figure 10.4. In one version of the design, a baseline is established for each of several participants, and the treatment is then introduced for each one. In essence, each participant is tested in an AB design. The key to this design is that the treatment is introduced at a different time for each participant. The idea is that if the dependent variable changes when the treatment is introduced for one participant, it might be a coincidence. But if the dependent variable changes when the treatment is introduced for multiple participants—especially when the treatment is introduced at different times for the different participants—then it is extremely unlikely to be a coincidence.
Notice that if the researchers had only studied one school or if they had introduced the treatment at the same time at all three schools, then it would be unclear whether the reduction in aggressive behaviours was due to the bullying program or something else that happened at about the same time it was introduced (e.g., a holiday, a television program, a change in the weather). But with their multiple-baseline design, this kind of coincidence would have to happen three separate times—a very unlikely occurrence—to explain their results.
In another version of the multiple-baseline design, multiple baselines are established for the same participant but for different dependent variables, and the treatment is introduced at a different time for each dependent variable. Imagine, for example, a study on the effect of setting clear goals on the productivity of an office worker who has two primary tasks: making sales calls and writing reports. Baselines for both tasks could be established. For example, the researcher could measure the number of sales calls made and reports written by the worker each week for several weeks. Then the goal-setting treatment could be introduced for one of these tasks, and at a later time the same treatment could be introduced for the other task. The logic is the same as before. If productivity increases on one task after the treatment is introduced, it is unclear whether the treatment caused the increase. But if productivity increases on both tasks after the treatment is introduced—especially when the treatment is introduced at two different times—then it seems much clearer that the treatment was responsible.
In yet a third version of the multiple-baseline design, multiple baselines are established for the same participant but in different settings. For example, a baseline might be established for the amount of time a child spends reading during his free time at school and during his free time at home. Then a treatment such as positive attention might be introduced first at school and later at home. Again, if the dependent variable changes after the treatment is introduced in each setting, then this gives the researcher confidence that the treatment is, in fact, responsible for the change.
Data Analysis in Single-Subject Research
In addition to its focus on individual participants, single-subject research differs from group research in the way the data are typically analysed. As we have seen throughout the book, group research involves combining data across participants. Group data are described using statistics such as means, standard deviations, Pearson’s r, and so on to detect general patterns. Finally, inferential statistics are used to help decide whether the result for the sample is likely to generalise to the population. Single-subject research, by contrast, relies heavily on a very different approach called visual inspection. This means plotting individual participants’ data as shown throughout this chapter, looking carefully at those data, and making judgements about whether and to what extent the independent variable had an effect on the dependent variable. Inferential statistics are typically not used.
In visually inspecting their data, single-subject researchers take several factors into account. One of them is changes in the level of the dependent variable from condition to condition. If the dependent variable is much higher or much lower in one condition than another, this suggests that the treatment had an effect. A second factor is trend, which refers to gradual increases or decreases in the dependent variable across observations. If the dependent variable begins increasing or decreasing with a change in conditions, then again this suggests that the treatment had an effect. It can be especially telling when a trend changes directions—for example, when an unwanted behaviour is increasing during baseline but then begins to decrease with the introduction of the treatment. A third factor is latency, which is the time it takes for the dependent variable to begin changing after a change in conditions. In general, if a change in the dependent variable begins shortly after a change in conditions, this suggests that the treatment was responsible.
The results of single-subject research can also be analysed using statistical procedures—and this is becoming more common. There are many different approaches, and single-subject researchers continue to debate which are the most useful. One approach parallels what is typically done in group research. The mean and standard deviation of each participant’s responses under each condition are computed and compared, and inferential statistical tests such as the t test or analysis of variance are applied (Fisch, 2001)[3]. (Note that averaging across participants is less common.) Another approach is to compute the percentage of non-overlapping data (PND) for each participant (Scruggs & Mastropieri, 2001)[4]. This is the percentage of responses in the treatment condition that are more extreme than the most extreme response in a relevant control condition. In the study of Hall and his colleagues, for example, all measures of Robbie’s study time in the first treatment condition were greater than the highest measure in the first baseline, for a PND of 100%. The greater the PND, the stronger the treatment effect. Still, formal statistical approaches to data analysis in single-subject research are generally considered a supplement to visual inspection, not a replacement for it.
