Ch. 5 Flashcards
experiment
A type of study designed specifically to answer the question of whether there is a causal relationship between two variables.
independent variable
The variable the experimenter manipulates.
dependent variable
The variable the experimenter measures (it is the presumed effect).
conditions
The different levels of the independent variable to which participants are assigned.
control
Holding extraneous variables constant in order to separate the effect of the independent variable from the effect of the extraneous variables.
extraneous variables
Any variable other than the dependent and independent variable.
manipulate
Changing the level, or condition, of the independent variable systematically so that different groups of participants are exposed to different levels of that variable, or the same group of participants is exposed to different levels at different times.
Notice that the manipulation of an independent variable must involve the active intervention of the researcher.
Comparing groups of people who differ on the independent variable before the study begins is not the same as manipulating that variable.
there are many situations in which the independent variable cannot be manipulated for practical or ethical reasons and therefore an experiment is not possible.
single factor two-level design
An experiment design involving a single independent variable with two conditions.
single factor multi level design
When an experiment has one independent variable that is manipulated to produce more than two conditions.
Control of Extraneous Variables
Extraneous variables pose a problem because many of them are likely to have some effect on the dependent variable.
This influencing factor can make it difficult to separate the effect of the independent variable from the effects of the extraneous variables, which is why it is important to control extraneous variables by holding them constant.
Extraneous Variables as “Noise”
Extraneous variables make it difficult to detect the effect of the independent variable in two ways.
One is by adding variability or “noise” to the data.
Thus one reason researchers try to control extraneous variables is so their data look more like the idealized data in Table 5.1, which makes the effect of the independent variable easier to detect (although real data never look quite that good).
One way to control extraneous variables is to hold them constant.
This technique can mean holding situation or task variables constant by testing all participants in the same location, giving them identical instructions, treating them in the same way, and so on.
It can also mean holding participant variables constant.
The obvious downside to this approach is that it would lower the external validity of the study—in particular, the extent to which the results can be generalized beyond the people actually studied.
confounding variable
An extraneous variable that varies systematically with the independent variable, and thus confuses the effect of the independent variable with the effect of the extraneous one.
To confound means to confuse, and this effect is exactly why confounding variables are undesirable.
Because they differ systematically across conditions—just like the independent variable—they provide an alternative explanation for any observed difference in the dependent variable.
One way to avoid confounding variables is by holding extraneous variables constant.
treatment
Any intervention meant to change people’s behavior for the better.
This intervention includes psychotherapies and medical treatments for psychological disorders but also interventions designed to improve learning, promote conservation, reduce prejudice, and so on.
To determine whether a treatment works, participants are randomly assigned to either a treatment condition or control condition
If participants in the treatment condition end up better off than participants in the control condition then the researcher can conclude that the treatment works.
treatment condition
The condition in which participants receive the treatment.
control condition
The condition in which participants do not receive the treatment.
randomized clinical trial
An experiment that researches the effectiveness of psychotherapies and medical treatments.
no-treatment control condition
The condition in which participants receive no treatment whatsoever.
placebo
A simulated treatment that lacks any active ingredient or element that is hypothesized to make the treatment effective, but is otherwise identical to the treatment.
placebo effect
An effect that is due to the placebo rather than the treatment.
they are probably driven primarily by people’s expectations that they will improve.
Having the expectation to improve can result in reduced stress, anxiety, and depression, which can alter perceptions and even improve immune system functioning.
pose a serious problem for researchers who want to determine whether a treatment works.
placebo control condition
Condition in which the participants receive a placebo rather than the treatment.
placebo that looks much like the treatment but lacks the active ingredient or element thought to be responsible for the treatment’s effectiveness.
wait-list control condition
Condition in which participants are told that they will receive the treatment but must wait until the participants in the treatment condition have already received it.
Of course, the principle of informed consent requires that participants be told that they will be assigned to either a treatment or a placebo control condition—even though they cannot be told which until the experiment ends.
In many cases the participants who had been in the control condition are then offered an opportunity to have the real treatment.
This disclosure allows researchers to compare participants who have received the treatment with participants who are not currently receiving it but who still expect to improve (eventually).
A final solution to the problem of placebo effects
is to leave out the control condition completely and compare any new treatment with the best available alternative treatment.
between-subjects experiment
An experiment in which each participant is tested in only one condition.
This matching is a matter of controlling these extraneous participant variables across conditions so that they do not become confounding variables.
random assignment
Means using a random process to decide which participants are tested in which conditions.
Do not confuse random assignment with random sampling.
Random sampling is a method for selecting a sample from a population, and it is rarely used in psychological research.
Random assignment is a method for assigning participants in a sample to the different conditions,
random assignment should meet two criteria.
One is that each participant has an equal chance of being assigned to each condition (e.g., a 50% chance of being assigned to each of two conditions).
The second is that each participant is assigned to a condition independently of other participants.
Thus one way to assign participants to two conditions would be to flip a coin for each one.
If the coin lands heads, the participant is assigned to Condition A, and if it lands tails, the participant is assigned to Condition B.
For three conditions, one could use a computer to generate a random integer from 1 to 3 for each participant.
One problem with coin flipping and other strict procedures for random assignment is
that they are likely to result in unequal sample sizes in the different conditions.
Unequal sample sizes are generally not a serious problem, and you should never throw away data you have already collected to achieve equal sample sizes.
However, for a fixed number of participants, it is statistically most efficient to divide them into equal-sized groups.
It is standard practice, therefore, to use a kind of modified random assignment that keeps the number of participants in each group as similar as possible.
block randomization
All the conditions occur once in the sequence before any of them is repeated.
Within each of these “blocks,” the conditions occur in a random order.
Again, the sequence of conditions is usually generated before any participants are tested, and each new participant is assigned to the next condition in the sequence.
Random assignment is not guaranteed to control all extraneous variables across conditions.
The process is random, so it is always possible that just by chance, the participants in one condition might turn out to be substantially older, less tired, more motivated, or less depressed on average than the participants in another condition.
However, there are some reasons that this possibility is not a major concern.
One is that random assignment works better than one might expect, especially for large samples.
Another is that the inferential statistics that researchers use to decide whether a difference between groups reflects a difference in the population takes the “fallibility” of random assignment into account.
Even if random assignment does result in a confounding variable and therefore produces misleading results, this confound is likely to be detected when the experiment is replicated.
Matched-Groups design
An experiment design in which the participants in the various conditions are matched on the dependent variable or on some extraneous variable(s) prior the manipulation of the independent variable.
This guarantees that these variables will not be confounded across the experimental conditions.
Within-Subjects Experiments
An experiment in which each participant is tested under all conditions.
The primary advantage of this approach is that it provides maximum control of extraneous participant variables.
Within-subjects experiments also make it possible to use statistical procedures that remove the effect of these extraneous participant variables on the dependent variable and therefore make the data less “noisy” and the effect of the independent variable easier to detect.
Within-subjects experiments also make it easier for participants to guess the hypothesis
order effect
An effect that occurs when participants’ responses in the various conditions are affected by the order of conditions to which they were exposed.
carryover effect
An effect of being tested in one condition on participants’ behavior in later conditions.
practice effect
An effect where participants perform a task better in later conditions because they have had a chance to practice it.
fatigue effect
An effect where participants perform a task worse in later conditions because they become tired or bored.
context effect (or contrast effect)
Unintended influences on respondents’ answers because they are not related to the content of the item but to the context in which the item appears.
counterbalancing
Varying the order of the conditions in which participants are tested, to help solve the problem of order effects in within-subjects experiments.
complete counterbalancing
A method in which an equal number of participants complete each possible order of conditions.
Latin square design
A more efficient way of counterbalancing is through a Latin square design which randomizes through having equal rows and columns.
random counterbalancing
A method in which the order of the conditions is randomly determined for each participant.
Using this technique every possible order of conditions is determined and then one of these orders is randomly selected for each participant.
This is not as powerful a technique as complete counterbalancing or partial counterbalancing using a Latin squares design.
Use of random counterbalancing will result in more random error, but if order effects are likely to be small and the number of conditions is large, this is an option available to researchers.
There are two ways to think about what counterbalancing accomplishes
One is that it controls the order of conditions so that it is no longer a confounding variable.
Thus any overall difference in the dependent variable between the two conditions cannot have been caused by the order of conditions.
A second way to think about what counterbalancing accomplishes is that if there are carryover effects, it makes it possible to detect them.
One can analyze the data separately for each order to see whether it had an effect.
Simultaneous Within-Subjects Designs
often used when participants make multiple responses in each condition.
Imagine, for example, that participants judge the guilt of 10 attractive defendants and 10 unattractive defendants.
Instead of having people make judgments about all 10 defendants of one type followed by all 10 defendants of the other type, the researcher could present all 20 defendants in a sequence that mixed the two types.
The researcher could then compute each participant’s mean rating for each type of defendant.
Between-Subjects or Within-Subjects?
Between-subjects experiments have the advantage of being conceptually simpler and requiring less testing time per participant.
They also avoid carryover effects without the need for counterbalancing.
Within-subjects experiments have the advantage of controlling extraneous participant variables, which generally reduces noise in the data and makes it easier to detect any effect of the independent variable upon the dependent variable.
Within-subjects experiments also require fewer participants than between-subjects experiments to detect an effect of the same size.
A good rule of thumb, then, is that if it is possible to conduct a within-subjects experiment (with proper counterbalancing) in the time that is available per participant—and you have no serious concerns about carryover effects—this design is probably the best option.
If a within-subjects design would be difficult or impossible to carry out, then you should consider a between-subjects design instead.
Remember also that using one type of design does not preclude using the other type in a different study.
There is no reason that a researcher could not use both a between-subjects design and a within-subjects design to answer the same research question.
In fact, professional researchers often take exactly this type of mixed methods approach
Four Big Validities
internal validity, external validity, construct validity, and statistical validity.
Internal Validity
Refers to the degree to which we can confidently infer a causal relationship between variables.
Thus experiments are high in internal validity because the way they are conducted—with the manipulation of the independent variable and the control of extraneous variables (such as through the use of random assignment to minimize confounds)—provides strong support for causal conclusions.
In contrast, non-experimental research designs (e.g., correlational designs), in which variables are measured but are not manipulated by an experimenter, are low in internal validity.
The purpose of an experiment, however, is to show that two variables are statistically related and to do so in a way that supports the conclusion that the independent variable caused any observed differences in the dependent variable.
The logic is based on this assumption: If the researcher creates two or more highly similar conditions and then manipulates the independent variable to produce just one difference between them, then any later difference between the conditions must have been caused by the independent variable.
External Validity
Refers to the degree to which we can generalize the findings to other circumstances or settings, like the real-world environment.
As a general rule, studies are higher in external validity when the participants and the situation studied are similar to those that the researchers want to generalize to and participants encounter every day, often described as mundane realism
psychological realism
Where the same mental process is used in both the laboratory and in the real world.
mundane realism
When the participants and the situation studied are similar to those that the researchers want to generalize to and participants encounter every day.
should be careful, however, not to draw the blanket conclusion that experiments are low in external validity
One reason is that experiments need not seem artificial.
A second reason not to draw the blanket conclusion that experiments are low in external validity is that they are often conducted to learn about psychological processes that are likely to operate in a variety of people and situations.
Construct Validity
One of the “big four” validities, whereby the research question is clearly operationalized by the study’s methods.
the quality of the experiment’s manipulations.
very high because the experiment’s manipulations very clearly speak to the research question.
operationalization
The specification of exactly how the research question will be studied in the experiment design.
Statistical Validity
Concerns the proper statistical treatment of data and the soundness of the researchers’ statistical conclusions.
When considering the proper type of test, researchers must consider the scale of measure their dependent variable was measured on and the design of their study.
Further, many inferential statistics tests carry certain assumptions (e.g., the data are normally distributed) and statistical validity is threatened when these assumptions are not met but the statistics are used nonetheless.
One common critique of experiments is that a study did not have enough participants. The main reason for this criticism is that it is difficult to generalize about a population from a small sample.
At the outset, it seems as though this critique is about external validity but there are studies where small sample sizes are not a problem.
small sample sizes are actually a critique of statistical validity. The statistical validity speaks to whether the statistics conducted in the study are sound and support the conclusions that are made.</sub>
the likelihood of detecting an effect of the independent variable on the dependent variable depends on not just whether a relationship really exists between these variables, but also the number of conditions and the size of the sample.
Prioritizing Validities
These four big validities–internal, external, construct, and statistical–are useful to keep in mind when both reading about other experiments and designing your own.
However, researchers must prioritize and often it is not possible to have high validity in all four areas.
This discrepancy does not invalidate the study but it shows where there may be room for improvement for future follow-up studies.
Morling (2014) points out that many psychology studies have high internal and construct validity but sometimes sacrifice external validity
Practical Considerations:
Recruiting Participants-
subject pool
An established group of people who have agreed to be contacted about participating in research studies.
Practical Considerations:
Recruiting Participants-
The Volunteer Subject
Even if the participants in a study receive compensation in the form of course credit, a small amount of money, or a chance at being treated for a psychological problem, they are still essentially volunteers.
This is worth considering because people who volunteer to participate in psychological research have been shown to differ in predictable ways from those who do not volunteer.
Specifically, there is good evidence that on average, volunteers have the following characteristics compared with non-volunteers:
• They are more interested in the topic of the research.
• They are more educated.
• They have a greater need for approval.
• They have higher IQ.
• They are more sociable.
• They are higher in social class.
This difference can be an issue of external validity if there is a reason to believe that participants with these characteristics are likely to behave differently than the general population.
Practical Considerations:
Recruiting Participants-
Advertisements or personal appeals
Participants who are not in subject pools can also be recruited by posting or publishing advertisements or making personal appeals to groups that represent the population of interest.
Practical Considerations:
Recruiting Participants-
In field experiments
In many field experiments, the task is not recruiting participants but selecting them.
For example, conducted a field experiment on the effect of being smiled at on helping, in which the participants were shoppers at a supermarket.
A confederate walking down a stairway gazed directly at a shopper walking up the stairway and either smiled or did not smile.
Shortly afterward, the shopper encountered another confederate, who dropped some computer diskettes on the ground.
The dependent variable was whether or not the shopper stopped to help pick up the diskettes.
Two aspects of this study that are worth addressing here.
First, notice that these participants were not “recruited,” which means that the IRB would have taken care to ensure that dispensing with informed consent in this case was acceptable (e.g., the situation would not have been expected to cause any harm and the study was conducted in the context of people’s ordinary activities).
Second, even though informed consent was not necessary, the researchers still had to select participants from among all the shoppers taking the stairs that day.
It is extremely important that this kind of selection be done according to a well-defined set of rules that are established before the data collection begins and can be explained clearly afterward.
In this case, with each trip down the stairs, the confederate was instructed to gaze at the first person he encountered who appeared to be between the ages of 20 and 50.
Only if the person gazed back did they become a participant in the study.
The point of having a well-defined selection rule is to avoid bias in the selection of participants.
Practical Considerations:
Standardizing the Procedure-
experimenter expectancy effect
When the experimenter’s expectations about how participants “should” behave in the experiment affect how the participants behave.
Practical Considerations:
Standardizing the Procedure-
Here are several ways to do this
The way to minimize unintended variation in the procedure is to standardize it as much as possible so that it is carried out in the same way for all participants regardless of the condition they are in.
• Create a written protocol that specifies everything that the experimenters are to do and say from the time they greet participants to the time they dismiss them.
• Create standard instructions that participants read themselves or that are read to them word for word by the experimenter.
• Automate the rest of the procedure as much as possible by using software packages for this purpose or even simple computer slide shows.
• Anticipate participants’ questions and either raise and answer them in the instructions or develop standard answers for them.
• Train multiple experimenters on the protocol together and have them practice on each other.
• Be sure that each experimenter tests participants in all conditions.
Practical Considerations:
Standardizing the Procedure-
double-blind study
A method to reduce experimenter bias, where neither the participant nor the experimenter is knowledgeable about the condition to which the participant is assigned.
Practical Considerations:
Record Keeping-
It is essential to keep good records when you conduct an experiment.
It is typical for experimenters to generate a written sequence of conditions before the study begins and then to test each new participant in the next condition in the sequence.
As you test them, it is a good idea to add to this list basic demographic information; the date, time, and place of testing; and the name of the experimenter who did the testing.
It is also a good idea to have a place for the experimenter to write down comments about unusual occurrences (e.g., a confused or uncooperative participant) or questions that come up.
This kind of information can be useful later if you decide to analyze sex differences or effects of different experimenters, or if a question arises about a particular participant or testing session.
Since participants’ identities should be kept as confidential (or anonymous) as possible, their names and other identifying information should not be included with their data.
In order to identify individual participants, it can, therefore, be useful to assign an identification number to each participant as you test them.
Simply numbering them consecutively beginning with 1 is usually sufficient.
This number can then also be written on any response sheets or questionnaires that participants generate, making it easier to keep them together.
Practical Considerations:
Manipulation Check-
Verifying the experimental manipulation worked by using a different measure of the construct the researcher is trying to manipulate.
The purpose of a manipulation check is to confirm that the independent variable was, in fact, successfully manipulated.
Are particularly important when the results of an experiment turn out null.
In cases where the results show no significant effect of the manipulation of the independent variable on the dependent variable, a manipulation check can help the experimenter determine whether the null result is due to a real absence of an effect of the independent variable on the dependent variable or if it is due to a problem with the manipulation of the independent variable.
Usually done at the end of the procedure to be sure that the effect of the manipulation lasted throughout the entire procedure and to avoid calling unnecessary attention to the manipulation (to avoid a demand characteristic).
However, researchers are wise to include a manipulation check in a pilot test of their experiment so that they avoid spending a lot of time and resources on an experiment that is doomed to fail and instead spend that time and energy finding a better manipulation of the independent variable.
Practical Considerations:
Pilot Testing-
Is a small-scale study conducted to make sure that a new procedure works as planned.
In a pilot test, you can recruit participants formally (e.g., from an established participant pool) or you can recruit them informally from among family, friends, classmates, and so on.
The number of participants can be small, but it should be enough to give you confidence that your procedure works as planned.
There are several important questions that you can answer by conducting a pilot test:
• Do participants understand the instructions?
• What kind of misunderstandings do participants have, what kind of mistakes do they make, and what kind of questions do they ask?
• Do participants become bored or frustrated?
• Is an indirect manipulation effective? (You will need to include a manipulation check.)
• Can participants guess the research question or hypothesis (are there demand characteristics)?
• How long does the procedure take?
• Are computer programs or other automated procedures working properly?
• Are data being recorded correctly?
To answer some of these questions you will need to observe participants carefully during the procedure and talk with them about it afterward.
Participants are often hesitant to criticize a study in front of the researcher, so be sure they understand that their participation is part of a pilot test and you are genuinely interested in feedback that will help you improve the procedure.
If the procedure works as planned, then you can proceed with the actual study.
If there are problems to be solved, you can solve them, pilot test the new procedure, and continue with this process until you are ready to proceed.
