Chapter 9 Flashcards
Once you have a research idea, have stated your hypothesis, and have decided on a research design to test it, what do the next steps involve?
detailing the operationalizations for each variable and creating an experience for your participants.
Should all of the details of your study be determined before applying for ethical approval?
yes.
What are experimenters ensuring when making all efforts to control as many elements fo the study as possible?
internal validity.
Are there clear cut rules for translating conceptual variables into specific operations?
No
What is exactly how the variable is operationalized dependent upon?
the variable, cost, practicality, ethical concerns, and researcher’s creativity.
What are straight forward manipulations?
operationalizations that involve manipulating the independent variable using instructions or other stimulus materials in a simple and obvious way.
Stimuli may be presented verbally, in written form, or via video.
What is one way to think about operationalizations for manipulating independent variables?
distinguishing straight forward manipulations from staged manipulations.
if researchers manipulate a variable simply by presenting material to the participants, what is this called?
straightforward manipulations.
What type of research typically relies on straightforward manipulations?
memory research
What is an example of a straightforward manipulation experiment on memory. (give IV and DV)
Memory study where participants are shown a picture or words and asked to remeber or forget it.
These materials (pictures vs words), and specific instructions (remember or forget), constitute two independnet variables operationalized using straightforward manipulation.
When is a study said to have high mundane realism?
whenever the tasks involveed in a study mimic experiences and conditions present in everyday life.
What is an example of potential tasks that would have high mundane realism? Low mundane realism?
(1) reading job advertisments and judging how appealing they are is a task that job seekers do in everyday life.
(2) deliberately trying to forget information and then trying to remember it is arguably less realistic.
What is mundane realism?
the extent to which the experiences in a study resemble closely an experience of everyday life.
Is it sometimes necessary to create a series of events that occur during the experiment to manipulate the independent variable successfully?
What type of manipulation is this?
yes.
Staged manipulation
What are staged manipulations?
Operationalizations of an independent variable that involve creating a complex situation. Participants then experience the situation and their responses are recorded. Deception is often used to conceal the fact that the situation is a ruse.
What are examples of what staged manipulations can be?
can be elaborate situations involving actors,
can take the form of a cover story
What is often involved in staged manipulations? Does this have concerns?
deception
yes, lots of ethical concerns explored in chapter 3
What are the 2 main reasons that staged manipulations are employed? provide examples for each.
(1) researcher may be trying to create a certain psychological state in participants.
ex: researchers have created situations where people believed they were interacting with another person, although the other person’s contribution was actually a pre-recorded video.
(2) staged manipulations can be used to simulate situations that occur in the real world.
Example: White and Caird used the University of Calgary driving simulator to study the effects of conversing with a passenger on driving errors. Participants were randomly assigned to drive either alone or with a passenger. The passenger, a confederate playing the role of another participant, engaged the real participant in conversation throughotu the driving simulation. Relative to the “alone” condition, the presence of this conversation did not affect participant’s ability to actually notice and respond to others on the road. The conditions in this staged manipulation simulated common real-world environments that people experience while driving.
What do staged manipulations sometimes employ?
a confederate (also known as an accomplice). The nature of this can differ quite widely from study to study.
Are staged manipulations more complex than straightforward manipulations? Why?
yes.
It relies on convincing the participants that they are partaking in a certain kind of situation or interaction.
What is a confederate oftne introduced as?
usually introduced as another participant.
What may confederate’s be used for in staged manipulations?
may be used to create a particular social situation or administer the independent variable.
What can staged manipulations demand?
What are staged manipulations used for?
a great deal of ingenuity and some acting ability on the part of the confederates.
Staged manipulations are used to involve the participants in an ongoing social situation, which the individuals perceive not as an experiment but as a real experience
What is experimental realism?
The extent to which the experiences in a study are experienced by participants as impactful and engaging.
What is the difference between experimental realism and mundane realism
with experimental realism, the tasks of a study might not resemble real-world experiences (i.e low mundane realism), but they can still engage participants in a meaningful way, producing psychological experiences that are impactful (i.e high experimental realism)
When is a study said to have experimental realism?
when a study engages and involves participants in a deep way (like with a staged manipulation)
When is it easier to have high experimental realism?
in staged rather than straightforward manipulations
What do researchers assume about results from an operationalization with high experimental realism?
they assume that the result will be natural behaviour that truly reflects the feelings and intentions of the participants.
Why can it be difficult to replicate staged manipulations? When is it easier to replicate results? Why might this alternative way not always be possible?
they often allow for a great deal of subtle interpersonal communication that is hard to put into words.
In general, it is easier to replicate results when the manipulation is relatively straightforward. However, this might not always be possible because the nature of the variable you are studying may demand more complicated procedures using staged manipulations.
What must researchers consider when deciding on precise operationalizations for each level of the independent variable
manipulation strength.
What is manipulation strength?
The degree to which levels of an independent variable differ from each other. In a weak manipulation, conditions are subtly different. In a strong manipulation, conditions are maximally different.
In general, what do we want the strength of our manipulation to be? What does this mean?
as strong as possible.
Essentially this means making the levels of the independent variable maximally different, while keeping everything else between the 2 groups the same .
What does strong manipulation increase the chances of?
increases the chances that your results will reveal an effect of the independent variable on the dependent variable, if one really exists.
Can comparing high to low lead to different conclusions than comparing high to no?
yes.
What is a note of caution about the strongest possible manipulations?
the strongest possible manipulations can sometimes be ethically problematic. If you are putting participants in harm to have a stronger manipulation you would be violating the ethical principle of concern for welfare.
is cost of manipulation an issue when deciding how to manipulate the independent variable?
yes.
Who offer grants to support research? Who needs to support these sources?
some government and private agencies (SSHRC, NSERC)
because research is often costly, continued public support of these agencies is very important.
What might researchers with limited monetary resources not be able to afford?
may not be able to afford expensive equipment, salaries for confederates, or payments to participants to reduce attrition rates for longitudinal studies.
What type of manipulation may require more of a researcher’s time?
a manipulation in which participants must be run individually may require more of a researchers tie than one that allows running many individuals at once.
Are straightforward or complex, staged, experimental manipulations more costly?
straightforward are much less costly.
What is a manipulation check?
A measure used to determine whether the manipulation of the independent variable has had its intended effect on a participant.
What can manipulation checks provide evidence for?
the validity of your manipulation
What would be an example of a manipulation check?
If you are manipulating anxiety to measure its affects on memory, for example, a manipulation check will tell you whether participants in the high-anxiety group really were more anxious than those in the low anxiety condition. The manipulation check in this case might involve a self-report of anxiety, a behavioural measure (ex: number of nervous movements), or a physiological measure (ex: heart rate change).
Why is it difficult to decide when to administer a manipulation check?
because it might distract participants or inform them about the purpose of the experiment.
When can manipulation checks be placed after the measuring the dependent variable? What is the risk of this?
if the effects of the independent variable are expected to last long enough.
There is always risk that the manipulation was successful, but by the time you measure it late in the study, the effects have worn off.
What are the 2 advantages of a manipulation check? Examples?
- if the manipulation check is used in an early pilot study and reveals that your manipulation is not effective, you can change the procedures before running the actual experiment.
For instance, if the manipulation check shows that either the low nor high-anxiety group was very anxious, you could change the procedures to increase anxiety further in the anxiety condition. AKA. you could increase the strength of your manipulation for the independent variable.
- a manipulation check is advantageous if the results show no effect of the independent variable on the dependent variable. In order tot conclude that the independent variable truly doesn’t effect your dependent variable, you will first need to rule out the possibility that you failed to manipulate the independent variable at all.
If both groups are equally anxious after you manipulate anxiety, anxiety can’t affect the dependent measure.
What happens if the manipulation check shows that the manipulation was successful but you still did not find an effect?
Then you know at least that the results were not due to a problem with manipulation, the reason for not finding a relationship might lie elsewhere. Perhaps you had a poor dependent measure or perhaps there really is no relationship between variables.
What are the 3 broad categories of measures?
self-report, behavioural, and psychological.
What are examples of techniques or tools used for self report measures(3 things)? What are examples of variables operationalized using these techniques/tools (3 things)?
Techniques:
- paper and pencil questionnaires
- face to face interviews
- online questionnaires
Variables:
- attitudes about something
- intentions to do something
- a person’s values, self-esteem, mood, anxiety, relationship satisfaction, personality traits.
What are examples of techniques (5 things) or tools used for behavioural measures?
Techniques:
- audio or video recorder
- eye tracker
- electronic activated recorder
- weight scale
- still camera
Variables:
- self-control
- creativity
- reaction time
- facial expression of emotion
- attention
- liking
- efficacy of a bulimia intervention
- memory
- generosity
What are examples of techniques or tools used for physiological measures?
Techniques:
- GSR, EMG, ECG, EEG
- blood analysis
- saliva analysis
- heart rate
- breathing rate
- blood pressure
- MRI
- fMRI
Variables:
- Stress
- genetic marker for mental illness
- physical fitness
- size of amygdala or damage to hippocampus
- brain activation when looking at image of romantic partner.
-
What is a self-report measure?
An operationalization of a variable that involves asking people to explicitly indicate something about themselves (e.g., personality, behaviour, attitudes).
What can a self-report measure be used to measure? What type of scale is usually preferred?
many different aspects of human thought and behaviour including explicit attitudes, judgements about someone’s personality characteristics, intended behaviours, emotional states, and confidence in one’s judgements.
When using self-report measures, using an already published scale that has been validated is typically preferred but you can also create your own measure (less preferred)
How are responses typically made with self-report measures?
using rating scales with descriptive anchors
What is a behavioural measure? How many behaviours can you measure this way?
a direct observation of behaviours. There is almost an endless number of possible behaviours you could chose to observe and record.
How many aspects of behaviour can one record?
there are many. A researcher may chose to note whether a given behaviour occurs, the number of times a behaviour occurs, how quickly a behaviour comes after a stimulus, how long a behaviour lasts,
What is the rate of behaviour?
number of times a behaviour occurs in a given time period.
What is duration?
how long a behaviour lasts
what is reaction time?
how quickly a response occurs after a stimulus.
How do researchers decide which aspect of behaviour they should measure?
it depends on which is most theoretically relevant or which aspect of behaviour is most likely to be affected by an independent variable.
What is the galvanic skin response? What is it used as an indicator for?
a physiological measure it is called the GSR it is the electrical conductance of the skin, which changes when sweating occurs. It is generally used ot measure emotional arousal, anxiety, or stress.
What is an electromyogram? What is it used as an indicator for?(2 things)
called an EMG it measures muscle tension
can be used as an indicator of tension or stress
Can also be used as a measure of which muscles in the face are being activated, as an indicator for emotional expressions.
What is a physiological measure?
An operationalization of a variable that involves observing and recording a response from the body.
What is an electrocardiogram? What is it used as an indicator for?
called an ECG it measures heartbeat regularity and rate.
can be used as an indicator of tension or stress
What is the electroencephalogram? What can it be used to measure?
called an EEG is a measure of the electrical activity of brain cells.
can be used to measure activity in different parts of the brain as learning occurs or brain activity during different stages of sleep.
What is an magnetic resonance imaging?
called MRI caputres images of brain structures
Allows scientists to compare brain structures between people with a particular condition (schizophrenia) and those without the condition.
what is functional magnetic resonance imaging?
called fMRI measures blood flow in the brain to make inferences about which regions are involved while a participant performs a specific task.
What are examples from physiological measures being measured through bodily fluid?
cortisol (stress hormone) can be measured through saliva.
Is it always clear what type of measure is supposed to be used?
sometimes but in other cases the same variable can be measured using more than one category of measurement.
What is a measuring of helping usually ?
a behavioural measure
How are personality traits usually measured?
self-report
What approaches lead to the strongest conclusions?
a multi method approach
is employing a diversity of measurement types a good idea? Do the different measurement types have to be in the same study?
yes.
no they can be across a series of studies.
What should the dependent variable be sensitive enough to do?
detect any resulting differences between groups.
what self report measure is more sensitive?
A response of yes or no vs a response on a numbered scale?
numbered scale it allows a gradation which makes it easier to detect differences in the amount of the variable.
What is sensitivity?
The ability of a measure to detect differences or changes.
When is the issue of sensitivity particularly important?
when using behavioural measures of performance
how can memory be measured?
using recall, recognition , or reaction time.
What is a ceiling effect?
The failure of a measure to detect difference because it was too easy and everyone does well (see alsofloor effect)
How does the ceiling effect relate to sensitivity? What problems can this lead to? Example?
if everyone does well then there isn’t much variability in the scores and so the measure lacks sensitivity to detect differences. This can lead to problems interpreting the results.
Ex: an independent variable might appear to have no effect on the dependent measure, but it is actually because the performance of all participants is at ceiling.
What is a floor effect?
The failure of a measure to detect a difference because it was too difficult.
What do we ideally want from our scores?
we want the scores for our measures to be not too high or too low and have lots of variability (i.e. good spread of scores), exhibiting the ability to detect differences.
How do you diagnose a potential ceiling or floor effect?
look for average values close tot the minimum or maximum possible score.
Why do researchers sometimes include multiple measures of the same variable? How does it increase our confidence in the results?
because any variable can be measured through various operationalizations.
If the independent variable has the same effect on several measures of the same dependent variable, our confidence in the results can be increased.
Is it useful to know whether the same independent variable affects some measures of a variable but not others?
yes
Can researchers be interested in studying effects for several entirely different dependent variables?
yes.
When you have more than measure, what question arises? What is this question similar to? What specifically do we worry about?
the question of what order to sequence them
aka is it possible that responses to one measure will be different if it comes before or after another measure?
This question is similar to order effects for within-subjects designs.
Specifically we worry about carry over and any influence of fatigue.
What are 2 ways to deal with the issue of what order to sequence measures for the dependent variable when you have more than one?
- present the most important measures first and less important measures later. This way order will not be a problem in interpreting the results for the most important variables. Even though order effects may be a potential problem for some of the measures, the overall impact on the study is minimized.
- counterbalance the order of presenting the measures, or rely on complete randomization of the order.
What is important to include in the research report when using multiple measures for the dependent variable? Why?
to ensure ethical, open disclosure whenever multiple measures are used, it is important to include all of them in your research report, even if results are mixed. It is absolutely unethical to report only what happened with the measures that “worked”, or to only report the results that confirm your hypothesis.
Once a researcher operationalizes their variables, and consults other researchers for feedback, would should be the next step?
the next stop will then be to plan the experience from the participant’s viewpoint, or “set the stage”
Are some measures more expensive than others? Give examples.
Where do researchers often get their resources from?
yes.
self-report questionnaire measures are generally inexpensive or even free, whereas measures that require trained observers or elaborate equipment can become quite costly.
Often researchers need resources from the university or outside agencies to carry out such research.
What should you consider doing after deciding on specific operationalizations for all variables?
consider asking another researcher for feedback on your design before moving forward.
What rationals are given on the consent forms? What do you need to do if you decide that deception is necessary?
on the consent forms the researcher will explain to participants why the experiment is being conducted. Sometimes the rational is completely truthful, although only rarely will you want to tell participants the actual hypothesis (think demand characteristics).
If you decide that any deception is necessary, you will need to plan a debriefing session at the end of the experiment.
if collecting data online what should you include at the beginning and end?
include a welcoming message and a closing message thanking participants for their time.
What should you do if collecting data in person or over the phone?
prepare a step-by step script that starts with welcoming the participant to the study and finishing with debriefing and thanking the participant for their time.
As a researcher, why is practicing the script important?
it helps ensure experimental control, especially with staged manipulations.
What does good research design mean?
eliminating as many alternative explanations for the results as possible.
What is demand characteristics?
Cues embedded in a study that inform the participant how he or she is expected to behave.
What might happen when participants have formed expectations about the study’s purpose?
they might deliberately act in ways to confirm the hypothesis or even undermine the hypothesis.
What did Nicholas and manner find relating to demand characteristic?
they found that participants that had been told the hypothesis tended to act in ways that confirmed it, especially among those who liked the experimenter.
What are 3 ways to control for demand characteristic?
to use deception to mislead participants about the purpose of the study.
a researcher may attempt to disguise the dependent measure by using an unobtrusive measure or by placing the measure among a set of unrelated filler items in a questionnaire.
assess whether or not demand characteristic was a problem by asking what they thought the research was about.
What are filler items?
Items included in a questionnaire measure to help disguise the true purpose of the measure.
What should you do if someone indicates that they understood the hypothesis after participating?
their data should be analyzed separately to see if this knowledge influenced their responding.
When may demand characteristics be eliminated?
when people are not aware that an experiment is taking place or that their behaviour is being observed. Thus, observational research in which the observer is concealed or is using unabtrusive measures can minimize the problem of demand characteristic.
What is the special kind of participant expectation that arises in research on effects of treatments?
the placebo effect
Could just administering a pill or other form of treatment (not necessarily actually the IV) be sufficient to cause an improvement in behaviour?
yes.
How can you control for a placebo affect?
by adding a placebo group
What is a placebo group?
In experiments, a control group given the expectation of improvement through treatment, in order to control for the psychological effects of receiving a treatment.
How do we differentiate form placebo and actual effect in experiments?
if the IV actually worked then the effect will be higher in the experimental group compare to the control .
When would we say that the effect is likely just due to placeb
if the placebo group improves as much as the experimental group.
What is a balanced placebo design?
a design in which placebo effect is the main focus. Ex: trying to distinguish if nicotine inhalers work because of the nicotine or because people expect them to work. Give people nicotine and tell the, give people no nicotine and tell them, give people nicotine and don’t tell them, give people no nicotine and don’t tell them.
In some areas of research can the use of placebo control groups have ethical implications?
Example?
yes.
Suppose you are studying a treatment that really has a positive effect on people (e.g. reducing symptoms of depression). In this case it is important to help people in the control condition. For example, participants may be placed in a waitlist control condition and they are put on a waitlist for the treatment.
Is there debate around antidepressants and placebo effects?
yes. there has been lots of debate around the extent to which the beneficial effects of antidepressants are due to placebo effects. 2 meta-analyses have concluded that with people to slight to moderate depresison, antidepressants have very little more of an effect than placebo, but antidepressants are more effective for people with severe depression.
What is experimenter bias? What is it also called?
Any intentional or unintentional influence that the experimenter exerts on participants to confirm the hypothesis; also called experimenter expectancy effects.
What are the 2 potential sources of experimenter bias?
- experimenter might unintentionally treat participants differently depending on what condition they are in. Certain words might be emphasized when reading instructions to one groupp but not the other or the experimenter might smile mroe when interacting with people in one of the conditions
- when experimenters record participant behaviours, with subtle differences emerging in how experimenter interprets and records behaviours for people in different conditions
How can experimenter bias be communicated to humans?
by both verbal and non-verbal means.
Can expectations influence evaluations of behaviour (experimenter bias)? example?
yes.
example: therapists were mailed a realistic description of a case study in which an adolescent had symptoms to suggest ADHD but did not fully meet diagnostic criteria. This adolescent was morel ikely to be misdiagnosed with ADHD when presented in boys than as a girl, even though all other information was the same. This means the therapists expectations regarding gender affected their diagnosis.
What are the ways to address experimenter bias?
- run everyone in all conditions simultaneously, so that the experimenter behaviour’s behaviour is exactly the same for all participants. This solution is feasible only under certain circumstances, such as when the study relies on printed materials and the experimenters instructions to the participants are the same for everyone.
- researchers can use computer survey software to administer independent variables and record responses. Such automated procedures leave little room for experimenter bias to influence results.
- all experimenters should be well-trained and should practice behaving consistently with all participants
- If particularly concerned, researchers will use experimenters who are unaware of the hypothesis.
What is a single blind procedure?
An experiment in which participants do not know to which condition they have been assigned, but the experimenter does.
What is a double blind procedure?
An experiment in which neither the experimenter nor the participant knows to which condition the participant has been assigned.
How many experimenters do double blind procedures usually require?
usually 2 different ones. One who adminsiters the IV and one who takes over and administers the dependent variable, without knowing what independent variable was assigned.
Do solutions to experimenter bias need to be built into the procedures? Why?
Because experimenter bias takes the form of expectancy effects.
Are there ways to design a study specifically to measure expectancy effects?
yes
Before collecting data, what must researchers do?
must seek approval from their institution’s Research Ethics Board (REB).
What will the REB do?
examine the procedure, materials, informed consent form. Deception and other anticipated risk must be explained and justified, and weighed against potential benefits. Researchers must also explain how they will ensure anonymity and confidentiality (if possible)
What is the checklist of what to do before applying for REB approval?
- operationalize your independent variable (if applicable)
- operationalize all measures (including dependent variables)
- sought feedback from colleagues on the method (e.g. to avoid confounds)
- listed the exact procedures each participant will experience
- created all materials participants will use.
- created the informed consent form
- planned a way to debreif participants (if necessary)
- justified who will be included and excluded from participating (if targeting a specific populations)
- determined the number of participants to be run in the study
- prepared participation recruitment materials
- determiend how confidentiality and anonymity will be maintained during and after data collection
- minimized any forseeable risks ; noted any feesable benefits
- completed institutional researhc board application form.
-
Who is the method used to recruit participants justified to?
the REB (humans)
the Animal Care Commitee (non-humans)
Does the method used to select participants have implications for generalizing the research results to the population of interest.
yes.
Do most research projects involve sampling participants from a population of interest?
yes.
How can samples be drawn from a population? Why is sampling important?
- through probability sampling (random assignment)
- through non-probability sampling (chapter 7)
Sampling is important because it informs whether we can generalize the results to a population, and if so, which populations.
Whenever a specific population is targeted does it have to be justified? What ethical principle does this relate to?
yes.
justice.
Are special procedures needed when studying sensitive populations like Indigenous peoples’?
yes.
generally, what does increasing your sample size do?
assuming there is an effect to be found, increasing sample size increases likelihood that you will find an effect.
Are larger samples more likely to accurately represent the population of interest?
yes.
What does the debriefing session provide?
an opportunity for researchers to explain the ethical and educational implications of the study, verbally and/or in writing.
When should debriefing sessions always occur? What should they include?
when any form of deception has been used. They should include an explanation of why the deception was necessary, reassurance that believing the deception does not reflect poorly on the person, and an apology to attempt to repair any negtive feelings.
What might researchers do in the debrief if if the study is expected to trigger psychological disturbances?
provide resources for cousnelling services.
How can debriefing be useful to researchers?
can provide an opportunity to ask what participants were thinking during the study, their thoughts on the purpose of the study. Can also ask to refrain discussing the study with others.
How is debriefing an important opportunity for participants?
it gives them an opportinity to raise any concerns they have about the study, and for the researchers to repair any negative moods. It is important that participants leave feeling as good or better than before.
What does funelled debriefing help identify?
if participants were affected by demand characteristic.
When do researchers sometimes chose to conduct pilot studies?
when procedures are elaborate or costly, or when ther will only be one single opportunity to collect data.
What is a pilot study? Where is the sample for the pilot study typically drawn? Does a pilot study have to be included in ethics approval?
A small-scale study with a small sample, conducted prior to conducting an actual study, designed to test and refine procedures.
sample typically drawn from same population as the sample the researcher hopes to test.
Must be included in ethics approval because data are collected.
What do pilot studies often reveal?
whether participants understand the instructions, whether the experimental setting seems plausible, whether any questions are confusing and so on.
In what type of manipulation are pilot studies particularly important? What are two ways pilot study participants can give feedback in these situations?
when using staged manipulation to ensure the scenario is meanigful and believable.
- questioned in detial a abotu their experience during the experiment
- “think aloud” protocol where participants are asked to verbalize their thoughts about whats happening during the study.
What happens if you would like to make any major changes to the method or materials of your study after your pilot test?
you need to make an amendment to your original ethics application for updated approval.
How do researchers make several implicit contracts with participants?
ex: if a participant agrees to be present for a study at a specific time, it is crucial that the researcher or research assistant is there. Participants value punctuality.
Similarly, if researchers promise to send a summary of results o participants, they should do so.
These details are important for maintainign trust with the participant and supporting the advancment of science.
After data has been collected, what is the next step?
to analyze them
What do statistical analyses allow the researchers to do?
examine and interpret the pattern of data obtained in the study.
- are the variables related to one another
- does the independent variable have an effect on the dependent variable?
Why might researchers chose to do follow up studies?
to see if results can be replicared with a new sample, to rule out alternative explanations, or to deal with problems in the first study.
What is the final step of research?
to write a report that details why you conducted that research, how you obtained participants, what procedures you used, and what you found.
Researchers report their results at scientific conferences and submit them for publication in journals.
What happens when you submit your paper to a peer reviewed journal?
the editor send your paper for peer review. 2 or more scientists read your paper and provide an evaluation. these evaluators can either recommend acceptance (often with stipulation that revisions be made), or rejection (which is more common)
