Chapter 6 - Ensuring Research Validity Flashcards
What is research validity?
The correctness or truthfulness of an inference that is made from a research study
What is statistical conclusion validity?
The degree to which a claim about the existence and **strength **of a **reported relationship **is correct.
The validity with which we can infer that the IV and DV covary. (So with every variation in the IV, there is a corresponding variation in the DV… in other words, the IV and the DV are statistically related)
So not just is there a relationship, but is it statistically significant? (i.e., has the analysis procedure of “significance testing” indicated that observed relationship is probable not due to chance, but is, in fact, a real relationship?
If the researcher’s inference about covariation between the variables is correct, then the study has statistical conclusion validity.
What is construct validity?
The degree to which a construct is adequately represented by the operations used in the study.
For every construct (for instance, the construct representing the research participant, the construct representing the DV , the construct representing the IV, and the construct representing the setting), you need to identify a set of operations that adequately represents them.
Does “# of fights per week” actually represent “marital discord”?
What are some threats to construct validity?
*** Participant Reactivity to the Experimental Situation **
* Experimenter Effects
Also…
*** Inadequate explanation of the construct **(if we don’t understand the construct well, it will be hard to measure it well…)
* Construct confounding (try to use operations that measure only your construct)
Etc.
(More in table 6.2 on page 182)
Threat to construct validity: “Participant Reactivity to the Experimental Situation.”
Mention:
* demand characteristics.
* positive self-presentation
This is talking about the fact that research participants are real people that bring their own motives (like wanting to look like a competent person) and tendencies with them.
Participants are “sentient curious creatures, constantly thinking about what is happening to them, evaluating the proceedings, figuring out what they are supposed to do, and planning their response.”
Participant Effect When participants enter an experiment, they start picking up on cues about the experiment; this stuff is called demand characteristics (this stuff defines the experiemental “demands” from the participants POV)
- Examples are: instructions, rumors, setting, etc.
Positive self-presentation: this is participants motivation to respond in a way that paints them in the most positive light. (They want to avoid looking dumb, difficult, easily manipulated, etc.)
Because of this positive self-presentation motive, researchers need to try to make sure that participant perceptions are held nonstant across the groups through thall phases and conditions of the experiment
Threat to construct validity: “Experimenter Effects”
Mention:
* experimenter expectancies
* experimenter attributes
The experimenter isn’t a passive, noninteractive observer; they’re an active agent that can influence the outcome of the experiement.
Experimenter Effects: Actions and characteristics of researchers that influence he responses of participants.
The experimenter wants to understand & predict behavior. They *want *their hypothesis to be right, want their research to be published. This can cause bias (intentional or uninentional)
The experimenter may not even realize they’re giving subtle cues to the participants that might influence their performance in the desired direction.
The experimenter may also accidently influence recording data that supports the research prediction.
- experimenter expectancies: biasing effects that can be attributed to the researchers expectations about the experiment outcome (Example: Clever Hans)
- experimenter attributes: bias that could come from experimenter attributes like biosocial (age, race, sex, etc.), psychosocial (anxiety level, hostility, need for social approval, warmth, etc.) and situational factors (experience level of researcher, do they know each other, attitude of the participant)
What is internal validity?
Internal ValidityThe extent to which the independent variable (and not other extraneous or confounding variables) produced the observed effect.
The extent design of a research study and the data that come from it allow you to draw defensible conculsions about cause and effect
A study is “strong” in internal validity if most plausible extraneous and confounding variables have been controlled.
