Data Quality, Research Design, and Evaluating Research Flashcards
Projective Tests purpose
Reveal the unknown or hidden
aspects of personality to the person and
researcher
Basic Method
Give someone an ambiguous
stimulus and ask them to tell you about it
Analysis
No clear cut, quantitative results
◦ Analysis of content of stories, letters, and
speeches
◦ Mostly used by clinical psychologists
◦ Provide you with B-Data
Objective Tests purpose
For the person to reveal to the researcher
what they think or know about themselves
Basic Methods
Basic Methods
Rational method
Write items that seem
directly, obviously, and rationally related to what
is to be measured (S-data
Factor analytic
Identify which items group
together using factor analysis (mainly S-data)
Empirical
Identify items based on how people
in preidentified groups respond (B-data)
Experimental Study:
Test differences between groups to determine if the difference is larger than would be expected by chance
Independent variable
a characteristic of an experiment that researchers manipulate to see if it causes a change in the dependent variable
Dependent Variable
The variable the researcher observes/ measures
Correlational Study:
Correlational method: A research
technique that establishes the relationship between
two variables by measuring both variables as they
occur naturally in a sample of participants
There are no experimental groups
* Questionnaires are administered
Comparing & Contrasting
Experimental and Correlational Methods
Both attempt to assess the relationship between two (or more)
variables
* The statistics (with two groups) are interchangeable
* The experimental method manipulates the presumed causal
variable, and the correlational method just measures it.
* Reasons for not knowing causal direction in correlational studies:
* Third-variable problem
* Unknown direction of cause (the directionality problem)
Experimental and Correlational Methods
Complications with experiments
◦ Uncertainty about what was really manipulated
A version of the third-variable problem
◦ Can create unlikely or impossible levels of a variable (“sledgehammer
manipulation”)
◦ Often require deception (ethically precarious)
◦ Not always possible
* Takeaway:
* Experiments are not always better
* An ideal research program includes investigations with both designs
Statistical significance
A result that would only occur by chance less than 5 percent
of the time
Null-hypothesis significance testing (NHST)
Determines the chance of
getting the result if nothing were really going on
p-level
Probability of obtaining a result if there is no difference between
groups or no relationship between variables (p<.05; p<.01; p<.001)
Problems with NHST
The logic is difficult to describe (and understand)
* The criterion for significance is an arbitrary rule of thumb (although the field has
agreed-upon rules of thumb)
* Nonsignificant results are sometimes misinterpreted to mean “no result” or
no relationship or difference
* Only provides information about the probability of one type of error
* Type I error vs. Type II error
Effect size
An index of the magnitude or strength of the relationship between the
variables
Correlation coefficient (Pearson’s r)
The most commonly used* measure of effect size
◦ Between -1 and +1
◦ If 0, there is no relationship
Positive correlation
as one variable goes up, so does the other; likewise, as one
variable goes down, so does the othe
Negative correlation
as one variable goes up, the other goes down
Confidence Intervals
Provides the range of values within which the true population
correlation is likely to be found
Variance
The sum of squared deviations from the mean
* The squaring is a computational convenience but has no other rationale
Not a good way to explain the magnitude of a correlation
Pearson’s r
r=.05 (very small); r=10 (small); r=.20 (medium); r=.30 (large); r=.40
(very large)
Cohen’s d
d=0.2 = small effect; d=0.5 = moderate effect; d=0.8 = large effect.
Replication
Finding the same result repeatedly, with different
participants and in different labs
Publication bias
Studies with strong results are more likely to
get published
* Many small studies with weak effects do not get reported
◦ Some researchers only report selected analyses
◦ Researchers are also rewarded for interesting results
Questionable research practices or p-hacking:
Hacking around in one’s data
until one finds the necessary degree of statistical significance that allows one’s
findings to be published
Researchers may:
1. Delete unusual responses
2. Adjust results to remove the influence of seemingly extraneous factors
3. Neglect to report experimental conditions or experiments that fail to get
expected results
Replication
How to make research more dependable
Use larger numbers of participants
* Disclose all methods
* Share data
* Report studies that don’t “work
Ethical Issues: Purposes of Personality Testing
There are several ways in which understanding personality can help society:
* Learning about people (researchers, government agencies)
* Helping people (schools, career counselors, clinicians)
* Selection or retention (employers, Central Intelligence Agency)
However, testing is controversial: But do you trust the test or the person more?
Neither is perfect
Question the Uses of Psychological Research
Psychological research might be used for harmful purposes
* The subfield of behaviorism has tried to control behavior
* Studying race may lead to more prejudice and discrimination
* For a study on any topic, it is worth asking:
* Why is this research being done?
* How will the results of this research be used?
Representation
Representation of various populations among participants is far from
ideal
* WEIRD samples abound—Western, Educated, Industrial, Rich,
Democratic
Psychologists also lack diversity
* Limited diversity in researchers has led to limited diversity in
research
Efforts toward inclusiveness are becoming more common
* We should be exploring the unknown
The Fundamental Attribution Error
basing someone’s behavior on something they have done without knowing them (Ha! I knew they were a bad person because they just did X)
The Correspondence Bias
giving yourself praise and credit when you do something right but when you do something wrong its not your fault
Trait:
A relatively stable and long-lasting attribute of personality
“an enduring personality characteristic that describes or
determines an individual’s behavior across a range of situations”
Most research within the trait approach relies on correlational
design
Traits should be able to predict behavior
This approach focuses on individual differences
How do people differ from one another? (We are all unique, but
zooming out—but what are the overarching, important ways
that we differ from others?)
Strength: Assesses and attempts to understand how people
differ
Weakness: Neglects aspects of personality common to all
people and how each person is unique
Personality traits are not the only factors that control
behavior; people may act differently depending on
the situation. Some other reasons for inconsistency
Might be age
* Older people have embarked on a career path
and may have started families
* Younger people are still forming their identities
* May also be related to mental health
* Consistent people are less neurotic, more
controlled, more mature, and more positive in
their relations with others
person-situation debate
focuses on this
question: Which is more important for determining
what people will do: The person or the situation?
The situationist argument has three parts
The upper limit as to how well a person can
predict another’s behavior is low
2. Situations must be more important than
personality traits (i.e., stronger predictors of
behavior)
3. Everyday intuitions about people are wrong,
because people see others as being more
consistent across situations than they really are
SITUATIONISM’S VIEW OF HUMAN NATURE
People are free to do whatever they want.
* Everybody is equal, and differences are a function of
the situation.
* “If the situation can really be all powerful, then
nothing we do is ever really our fault” (p. 138)
PERSONALITY’S VIEW OF HUMAN NATURE
Behavior is partly determined by personality.
Every individual is unique.
People can develop consistent identities and styles that
allow them to be themselves across situations.
Situationists argue that the predictive capacity of
traits is limited
Mischel (1968) observed that correlations rarely
exceeded r = .30
* The implication for many personality researchers
is that the correlations are so small that
personality traits don’t matter
There are three complications to consider
The Response
from
Personality
Scientists
Some people may be more consistent than others
* Behaviors of a person vary around their average
level from occasion to occasion
* You need to know how the person will act, in
general, across the various relevant situations of
life
Resolution of the
Person-Situation
Debate
People maintain their personalities even as
they adapt their behavior to particular
situations”
People can flexibly adapt to situations AND
have a generally consistent personal style.
Conclusion: People are psychologically
different, and these differences matter.
Mischel’s (2009) conclusion to the person-
situation debate.
Interactionism
The effect of a personality variable may depend
on the situation, or vice versa.
* Certain types of people go to or find themselves
in different types of situations.
* People change the situations that they are in
