Data Quality, Research Design, and Evaluating Research

Question 1

Projective Tests purpose

Answer 1

Reveal the unknown or hidden
aspects of personality to the person and
researcher

Question 2

Basic Method

Answer 2

Give someone an ambiguous
stimulus and ask them to tell you about it

Question 3

Analysis

Answer 3

No clear cut, quantitative results
◦ Analysis of content of stories, letters, and
speeches
◦ Mostly used by clinical psychologists
◦ Provide you with B-Data

Question 4

Objective Tests purpose

Answer 4

For the person to reveal to the researcher
what they think or know about themselves

Question 5

Basic Methods

Answer 5

Basic Methods

Question 6

Rational method

Answer 6

Write items that seem
directly, obviously, and rationally related to what
is to be measured (S-data

Question 7

Factor analytic

Answer 7

Identify which items group
together using factor analysis (mainly S-data)

Question 8

Empirical

Answer 8

Identify items based on how people
in preidentified groups respond (B-data)

Question 9

Experimental Study:

Answer 9

Test differences between groups to determine if the difference is larger than would be expected by chance

Question 10

Independent variable

Answer 10

a characteristic of an experiment that researchers manipulate to see if it causes a change in the dependent variable

Question 11

Dependent Variable

Answer 11

The variable the researcher observes/ measures

Question 12

Correlational Study:

Answer 12

Correlational method: A research
technique that establishes the relationship between
two variables by measuring both variables as they
occur naturally in a sample of participants
There are no experimental groups
* Questionnaires are administered

Question 13

Comparing & Contrasting
Experimental and Correlational Methods

Answer 13

Both attempt to assess the relationship between two (or more)
variables
* The statistics (with two groups) are interchangeable
* The experimental method manipulates the presumed causal
variable, and the correlational method just measures it.
* Reasons for not knowing causal direction in correlational studies:
* Third-variable problem
* Unknown direction of cause (the directionality problem)

Question 14

Experimental and Correlational Methods

Answer 14

Complications with experiments
◦ Uncertainty about what was really manipulated
 A version of the third-variable problem
◦ Can create unlikely or impossible levels of a variable (“sledgehammer
manipulation”)
◦ Often require deception (ethically precarious)
◦ Not always possible
* Takeaway:
* Experiments are not always better
* An ideal research program includes investigations with both designs

Question 15

Statistical significance

Answer 15

A result that would only occur by chance less than 5 percent
of the time

Question 16

Null-hypothesis significance testing (NHST)

Answer 16

Determines the chance of
getting the result if nothing were really going on

Question 17

p-level

Answer 17

Probability of obtaining a result if there is no difference between
groups or no relationship between variables (p<.05; p<.01; p<.001)

Question 18

Problems with NHST

Answer 18

The logic is difficult to describe (and understand)
* The criterion for significance is an arbitrary rule of thumb (although the field has
agreed-upon rules of thumb)
* Nonsignificant results are sometimes misinterpreted to mean “no result” or
no relationship or difference
* Only provides information about the probability of one type of error
* Type I error vs. Type II error

Question 19

Effect size

Answer 19

An index of the magnitude or strength of the relationship between the
variables

Question 20

Correlation coefficient (Pearson’s r)

Answer 20

The most commonly used* measure of effect size
◦ Between -1 and +1
◦ If 0, there is no relationship

Question 21

Positive correlation

Answer 21

as one variable goes up, so does the other; likewise, as one
variable goes down, so does the othe

Question 22

Negative correlation

Answer 22

as one variable goes up, the other goes down

Question 23

Confidence Intervals

Answer 23

Provides the range of values within which the true population
correlation is likely to be found

Question 24

Variance

Answer 24

The sum of squared deviations from the mean
* The squaring is a computational convenience but has no other rationale
Not a good way to explain the magnitude of a correlation

Question 25

Pearson’s r

Answer 25

r=.05 (very small); r=10 (small); r=.20 (medium); r=.30 (large); r=.40
(very large)

Question 26

Cohen’s d

Answer 26

d=0.2 = small effect; d=0.5 = moderate effect; d=0.8 = large effect.

Question 27

Replication

Answer 27

Finding the same result repeatedly, with different
participants and in different labs

Question 28

Publication bias

Answer 28

Studies with strong results are more likely to
get published
* Many small studies with weak effects do not get reported
◦ Some researchers only report selected analyses
◦ Researchers are also rewarded for interesting results

Question 29

Questionable research practices or p-hacking:

Answer 29

Hacking around in one’s data
until one finds the necessary degree of statistical significance that allows one’s
findings to be published
Researchers may:
1. Delete unusual responses
2. Adjust results to remove the influence of seemingly extraneous factors
3. Neglect to report experimental conditions or experiments that fail to get
expected results

Question 30

Replication
How to make research more dependable

Answer 30

Use larger numbers of participants
* Disclose all methods
* Share data
* Report studies that don’t “work

Question 31

Ethical Issues: Purposes of Personality Testing

Answer 31

There are several ways in which understanding personality can help society:
* Learning about people (researchers, government agencies)
* Helping people (schools, career counselors, clinicians)
* Selection or retention (employers, Central Intelligence Agency)
However, testing is controversial: But do you trust the test or the person more?
Neither is perfect

Question 32

Question the Uses of Psychological Research

Answer 32

Psychological research might be used for harmful purposes
* The subfield of behaviorism has tried to control behavior
* Studying race may lead to more prejudice and discrimination
* For a study on any topic, it is worth asking:
* Why is this research being done?
* How will the results of this research be used?

Question 33

Representation

Answer 33

Representation of various populations among participants is far from
ideal
* WEIRD samples abound—Western, Educated, Industrial, Rich,
Democratic
Psychologists also lack diversity
* Limited diversity in researchers has led to limited diversity in
research
Efforts toward inclusiveness are becoming more common
* We should be exploring the unknown

Question 34

The Fundamental Attribution Error

Answer 34

basing someone’s behavior on something they have done without knowing them (Ha! I knew they were a bad person because they just did X)

Question 35

The Correspondence Bias

Answer 35

giving yourself praise and credit when you do something right but when you do something wrong its not your fault

Question 36

Trait:

Answer 36

A relatively stable and long-lasting attribute of personality
 “an enduring personality characteristic that describes or
determines an individual’s behavior across a range of situations”

Question 37

Most research within the trait approach relies on correlational
design

Answer 37

Traits should be able to predict behavior
This approach focuses on individual differences
 How do people differ from one another? (We are all unique, but
zooming out—but what are the overarching, important ways
that we differ from others?)
 Strength: Assesses and attempts to understand how people
differ
 Weakness: Neglects aspects of personality common to all
people and how each person is unique

Question 38

Personality traits are not the only factors that control
behavior; people may act differently depending on
the situation. Some other reasons for inconsistency

Answer 38

Might be age
* Older people have embarked on a career path
and may have started families
* Younger people are still forming their identities
* May also be related to mental health
* Consistent people are less neurotic, more
controlled, more mature, and more positive in
their relations with others

Question 39

person-situation debate

Answer 39

focuses on this
question: Which is more important for determining
what people will do: The person or the situation?

Question 40

The situationist argument has three parts

Answer 40

The upper limit as to how well a person can
predict another’s behavior is low
2. Situations must be more important than
personality traits (i.e., stronger predictors of
behavior)
3. Everyday intuitions about people are wrong,
because people see others as being more
consistent across situations than they really are

Question 41

SITUATIONISM’S VIEW OF HUMAN NATURE

Answer 41

People are free to do whatever they want.
* Everybody is equal, and differences are a function of
the situation.
* “If the situation can really be all powerful, then
nothing we do is ever really our fault” (p. 138)

Question 42

PERSONALITY’S VIEW OF HUMAN NATURE

Answer 42

 Behavior is partly determined by personality.
 Every individual is unique.
 People can develop consistent identities and styles that
allow them to be themselves across situations.

Question 43

Situationists argue that the predictive capacity of
traits is limited

Answer 43

Mischel (1968) observed that correlations rarely
exceeded r = .30
* The implication for many personality researchers
is that the correlations are so small that
personality traits don’t matter

Question 44

There are three complications to consider
The Response
from
Personality
Scientists

Answer 44

Some people may be more consistent than others
* Behaviors of a person vary around their average
level from occasion to occasion
* You need to know how the person will act, in
general, across the various relevant situations of
life

Question 45

Resolution of the
Person-Situation
Debate

Answer 45

People maintain their personalities even as
they adapt their behavior to particular
situations”
 People can flexibly adapt to situations AND
have a generally consistent personal style.
 Conclusion: People are psychologically
different, and these differences matter.
 Mischel’s (2009) conclusion to the person-
situation debate.

Question 46

Interactionism

Answer 46

The effect of a personality variable may depend
on the situation, or vice versa.
* Certain types of people go to or find themselves
in different types of situations.
* People change the situations that they are in