Data Quality & Research Design 2

Question 1

Psychometrics regards two questions as paramount:

Answer 1

are the data reliable? are the data valid?

Question 2

Generalizability

Answer 2

the degree to which a measurement can be obtained
under diverse circumstances, such as time, context, participant population,
etc.
◦ the degree to which you can apply the results of your study to a broader context
◦ Is the result you get with one test largely equivalent to the result you would get using a
different test?

Question 3

Reliability: what it is and what affects it

Answer 3

The hallmark of reliability is consistency
* Measurement error means that the less error, the more reliable the
measurement.

Question 4

At least four factors will affect measurement:

Answer 4

1. Low precision
2. The state of the participant
3. The state of the experimenter
4. The state of the environment

Question 5

Reliability: how to improve it

Answer 5

At least four factors will enhance reliability:
1. Proofreading
2. Using a script
3. The person should match what is trying to be measured.
4. Aggregation
◦ Helps cancel out errors

Question 6

Validity:

Answer 6

The degree to which a measurement measures
what it is supposed to

Question 7

For a measure to be valid, it must be reliable

Answer 7

Constructs: Constructs cannot be directly
seen, but affect and help to explain things
that are visible
* Construct validation: Developing a group of
different measurements that yield more or
less the same result

Question 8

Research Design: Case Method

Answer 8

Closely studying a particular event or person to find
out as much as possible

Question 9

Advantages:

Answer 9

It describes the whole phenomenon—or the whole person—and
not just isolated variables
2. It can illuminate and maybe prevent future disasters
3. We need it to deal with particular individuals and events that
are not common
Disadvantage: Unknown generalizability

Question 10

Experimental Study

Answer 10

Test differences between groups to determine if the difference is larger
than would be expected by chance

Question 11

Independent variable

Answer 11

Imposed by the experimenter and is not affected by (and
therefore independent of ) any characteristic or behavior of the participants

Question 12

Dependent variable (DV):

Answer 12

Assumed to be dependent on, or caused by, the independent
variable

Question 13

Correlational Study:

Answer 13

Correlational method: A research technique
that establishes the relationship between two variables by
measuring both variables as they occur naturally in a sample of
participants
* There are no experimental groups
* Questionnaires are administered

Question 14

Comparing & Contrasting
Experimental and Correlational Methods

Answer 14

Both attempt to assess the relationship between two variables
* The statistics (with two groups) are interchangeable
* The experimental method manipulates the presumed causal
variable, and the correlational method measures it.
* Reasons for not knowing causal direction in correlational
studies:
* Third-variable problem
* Unknown direction of cause (the directionality problem)

Question 15

Experimental and Correlational Methods

Answer 15

Complications with experiments
◦ Uncertainty about what was really manipulated
 A version of the third-variable problem
◦ Can create unlikely or impossible levels of a variable (sledgehammer manipulation)
◦ Often require deception (ethically precarious)
◦ Not always possible
* Takeaway:
* Experiments are not always better
* An ideal research program includes both designs