Quantitative Data Collection

Question 1

What is “goodness of fit”?

Answer 1

Goodness of fit between:

• purpose
• design
• research question(s) or hypotheses
• conceptual and operational definitions
• data collection method

Question 2

What is “data consistency”?

Answer 2

Following data collection protocols to ensure intervention fidelity.
Measuring the data in the same manner for each subject.
Ensures interrater reliability - consistency of observations reflected as a coefficient kappa (Cohen’s kappa).

Question 3

What is “interrater reliability??

Answer 3

Consistency of observations reflected as a coefficient kappa (Cohen’s kappa).

Question 4

What are some methods of data collection?

Answer 4

Physiological measurements
Observational methods
Interviews
Questionnaires
Records or available data

Question 5

What is “reliability”?

Answer 5

The consistency with which the instrument measures the concept of interest.

Question 6

What are three aspects of reliability?

Answer 6

Stability (test-retest reliability)
Homogeneity/internal consistency
Equivalence/Interrated reliablity (Cohen’s kappa)

Question 7

What is “validity”?

Answer 7

The extent to which an instrument actually measures or reflects the abstract construct.

Question 8

What are two measures of validity?

Answer 8

Expert opinion/expert panels

Comparisons to other scales, events, etc.

Question 9

What is the purpose of “control” as part of quantitative design?

Answer 9

Measures that researches use to hold the conditions of the study uniform and avoid possible impingement of bias (extraneous variables) on the dependent variable.

Question 10

What are two principles of psychometrics?

Answer 10

Reliability

Validity

Question 11

What are some ways to have “control of extraneous variables”?

Answer 11

Using a homogeneous sample.
Use of consistent data-collection procedures.
Manipulation of the independent variable.
Randomization.

Question 12

What is “internal validity”?

Answer 12

The extent to which the study findings are “true”, rather than the result of extraneous variables. (Factors from WITHIN the study design).

Question 13

What is “external validity”?

Answer 13

The extent to which study findings can be generalized beyond the sample used in the study. (Apply findings OUTSIDE the study)

Question 14

What are some internal threats to validity? (6)

Answer 14

History threats
Maturation effects
Testing effects
Instrumentation threats
Mortality (attrition)
Selection bias

Question 15

What are some external threats to validity? (3)

Answer 15

Selection effects
Reactive effects
Measurement effects

Question 16

What are “history threats”?

Answer 16

History threats (internal threat)
The threat to internal validity that events outside of the experimental setting may affect the dependent variable.

Question 17

What are “maturation effects”?

Answer 17

Maturation (internal threat)
Developmental, biological, or psychological processes that operate within an individual as a function of time and are external to the events of the investigation.

Question 18

What are “testing effects”?

Answer 18

Testing effect (internal threat)
The effect on the scores of a posttest as the result of having taken a pretest.

Question 19

What are “instrumentation threats”?

Answer 19

Instrumentation threats (internal threat)
Changes in the measurement of the variables that may account for changes in the obtained measurement.

Question 20

What is “mortality (attrition)”?

Answer 20

Mortality (attrition) (internal threat)

The loss of a subject from time 1 data collection to time 2 data collection.

Question 21

What is “selection bias”?

Answer 21

Selection bias (internal threat)
The threat to internal validity that arises when pretreatment differences exist between the experimental group and the control group.

Question 22

What are “selection effects”?

Answer 22

Selection effects (external threat)
The threat to external validity that occurs when the ideal sample population participants are either too few or unfavorable to the researcher.

Question 23

What are “reactive effects”?

Answer 23

Reactivity (external threat)
The distortion created when those who are being observed change their behaviour because they know that they are being observed. Also known as the Hawthorne effect.

Question 24

What are “measurement effects”?

Answer 24

Measurement effects (external threats)
Changes in the generalizability of study findings to other populations, as a result of administration of a pretest.

Question 25

What is a double-blind procedure?

Answer 25

A double-blind procedure is a means of reducing bias in an experiment by ensuring that both those who administer a treatment and those who receive it do not know (are blinded to) which study participants are in the control and experimental groups.

Question 26

What is the halo effect?

Answer 26

This is a tendency of judges to overrate a performance because the participant has done well in an earlier rating or when rated in a different area. For example, a student that has received high grades on earlier papers may receive a high grade on a substandard paper because the earlier work created a halo effect.

Question 27

What is the “Rosenthal Effect of Pygmalion Effect”?

Answer 27

Changes in participants’ behaviors brought about by researcher expectations; a self-fulfilling prophecy. The term originally comes from Greek mythology and was popularized by G.B. Shaw. Named from a controversial study by Rosenthal and Jackson in which teachers were told to expect some of their students’ intelligence test scores to increase. They did increase based solely on the teachers’ expectations and perceptions.