Research Methods

Question 1

What’s a good guideline for reviewing manuscripts?

Answer 1

(Journal of Counseling Psychology Reviewer Guidelines, 2013)

Question 2

A good hypothesis is a ___________ research question that provides direction for an experimental inquiry

Answer 2

testable (Heppner, Wampold, & Kivlighan, 2008)

Question 3

Hypotheses should also be phrased as ______ statements so that they can be tested.

Answer 3

falsifiable (Popper, 1959)

Question 4

research designs often offer a trade-off between ______ and __________ validity

Answer 4

internal and external (Heppner, Wampold, & Kivlighan, 2008)

Question 5

What is an error that often occurs in reporting reliability and validity?

Answer 5

Authors don’t reference the sample from which estimates are derived (See Wilkinson, L. and the Task Force on Statistical Inference APA Board of Scientific Affairs, 1999).

Question 6

What are some reliability statistics?

Answer 6

Cronbach’s alpha, KR-20, Intraclass Correlation Coefficients, Kappa, or test-retest reliability (Heppner, Wampold, & Kivlighan, 2008)

Question 7

What are some different types of validity that are important?

Answer 7

1 face validity (the degree to which a measure appears “at face value” to measure the construct),
2 content validity (the content is relevant to the construct),
3 construct validity (the degree to which the measure accurately reflects the construct),
4 predictive validity (measure’s ability to predict something it should theoretically be able to predict),
5 concurrent validity (ability for measure to distinguish between groups it should theoretically be able to distinguish from),
6 convergent validity (degree to which measure is associated with measures it should be theoretically similar to),
7 discriminant validity (degree to which measure is dissimilar to measures it is theoretically dissimilar to).

Question 8

What are 4 different types of studies that vary in their different levels of internal and external validity?

Answer 8

(Gelso, 1979) Experimental Field (Moderate I, Moderate E), Descriptive Field (Low I, High E), Experimental Laboratory (High I, Low E), Descriptive Laboratory (Low I, Low E)

Question 9

What is MAXMINCON?

Answer 9

Kerlinger, 1986
1 MAXimize variance of experimental (important variables)
2 MINimize error variance
3 CONtrol for confound variables

Question 10

Pros of MAXMINCON

Answer 10

• greater likelihood of significant effects

- better able to establish causality (if design affords this)

Question 11

Cons of MAXMINCON

Answer 11

may not apply to applied settings where “perfect” control is not possible

Question 12

What are methods of statistical control?

Answer 12

multiple regression, ANCOVA, partial correlations, residualizing

Question 13

What are disadvantages of statistical control (compared to experimental control)?

Answer 13

• Assumes a linear relationship between confound variables and outcome variable
• Statistically controls for a “measured” confound variable. It is possible that aspects of the confound are not accounted for by the measurement, which always includes error.
• Cannot definitely rule out confounds, and therefore attribution of causality is attenuated
• May not be able to account for problems due to not being able to randomly assign participants
• Collinearity can be a problem if the confound variable (which is controlled for) correlates with predictor variables.

Question 14

What are citations for describing the differences between quantitative and qualitative research?

Answer 14

Johnson & Christensen (2008); Lichtman (2006)

Question 15

Qualitative research assumes there are multiple realities, in fact as many as there are participants. What field does this come from, and what ontology does this represent?

Answer 15

anthropology

interpretivist-constructivist relativist ontology
Guba & Lincoln, 1994

Question 16

What aspect of people does qualitative research examine?

Answer 16

experiential life of people (Polkinghorne, 2005)

Question 17

What is the definition of quantitative psychology research?

Answer 17

psychological research that performs mathematical modeling and statistical estimation or statistical inference or a means for testing objective theories by examining the relationship between variables (Cresswell, 2009)

Question 18

What sort of ontology does quantitative psychology research utilize?

Answer 18

Assumes a modified objectivist epistemology, viewing objectivity as an ideal (Guba & Lincoln, 1994)

Question 19

What is it called when you combine both qualitative and quantitative research?

Answer 19

Mixed methods (Hanson et al., 2005)

Question 20

What are advantages of a brief form measure?

Answer 20

• Participants more likely to complete (Robins et al., 2002)
• Eliminate item redundancy (robins et al., 2001)
• Long precedent for single-item measures for things such as subjective well-being, cultural/ethnic identity, relationship intimacy, intelligence, self-esteem, etc. (Gosling et al., 2003)
• Briefer
• Reduce test fatigue

Question 21

What are disadvantages of brief form measures?

Answer 21

• At best may only be a “reasonable proxy” (Gosling et al., 2003)
• Not accurate, not reliable, validity suspect

Question 22

According to Cohen (1994) significance testing does not tell us, “what is the probability that Ho is true.” Instead it tells us what?

Answer 22

“Assuming that Ho is true, what is the probability of these extreme data?” (not what we really want to know)

Question 23

Cohen (1994) refers to null hypotheses where the null effect size = 0 as the ________ hypothesis.

Answer 23

“nil hypothesis.”

Question 24

What is the bias against publishing negative (null) results called

Answer 24

the file drawer problem (Rosenthal, 1979)

Question 25

Who said, “amount as well as direction is vital” (in advocating confidence intervals)

Answer 25

Tukey (1969)

Question 26

Because the statistical size of an effect is heavily
dependent on the ____________ of the independent variables and the choice of a dependent variable in a particular study, a researcher can design
an experiment so that even a ______ effect is impressive.

Answer 26

(Prentice & Miller, 1992) (Rosenthal, 1990)
operationalization

small

Question 27

Questions involving moderators address _________or __________ a variable most strongly predicts or causes an outcome variable.

Answer 27

“when” or “for whom” (Frazier et al., 2004)

Question 28

mediators establish ________ or ___________one variable predicts or causes an outcome variable

Answer 28

“how” or “why” (Frazier et al., 2004)

Question 29

moderators often are introduced when there are unexpectedly __________ or _________relations between a predictor and an outcome across studies

Answer 29

weak or inconsistent (Baron & Kenny, 1986)

Question 30

How do you judge if a qualitative study is adequate/credible/matching reality?

Answer 30

Lincoln and Guba (1985):

1) Triangulation
2) Member checks (taking data interpretations back to participants)
3) Peer experimentation (colleagues comment on findings)
4) Involving participants in all phases of research
5) Clarifying researcher bias at the outset
6) Prolonged engagement in the field (long term and repeated observations)

Question 31

How do you judge if a qualitative study is transferable or applicable?

Answer 31

1) use of rich, thick description (so the readers can make judgments about what is
applicable/transferable)
2) examination of the typical or modal category (rather than the unique or obscure)
3) use of cross – site analysis (to have a greater number of cases to draw from)

Question 32

SEM and path analysis citation

Answer 32

Hoyle, 2012

Question 33

Judd et al (1995) make 3 recommendations about data analysis, what are they?

Answer 33

1. use model comparison approach
2. Compute estimates of effect size
3. compute effect estimates for interactions