Research Design and Statistics

Question 1

[internal validity]
A valid causal inference requires satisfaction
of three criteria: (a) statistical association, (b) temporal precedence, and (c) ________

Answer 1

nonspuriousness

Question 2

Spurious causes are ….

Answer 2

threats to internal validity

Question 3

External validity is the extent to which the causal association can be ___________
to or across variations in study instances

Answer 3

generalized

Question 4

An example of ___________, researchers sometimes use inadequate labels to describe study instances (e.g., label a treatment “progressive relaxation” when the treatment has many additional therapeutic components).

Answer 4

Construct Validity

Question 5

Randomized experiments are considered the gold standard for assessing ________.

Answer 5

causality

Question 6

In _________ trials, an intervention’s effects are examined under real-world conditions.
Such trials often take place outside of academic settings (e.g., community
mental health centers).

Answer 6

effectiveness

Question 7

In ______ trials, an intervention’s effects are examined under ideal circumstances,
particularly with respect to treatment implementation.

Answer 7

efficacy

Question 8

In _________ analyses, researchers
analyze outcome data from participants as a function of their original group assignment,
regardless of their level of exposure to treatment. The analysis is intended
to provide a conservative (and real-world) estimate of the treatment effect because
it is based on cases exposed to varying levels of treatment.

Answer 8

intent-to-treat

missing data is core problem

Question 9

single-case experiments are often designed to

increase _________

Answer 9

internal validity

Question 10

The _______ design is a single-case design that alternates the baseline (A) phase
(intervention absent) with an intervention (B) phase (intervention present). The
outcome of interest is assessed on multiple occasions within each phase.

Answer 10

ABAB

Question 11

In _________ designs, replication of an effect is sought over multiple baselines,
which can reflect different behaviors, settings, and/or children (just to name
a few).

Answer 11

multiple baseline

Question 12

Although inferential statistical procedures can be used to analyze data from single case
experiments, it is more common for clinicians to rely on ____________ of the data. Visual inspection is often supplemented with descriptive statistics.

Answer 12

visual inspection

Question 13

clinicians can examine _______ by comparing the averagenfrequency of the outcome across different phases of the experiment (e.g., during
the A vs. B phases).

Answer 13

mean changes

Question 14

Clinicians can also examine \_\_\_\_\_\_\_\_ in which they compare the last data point in an immediately prior phase to the first data point in
an immediately subsequent phase. If the latency of response is hypothesized to
be immediate (e.g., the behavior will reduce dramatically as soon as the intervention
is implemented), one might predict dramatic level changes between adjacent
(baseline-intervention) phases.

Answer 14

level shifts

Question 15

Clinicians can also examine _______ (or functional
form) changes by examining the rate of behavior change in different phases. For
example, the behavior might increase in a fairly linear (i.e., constant) manner during
the initial A phase and become fairly stable during the initial B phase

Answer 15

slope

Question 16

Quasi-experimental studies are experiments that lack _____ of units
to conditions.

Answer 16

random assignment

Question 17

_____________ (also called passive observational studies) are conducted
when the researcher is not actively manipulating anything (like exposure to an
intervention).

Answer 17

Correlational studies

Question 18

________ designs compare a group of participants who possess a certain characteristic
(e.g., diagnosis of attention deficit hyperactivity disorder [ADHD]) with
a group of participants who do not possess the characteristic.

Answer 18

Case–control

Question 19

In \_\_\_\_\_\_\_\_\_\_, an intact group (i.e., cohort) is followed over time to examine the
emergence of—and/or change—in some outcome of interest. These designs are classified
as longitudinal (also known as prospective) because individuals are assessed
on at least two occasions.

Answer 19

cohort designs

Question 20

If the multiple cohorts also differ in their age or some other salient developmental marker at the study’s inception, the study is called a __________
design. Such designs allow for the study of a longer developmental period over fewer
years of data collection because several developmental cohorts (e.g., toddlers, preschoolers,
and school-aged children) are embedded in the study.

Answer 20

cross-sequential

Question 21

______ is a threat to validity when naturally occurring changes are mistaken
for an intervention effect—when symptoms remit because of the passage of time
rather than the effects of an intervention.

Answer 21

Maturation

Question 22

____ is a threat to validity when some event (or constellation of events) occurs
during the study and impacts the results in a manner mistaken for an intervention
(e.g., pt exercising and it helps depression vs. treatment).

Answer 22

History

Question 23

____—also known as regression to the mean—occurs when
extreme scores tend to revert back to the mean on a subsequent evaluation.
Statistical
regression is more plausible in single-group studies in which extreme
performers (e.g., severely depressed individuals) comprise the study sample.

Answer 23

Statistical regression

Question 24

______ is a threat to validity when the pattern of participant drop-out impacts
the results in a way interpreted as an intervention effect.

Answer 24

Attrition

Question 25

____ is a threat to validity when exposing individuals to the pretest changes
them in ways that might be mistaken for an intervention effect.

Answer 25

Testing

Question 26

______ is a threat to validity when the measurement tool changes and
impacts the results in a manner mistaken for an intervention effect.

Answer 26

Instrumentation

Question 27

____ occurs in multiple-group studies when systematic differences among
intervention groups can be mistaken for an intervention effect.

Answer 27

Selection

Question 28

As such, the reliability of a measure is viewed as
the ratio of true score variance to ______ (True score variance is consistent
with consistency or dependability, concepts that are often invoked in discussions
of reliability.)

Answer 28

total variance.

Question 29

Kuder–Richardson Formula 20—often

abbreviated KR-20—which can be used when the items are

Answer 29

dichotomous

Question 30

______—also known as internal structure—is the extent to which the
structure of the measure is consistent with the theorized factor structure of the construct.

Answer 30

Structural validity

Question 31

____ matrix can be used to evaluate convergent and discriminant validity.

Answer 31

multitrait–multimethod

Question 32

In ____, three
primary decisions involve (a) choosing a method of factor extraction, (b) choosing
a method of factor rotation, and (c) deciding on the number of factors to retain.

Answer 32

EFA

Question 33

In _______ rotations, the factors are
assumed to be uncorrelated. In oblique rotations, the factors are assumed to be
correlated—often the case in psychology.

Answer 33

orthogonal

Question 34

Several _______ can help with this
last decision—including the chi-square test, root mean square error of approximation
(RMSEA), and standardized root mean square residual (SRMR). All fit indices
quantify (albeit in slightly different ways) how well the model-implied covariance
matrix reproduces the estimated population covariance matrix of the analysis
variables.

Answer 34

fit indices [CFA]

Question 35

One of the advantages of conducting
statistical analyses on_______ (as in the case of structural equation
modeling, described below), is the gain in statistical power that results when
measurement error is removed from the constructs of interest.

Answer 35

latent variables

Question 36

The ___ is used when (a) data are ordinal, or (b) when data are interval or ratio, but the
distribution is highly skewed (the median is less affected by skewness)

Answer 36

median

Question 37

In_______, the mean and median are identical.
In symmetrical
distributions with a single mode, the mean, median, and mode are identical

Answer 37

symmetrical distributions

Question 38

The interquartile range captures the middle 50% of the distribution and is
computed by subtracting the 25th percentile (first quartile) from the _____th percentile
(third quartile)

Answer 38

75

Question 39

The standard deviation (SD) captures the average distance of scores from the

Answer 39

mean

Question 40

z = (x − M) / _______ .

Answer 40

SD

Question 41

Properties of the Normal z Distribution When a z-score conversion is used,
the resulting z distribution has the following properties: (a) the mean is ___; (b) the
standard deviation is 1; and (c) each z score represents the position of the score in
relation to the mean, in standard deviation units. In other words, a z score of −1.27
denotes a score exactly 1.27 standard deviations below the mean.

Answer 41

0

Question 42

percentiles can be computed by conversion of T to z: (T – 50)/______.

Answer 42

10

Question 43

__________: (a) make
more distributional assumptions (e.g., that the distribution is normal), (b) assume
data are measured on an interval or ratio scale, (c) are conducted on actual data
(as opposed to on ranks derived from data), and (d) allow researchers to test more
specific hypotheses about the populations from which they are drawn.

Answer 43

parametric statistics

Question 44

the null hypothesis specifies

that the two ________ are equal (mt = mc).

Answer 44

population means

Question 45

When the sample data
would occur relatively infrequently assuming the null hypothesis (e.g., the data
would occur less than 5% of the time if the null hypothesis were true

Answer 45

p < .05 (sig level; alpha)

Question 46

results are not declared statistically significant even though the null hypothesis
is false

Answer 46

Type II error (Beta)

Question 47

Results were excepted even though the null was true

Answer 47

Type I error (alpha)

Question 48

_______ is the probability of correctly rejecting a false null hypothesis (i.e., finding an effect when one exists in the population)

Answer 48

Statistical power (1 -Beta)

Question 49

What helps increase power?

Answer 49

Increase N;
Increase alpha;
Directional hypotheses;
Large effects;
More reliable measures

Question 50

When the 95% confidence interval does not contain zero the results are:

Answer 50

significant

Question 51

An example of \_\_\_\_\_\_\_\_\_: (a) no longer meeting diagnostic
criteria and (b) scoring two standard errors below a pretest score on one of the
primary study outcomes.

Answer 51

Clinical significance (vs statistical)

Question 52

two-tailed or nondirectional means:

Answer 52

(leaving open the possibility that the sample mean will be larger or smaller than
the population mean of 100).

Question 53

A Single-sample Z test is used when

Answer 53

The population SD is known (e.g., intelligence tests).

Question 54

A single-sample T test is used when

Answer 54

The population SD is unknown.

Question 55

A relevant effect size is C______, which is a standardized mean difference and is
computed as the ratio of the difference between the two sample means to the
pooled standard deviation.

Answer 55

Cohen’s d (.2 small, .5 med, .8 large).

Question 56

omega squared, ω2—reflect the proportion of variance
in the outcome that is explained by the factors. Interpretive guidelines for omega
squared are small = 0.01, medium = 0.06, and large = 0.15.

Answer 56

Effect size used in ANOVA

Question 57

In models for which
there are three or more levels of a factor (e.g., low, medium, and high levels of
stress), the test of the factor’s main effect is an _______ statistical test.

Answer 57

omnibus

Question 58

omnibus tests are typically followed
by a series of additional tests (e.g., comparing each pairs of means). These
more focused contrasts are often referred to as _____.

Answer 58

post hoc tests

Question 59

Conducting multiple
statistical tests raises the familywise type I error rate associated with the full set of
analyses, so we use ______.

Answer 59

Corrections (e.g., Bonferoni, Tukey)

Question 60

A ____________ can be used
to contrast two or more treatment groups (e.g., CBT, interpersonal therapy [IPT],
and control).

Answer 60

one-way between-subjects ANOVA

Question 61

A ___________ can be used to
examine a single cohort’s symptom levels over two or more assessments (e.g., pretest,
posttest, and follow-up measures for individuals exposed to a single intervention).

Answer 61

one-way within-subjects ANOVA

Question 62

What test design is this an example of:
one might be interested in examining whether ADHD diagnosis (present or absent)
and testing environment (quiet or noisy room) have an impact on performance.

Answer 62

two-way between subjects ANOVA

Question 63

What test design is this an example of:
exposing a group of children with ADHD to two levels of a psychostimulant drug dose
factor (e.g., 5 and 10 mg) crossed with two levels of a testing-environment
factor (e.g., quiet and noisy rooms). In other words, all children would be observed
under all four study conditions (e.g., 10 mg, quiet room) and performance would
be the dependent variable

Answer 63

two-way WITHIN subjects ANOVA

Question 64

\_\_\_\_\_\_\_\_\_ are used in two primary manners: (a) to increase statistical power
in randomized experiments (when the covariate is uncorrelated with intervention
conditions, but correlated with the dependent variable); and (b) to control for possible
confounding influences (i.e., controlling for variables associated with both
intervention conditions and the dependent variable) in nonrandomized designs.

Answer 64

ANCOVAs

Question 65

the multivariate ANOVA model (i.e.,

MANOVA model) allows for _______ to be analyzed in a single model.

Answer 65

multiple dependent variables

Question 66

In MANOVA, the actual analysis is performed on an optimized _________ (one that maximizes between group
differences while minimizing within-group differences). A number of test
statistics are generated by MANOVA (Pillais’s trace, Wilk’s lambda, Hotelling’s
trace, and Roy’s largest root).

Answer 66

linear combination of the multiple dependent variables

Question 67

Effect size is estimated by computing the square of the correlation (i.e., the coefficient
of determination), which is the proportion of shared variance between the
two variables. Common interpretative guidelines for r2 are as follows: small = .01,
medium = .09, and large = .25.

Answer 67

Correlation effect size (r2)

Question 68

Spearman’s
rank correlation coefficient and Kendall’s tau coefficient are both nonparametric
tests that are used when responses on the two variables are ________.

Answer 68

rank ordered

Question 69

unstandardized linear regression coefficient (b) meaning):

Answer 69

for every one-unit increase in x, Y changes by b units

Question 70

(Hierarchical regression analysis is sometimes confused with stepwise regression analysis,
which is an _______ approach to predictor entry used more often in exploratory
analyses.)

Answer 70

atheoretical

Question 71

Models that

do not include interaction effects are referred to as _______.

Answer 71

additive effects models

Question 72

The Mann–Whitney Test (also called the Mann–Whitney–Wilcoxon Test) is a nonparametric
alternative to the _________

Answer 72

independent samples t-test

Question 73

The Kruskal–

Wallis Test is a nonparametric alternative to the ______.

Answer 73

between-group ANOVA

Question 74

The Wilcoxon Signed Ranks Test is a nonparametric

alternative to the __________.

Answer 74

paired samples t-test.

Question 75

Testing for moderation is the same as testing for a _________
between a predictor and a moderator. For example, a researcher might test
whether an intervention effect is moderated by participant sex to see whether the
effects of the intervention on the outcome are stronger for men or women

Answer 75

statistical interaction

Question 76

A _______ is the mechanism through which a distal predictor operates in influencing an
outcome.

Answer 76

mediator

Question 77

________ centers on understanding participants’ lived experiences and
emphasizes subjective experience (e.g., understanding personal knowledge, motivations,
and perspectives).

Answer 77

Phenomenology

Question 78

The ultimate goal of ______ is to develop a theory (“grounded” in data)
about a concept of interest. This approach is used when current theory is lacking,
nonexistent, or incomplete.

Answer 78

grounded theory

Question 79

qualitative research data are often analyzed
using __________, a process of identifying and analyzing patterns or themes
within data. This can occur either deductively (i.e., starting from a particular theory
or hypothesis) or inductively (i.e., as in grounded theory above)

Answer 79

thematic analysis

Question 80

1) triangulation (the use
of multiple, varied sources of data, methods, and researchers in order to corroborate
results), 2) audits (the use of an external consultant to complete an independent
analysis), and 3)member checking (having participants in the study review and provide
feedback on the credibility of findings).

Answer 80

Checking reliability and validity in qualitative research

Question 81

The first step in ________ is to identify and engage stakeholders (e.g., administrators,
staff, clients, etc.). Next, a needs assessment might be conducted to assess
the relative priority of the needs, or “problems,” of a specific population in order
to determine where resources should be allocated

Answer 81

Program Evaluation

Question 82

Formative evaluations provide information to make needed changes early on, whereas _____ determine a program’s success once delivered.

Answer 82

summative evaluations

Question 83

CBA examines the balance of resources/
costs spent on a program compared to the benefits to answer the question “have
resources been well spent on this program?” This results in a benefit/cost ratio, the
worth of a program’s outcomes divided by the program’s costs, which can then be
compared to alternative programs. CBA is controversial in part because it assigns
monetary values to the benefits arising from a program.

Answer 83

Cost-benefit Analysis

Question 84

_______ sampling is when researchers collect data from individuals with specific
characteristics.

Answer 84

Purposive

Question 85

_______ sampling, a type of purposive sampling, involves participants
inviting others to participate in the study.

Answer 85

Snowball

Question 86

______ sampling uses
incentives to overcome possible biases that result from snowball and other chain-referral
sampling methods.

Answer 86

Respondent-driven

Question 87

In other words, unlike _______ variables which provide a causal explanation for the relationship between variables, moderator variables affect the strength of the relationship.

Answer 87

mediator

Question 88

_________ is useful for studying behaviors that occur infrequently, have a long duration, or leave a permanent record or other product (e.g., a completed worksheet or test).

Answer 88

Event sampling

Question 89

_______ is an alternative to behavioral sampling and is used when a goal of the study is to observe a behavior in a number of settings. It helps increase the generalizability of a study’s findings.

Answer 89

Situational sampling

Question 90

the independent variable (experimental variance);
• systematic error (error due to extraneous variables); and
• random error (error due to random fluctuations in subjects, experimental conditions, methods of measurement, etc.).

Answer 90

three factors that can cause variability in the study’s dependent variable.

Question 91

when a researcher includes an extraneous variable as an independent variable in a study, the extraneous variable is also known as a ________ variable

Answer 91

moderator

Question 92

________ is a particularly useful method in quasi-experimental research in which subjects cannot be randomly assigned to treatment groups.

Answer 92

Statistical control (such as using covariates)

Question 93

A study has internal validity when it allows an investigator to determine if there is a ____ relationship between independent and dependent variables.

Answer 93

causal

Question 94

Fatigue, boredom, hunger, and physical and cognitive development are
potential ______ effects that can limit a study’s Internal validity

Answer 94

maturational;
The best way to control maturation is to include more than one group in the study and randomly assign subjects to groups.

Question 95

History is controlled by including more than one group in the study and _________ subjects to groups.

Answer 95

randomly assigning

Question 96

The threat of _______ can be controlled by administering the DV measure only once as a posttest, by designing the measure in a way that minimizes memory and practice effects, or by including at least two groups in the study with all groups completing the pre- and posttests so that any difference between groups on the posttest

Answer 96

Testing

Question 97

_______ is controlled by including more than one group in the study and ensuring that all groups are subject to the same instrumentation effects, by using the same measuring devices and procedures with all subjects, and by making sure that measuring devices and procedures do not change during the course of the study.

Answer 97

Instrumentation

Question 98

The _________ threat is avoided by not including only extreme scorers in the study or by including more than one group and ensuring that all groups consist of subjects who are similarly extreme.

Answer 98

statistical regression

Question 99

______ is difficult to control, but pretesting can help determine if dropouts and non-dropouts differ with regard to their initial status on the DV.

Answer 99

Attrition

Question 100

Population validity is generalization to other people and ecological validity is generalization to other ______.

Answer 100

settings

Question 101

When a study’s results have been contaminated by _______, they cannot be generalized to people who have not been pretested. Pretest · sensitization is controlled by not administering a pretest or by using the Solomon four-group design, which allows an investigator to measure the impact of pretesting on both the external and internal validity of a research study.

Answer 101

pretest sensitization

Question 102

An interaction between _________ is often a problem when subjects are volunteers because volunteers tend to be more motivated than non-volunteers and, consequently, might be more responsive to the IV. In this situation, the study’s results
apply to volunteers but can’t be generalized to other people. The best way to eliminate this threat is to ensure that the sample is representative of the population of interest.

Answer 102

selection and treatment

Question 103

Research participants may respond to an independent variable in a particular way simply because they know their behavior is being observed, and this is known as _____

Answer 103

reactivity

Question 104

The behavior of subjects can also be altered by __________, which are cues in the experimental setting that inform subjects of the purpose of the study or suggest what behaviors are expected of them.

Answer 104

demand characteristics

Question 105

______can be controlled by using deception, unobtrusive (nonreactive) measures, or a single- or double-blind technique. When using a single-blind technique, subjects do not know which treatment group they have been assigned to; in a double-blind study, neither the subjects nor the experimenter know which group subjects have been assigned to.

Answer 105

Reactivity

Question 106

When a study involves exposing each subject to two or more levels of an independent variable (i.e., when the study utilizes a within-subjects design), the effects of one level of the independent variable can be affected by previous exposure to another level.

Answer 106

(Order Effects, Carryover Effects)

Question 107

The _______ designs are considered inappropriate when withdrawal of a treatment during the course of a research study would be unethical (e.g., when the treatment has successfully eliminated a self-injurious behavior)

Answer 107

reversal (ABAB)

Question 108

Because of its insensitivity to “outliers,·· the _____ is a useful measure of central tendency when a distribution contains one or a few extreme scores.

Answer 108

median

Question 109

One advantage of the _____ is that, of the three measures of central tendency, it is least susceptible to sampling fluctuations.

Answer 109

mean

Question 110

Regardless of the shape of the distribution of individual scores in the population, as the sample size increases, the sampling distribution of the mean approaches a normal distribution.
• The mean of the sampling distribution of the mean is equal to the population mean.
• The standard deviation of the sampling distribution of the mean is equal to the population standard deviation divided by the square root of the sample size:

Answer 110

Central Limit theorem

Question 111

The ________ is the foundation of inferential statistics. It is the sampling distribution that enables a researcher to make inferences about the relationship between variables in the population based on obtained sample data.

Answer 111

sampling distribution

Question 112

A _____ error is more likely when alpha is low, the sample size is small, and the independent variable is not administered in sufficient intensity.

Answer 112

Type II

Question 113

One tailed tests, when appropriate, and parametric tests vs non are helpful to increase _____.

Answer 113

Power

Question 114

The most effective way to maximize the robustness of a parametric test is to have an equal number of _______.

Answer 114

subjects in each group

Question 115

Numerator and denominator of F statistic are:

Answer 115

Mean Square Between / Mean Square Within

Question 116

ANCOVA and randomized block ANOVA serve to decrease ___________ variability in order to create a stronger test.

Answer 116

within group

Question 117

Put another way, the ___________ indicates the proportion of variability in Y that is
explained by, or accounted for by the variability in X. For example, if the correlation
coefficient for sales success and product knowledge is .60, then 36% (.60 squared= .36) of variability in sales success is accounted for by product knowledge.

Answer 117

squared correlation coefficient

Question 118

If the subscript contains two different letters or numbers (e.g., “xy”), it represents the correlation between two different variables. When the subscript contains the same letters or numbers (e.g., ‘‘xx”), it is a _________.

Answer 118

reliability coefficient

Question 119

Canonical correlation is an extension of multiple regression that is used when two or more continuous predictors are to be used to predict status on _______.

Answer 119

two or more continuous criteria

Question 120

________ analysis is also known as discriminant analysis and is the appropriate technique when two or more continuous predictors will be used to predict or estimate a person’s status on a single discrete (nominal) criterion.

Answer 120

Discriminant function; examines “hit rate” or number of correct classifications