Statistics and Research Design

Question 1

determines the probability of rejecting the null hypothesis when it is true; i.e., the probability of making a Type I error. The value of alpha is set by the experimenter prior to collecting or analyzing the data. In psychological research, alpha is commonly set at .01 or .05.

Answer 1

Alpha

Question 2

is a nonparametric statistical test that is used with nominal data (or data that are being treated as nominal data) - i.e., when the data to be compared are frequencies in each category. The single-sample chi-square test is used when the study includes one variable; the multiple-sample chi-square test when it includes two or more variables. (When counting variables for the chi-square test, independent and dependent variables are both included.)

Answer 2

Chi-Square Test (Single-Sample And Multiple-Sample)

Question 3

is a multivariate technique that is used to group people or objects into a smaller number of mutually exclusive and exhaustive subgroups (clusters) based on their similarities - i.e., to group people or objects so that the identified subgroups have within-group homogeneity and between-group heterogeneity.

Answer 3

Cluster Analysis

Question 4

refers to validating a correlation coefficient (e.g., a criterion-related validity coefficient) on a new sample. Because the same chance factors operating in the original sample are not operating in the subsequent sample, the correlation coefficient tends to “shrink” on cross-validation. In terms of the multiple correlation coefficient (R), shrinkage is greatest when the original sample is small and the number of predictors is large.

Answer 4

Cross-Validation/Shrinkage

Question 5

is the appropriate multivariate technique when two or more continuous predictors will be used to predict or estimate a person’s status on a single discrete (nominal) criterion.

Answer 5

Discriminant Function Analysis

Question 6

is measure of the magnitude of the relationship between independent and dependent variables and is useful for interpreting the relationship’s clinical or practical significance (e.g., for comparing the clinical effectiveness of two or more treatments). Several methods are used to calculate an effect size including Cohen’s d (which indicates the difference between two groups in terms of standard deviation units) and eta squared (which indicates the percent of variance in the dependent variable that is accounted for by variance in the independent variable).

Answer 6

Effect Size

Question 7

research involves conducting an empirical study to test hypotheses about the relationships between independent and dependent variables. A true experimental study permits greater control over experimental conditions, and its “hallmark” is random assignment to groups. A quasi-experimental study permits less control.

Answer 7

Experimental Research (True and Quasi-Experimental)

Question 8

(also known as the familywise error rate) is the probability of making a Type I error. As the number of statistical comparisons in a study increases, the experimentwise error rate increases.

Answer 8

Experimentwise Error Rate

Question 9

refers to the degree to which a study’s results can be generalized to other people, settings, conditions, etc. Threats include pretest sensitization (which occurs when pretesting affects how subjects react to the treatment), reactivity (which occurs when subjects respond differently to a treatment because they know they are participating in a research study), and multiple treatment interference (which occurs when subjects receive more than one level of an IV). Counterbalancing can be used to control multiple treatment interference and involves administering different levels of the IV to different groups of subjects in a different order.

Answer 9

External Validity (Pretest Sensitization, Reactivity, Multiple Treatment Interference)

Question 10

is the appropriate statistical test when a study includes two or more IVs (i.e., when the study has used a factorial design) and a single DV that is measured on an interval or ratio scale. It is also referred to as a two-way ANOVA, three-way ANOVA, etc., with the words “two” and “three” referring to the number of IVs.

Answer 10

Factorial ANOVA

Question 11

are research designs that include two or more “factors” (independent variables). They permit the analysis of main and interaction effects: A main effect is the effect of a single IV on the DV, while an interaction refers to the effects of one IV at different levels of another IV.

Answer 11

Factorial Design (Main And Interaction Effects)

Question 12

is the variable that is believed to have an effect on the dependent variable and is varied or manipulated by the researcher in an experimental research study. Each independent variable in a study must have at least two levels. The dependent variable (DV) is the variable that is believed to be affected by the independent variable and is observed and measured.

Answer 12

Independent and Dependent Variables

Question 13

Internal validity refers to the degree to which a research study allows an investigator to conclude that observed variability in a dependent variable is due to the independent variable rather than to other factors. Maturation is one threat to internal validity and occurs when a physical or psychological process or event occurs as the result of the passage of time (e.g., increasing fatigue, decreasing motivation) and has a systematic effect on subjects’ status on the DV. History is a threat when an event that is external to the research study affects subjects’ performance on the DV in a systematic way. Statistical regression is a threat when subjects are selected to participate because of their extreme status on the DV or a measure that correlates with the DV and refers to the tendency of extreme scores to “regress to the mean” on retesting. Selection threatens internal validity when groups differ at the beginning of the study because of the way subjects were assigned to groups and is a potential threat whenever subjects are not randomly assigned to groups.

Answer 13

Internal Validity (Maturation, History, Statistical Regression, Selection)

Question 14

is a method of behavioral sampling that involves dividing a period of time into discrete intervals and recording whether the behavior occurs in each interval. It is particularly useful for behaviors that have no clear beginning or end. Event sampling is a method of behavioral sampling that is useful for behaviors that are rare or that leave a permanent product. It involves recording each occurrence of a behavior during a predefined or preselected event.

Answer 14

Interval Recording/Event Sampling

Question 15

is a structural equation (causal) modeling technique that is used to verify a predefined causal model or theory. It is more complex than path analysis, and it allows two-way (non-recursive) paths and takes into account observed variables, the latent traits they are believed to measure, and the effects of measurement error.

Answer 15

LISREL

Question 16

is a form of the ANOVA that is used when a study includes one or more IVs and two or more DVs that are each measured on an interval or ratio scale. Use of the MANOVA helps reduce the experimentwise error rate and increases power by simultaneously analyzing the effects of the IV(s) on all of the DVs.

Answer 16

MANOVA (Multivariate Analysis of Variance)

Question 17

The mean, median, and mode are the most commonly used measures of central tendency. The mean is the arithmetic average of a set of scores, and it can be used when scores represent an interval or ratio scale. The median is the middle score in a distribution when scores have been ordered from lowest to highest. It is used with ordinal data (and with interval and ratio data when the distribution is skewed or contains one or a few outliers). Finally, the mode is the most frequently occurring score or category, and it is used as a measure of central tendency for nominal variables or variables that are being treated as nominal variables.

Answer 17

Measures of Central Tendency (Mean, Median, Mode)

Question 18

is a type of factorial ANOVA that is used when a study includes at least one between-groups independent variable and one within-subjects independent variable.

Answer 18

Mixed (Split Plot) ANOVA

Question 19

are a type of factorial design in which at least one IV is a between-groups variable and one IV is a within-subjects variable.

Answer 19

Mixed Designs

Question 20

affect the strength or direction of the relationship between independent and dependent variables. If a treatment is more effective for reducing cigarette smoking for men than for women, gender is a moderator variable. Mediating variables explain or account for the relationship between independent and dependent variables. As an example, authoritative parenting may have positive effects on academic achievement because authoritative parenting leads to high self-efficacy beliefs (the mediator) which, in turn, leads to a high level of academic achievement.

Answer 20

Moderator and Mediator Variables

Question 21

is a multivariate technique that is used for predicting a score on a continuous criterion based on performance on two or more continuous and/or discrete predictors. The output of multiple regression is a multiple correlation coefficient (R) and a multiple regression equation. Ideally, predictors included in a multiple regression equation will have low correlations with each other and high correlations with the criterion. High correlations between predictors is referred to as multicollinearity.

Answer 21

Multiple Regression/Multicollinearity

Question 22

is a symmetrical bell-shaped distribution that is defined by a specific mathematical formula. When scores on a variable are normally distributed, it is possible to conclude that a specific number of observations fall within certain areas of the normal curve that are defined by the standard deviation: In a normal distribution, about 68% of observations fall between the scores that are plus and minus one standard deviation from the mean, about 95% between the scores that are plus and minus two standard deviations from the mean, and about 99% between the scores that are plus and minus three standard deviations from the mean.

Answer 22

Normal Curve/Areas Under The Normal Curve

Question 23

In experimental research, an investigator tests a verbal research hypothesis by simultaneously testing two competing statistical hypotheses. The first of these, the null hypothesis, is stated in a way that implies that the independent variable does not have an effect on the dependent variable. The second statistical hypothesis, the alternative hypothesis, states the opposite of the null hypothesis and is expressed in a way that implies that the independent variable does have an effect.

Answer 23

Null And Alternative Hypotheses

Question 24

is a parametric statistical test used to compare the means of two or more groups when a study includes one IV and one DV that is measured on an interval or ratio scale. The one-way ANOVA yields an F-ratio that indicates if any group means are significantly different. The F-ratio represents a measure of treatment effects plus error divided by a measure of error only (MSB/MSW). When the treatment has had an effect, the F-ratio is larger than 1.0.

Answer 24

One-Way ANOVA

Question 25

are inferential statistical tests that are used when the data to be analyzed represent an interval or ratio scale and when certain assumptions about the population distribution(s) have been met - i.e., when scores on the variable of interest are normally distributed and when there is homoscedasticity (population variances are equal). An advantage of the parametric tests is that they are more “powerful” than the nonparametric tests. They include the Student’s t-test and the analysis of variance. Nonparametric tests are inferential statistical tests used to analyze nominal or ordinal data (or interval or ratio data when the assumptions for a parametric test have not been met). They include the chi-square test, the Mann-Whitney U test, and the Wilcoxon matched-pairs test.

Answer 25

Parametric and Non-Parametric Tests

Question 26

is a structural equation (causal) modeling technique that is used to verify a pre-defined causal model or theory. It involves translating the theory into a path diagram, collecting data on the variables of interest (the observed variables), and calculating and interpreting path coefficients.

Answer 26

Path Analysis

Question 27

each element in the target population has a known chance of being selected for inclusion in the sample. Methods of probability sampling include simple random sampling, stratified random sampling, and cluster sampling. In contrast to simple random sampling and stratified random sampling (which involve selecting individuals from the population), cluster sampling involves selecting units or groups of individuals from the population (e.g., schools, hospitals, clinics.)

Answer 27

Probability Sampling

Question 28

involves randomly assigning subjects to treatment groups and is sometimes referred to as “randomization.” It is considered the “hallmark” of true experimental research because it enables an investigator to conclude that any observed effect of an IV on the DV is due to the IV rather than to error. (Random assignment must not be confused with random selection, which refers to randomly selecting subjects from the population.)

Answer 28

Random Assignment

Question 29

is error that is unpredictable (random). Sampling error and measurement error are types of random error.

Answer 29

Random Error

Question 30

is the appropriate statistical test when blocking has been used as a method for controlling an extraneous variable (i.e., when the extraneous variable is treated as an independent variable). It allows an investigator to statistically analyze the main and interaction effects of the extraneous variable.

Answer 30

Randomized Block ANOVA

Question 31

is used to predict a score on one criterion based on the person’s obtained score on one predictor. It involves identifying the location of the regression line (“line of best fit”) and using the equation for that line, the regression equation, to make predictions. The least squares criterion is used to locate the regression line so that the amount of error in prediction is minimized.

Answer 31

Regression Analysis/Least Squares Criterion

Question 32

of a sampling distribution contains the sample values (e.g., means) that are unlikely to be obtained simply as the result of sampling error. When an inferential statistical test indicates that the obtained sample value falls in the rejection region, the null hypothesis is rejected and the alternative hypothesis is retained. The size of the rejection region is defined by alpha. The retention region is the region of a sampling distribution that contains the values that are likely to be obtained simply as the result of sampling error. When an inferential statistical test indicates that an obtained sample value is in the retention region, the null hypothesis is retained and the alternative hypothesis is rejected. The retention region is equal to one minus alpha.

Answer 32

Rejection and Retention Regions

Question 33

is the distribution of sample means that would be obtained if an infinite number of equal-size samples were randomly selected from the population and the mean for each sample was calculated. The sampling distribution is normally-shaped, its mean is equal to the population mean, and its standard deviation (the standard error of the mean) is equal to the population standard deviation divided by the square root of the sample size. The sampling distribution is used in inferential statistics to determine how likely it is to obtain a particular sample mean given the population mean, the population standard deviation, the sample size, and the level of significance.

Answer 33

Sampling Distribution of the Mean/Standard Error of the mean

Question 34

The four scales of measurement are one way to categorize the various ways of measuring variables. From least to most “mathematically sophisticated,” the scales are nominal, ordinal, interval, and ratio. A nominal scale yields “frequency data” (the frequency of observations in each nominal category). Ordinal, interval, and ratio scales provide scale values or scores.

Answer 34

Scales Of Measurement

Question 35

A correlation coefficient for two or more variables can be squared to obtain a measure of shared variability. For example, if the correlation between X and Y is .50, this means that 25% of variability in Y is shared with (or is accounted for by) variability in X.

Answer 35

Shared Variability

Question 36

include at least one A (baseline) and one B (treatment) phase and include multiple measurements of the DV at regular intervals during each phase. The AB design includes a single baseline phase and a single treatment phase. The reversal designs include, at a minimum, two baseline phases and one treatment phase (e.g., an ABA or ABAB design), with the treatment being withdrawn (“reversed”) during the second and subsequent baseline phases. Use of the multiple-baseline design involves sequentially applying a treatment to different “baselines” (e.g., to different behaviors, settings, tasks, or subjects).

Answer 36

Single Subject Designs

Question 37

are asymmetrical distributions in which the majority of scores are located on one side of the distribution. In a positively skewed distribution, most scores are in the low side of the distribution but a few scores are in the high (positive) side and the mean is greater than the median which, in turn, is greater than the mode. In a negatively skewed distribution, the majority of scores are in the high side of the distribution, but a few are in the low (negative) side and the mode is greater than the median, which is greater than the mean.

Answer 37

Skewed Distributions

Question 38

is a measure of dispersion (variability) of scores around the mean of the distribution. It is the square root of the variance and is calculated by dividing the sum of the squared deviation scores by N (or N - 1) and taking the square root of the result.

Answer 38

Standard Deviation

Question 39

refers to the probability of rejecting a false null hypothesis. Power cannot be directly controlled but is increased by having a large sample, maximizing the effects of the IV, increasing the size of alpha, and reducing error.

Answer 39

Statistical Power

Question 40

is predictable error. Extraneous (confounding) variables are a source of systematic error that affects the relationship between independent and dependent variables.

Answer 40

Systematic Error/Extraneous Variables

Question 41

is a type of analysis of variance that is used to assess linear and nonlinear trends when the independent variable is quantitative.

Answer 41

Trend Analysis

Question 42

occurs when a true null hypothesis is rejected. The probability of making a Type I error is equal to alpha, which is set by the investigator prior to collecting or analyzing the data.

Answer 42

Type I

Question 43

occurs when a false null hypothesis is retained. The probability of making a Type II error is equal to beta (which is usually unknown).

Answer 43

Type II Errors

Question 44

are experimental research designs in which each subject receives, at different times, each level of the IV (or combinations of the IVs) so that comparisons on the DV are made within subjects rather than between groups. The single-group time-series design is a type of within-subjects design.

Answer 44

Within-Subjects Designs