Statistics and Research Design

Question 1

Alpha

Answer 1

Alpha determines the probability of rejecting the null hypothesis when it is true; i.e., the probability of making a Type I error. The value of alpha is set by the experimenter prior to collecting or analyzing the data. In psychological research, alpha is commonly set at .01 or .05.

Question 2

Chi-Square Test (Single-Sample And Multiple-Sample)

Answer 2

The chi-square test is a nonparametric statistical test that is used with nominal data (or data that are being treated as nominal data) - i.e., when the data to be compared are frequencies in each category. The single-sample chi-square test is used when the study includes one variable; the multiple-sample chi-square test when it includes two or more variables. (When counting variables for the chi-square test, independent and dependent variables are both included.)

Question 3

Cluster Analysis

Answer 3

Cluster analysis is a multivariate technique that is used to group people or objects into a smaller number of mutually exclusive and exhaustive subgroups (clusters) based on their similarities - i.e., to group people or objects so that the identified subgroups have within-group homogeneity and between-group heterogeneity.

Question 4

Cross-Validation/Shrinkage

Answer 4

Cross-validation refers to validating a correlation coefficient (e.g., a criterion-related validity coefficient) on a new sample. Because the same chance factors operating in the original sample are not operating in the subsequent sample, the correlation coefficient tends to “shrink” on cross-validation. In terms of the multiple correlation coefficient (R), shrinkage is greatest when the original sample is small and the number of predictors is large.

Question 5

Discriminant Function Analysis

Answer 5

Discriminant function analysis is the appropriate multivariate technique when two or more continuous predictors will be used to predict or estimate a person’s status on a single discrete (nominal) criterion.

Question 6

Effect Size

Answer 6

An effect size is measure of the magnitude of the relationship between independent and dependent variables and is useful for interpreting the relationship’s clinical or practical significance (e.g., for comparing the clinical effectiveness of two or more treatments). Several methods are used to calculate an effect size including Cohen’s d (which indicates the difference between two groups in terms of standard deviation units) and eta squared (which indicates the percent of variance in the dependent variable that is accounted for by variance in the independent variable).

Question 7

Experimental Research (True and Quasi-Experimental)

Answer 7

research involves conducting an empirical study to test hypotheses about the relationships between independent and dependent variables. A true experimental study permits greater control over experimental conditions, and its “hallmark” is random assignment to groups. A quasi-experimental study permits less control.

Question 8

Experimentwise Error Rate

Answer 8

The experimentwise error rate (also known as the familywise error rate) is the probability of making a Type I error. As the number of statistical comparisons in a study increases, the experimentwise error rate increases.

Question 9

External Validity (Pretest Sensitization, Reactivity, Multiple Treatment Interference)

Answer 9

External validity refers to the degree to which a study’s results can be generalized to other people, settings, conditions, etc. Threats include pretest sensitization (which occurs when pretesting affects how subjects react to the treatment), reactivity (which occurs when subjects respond differently to a treatment because they know they are participating in a research study), and multiple treatment interference (which occurs when subjects receive more than one level of an IV). Counterbalancing can be used to control multiple treatment interference and involves administering different levels of the IV to different groups of subjects in a different order.

Question 10

Factorial ANOVA

Answer 10

The factorial ANOVA is the appropriate statistical test when a study includes two or more IVs (i.e., when the study has used a factorial design) and a single DV that is measured on an interval or ratio scale. It is also referred to as a two-way ANOVA, three-way ANOVA, etc., with the words “two” and “three” referring to the number of IVs.

Question 11

Factorial Design (Main And Interaction Effects)

Answer 11

Factorial designs are research designs that include two or more “factors” (independent variables). They permit the analysis of main and interaction effects: A main effect is the effect of a single IV on the DV, while an interaction refers to the effects of one IV at different levels of another IV.

Question 12

Independent and Dependent Variables

Answer 12

The independent variable (IV) is the variable that is believed to have an effect on the dependent variable and is varied or manipulated by the researcher in an experimental research study. Each independent variable in a study must have at least two levels. The dependent variable (DV) is the variable that is believed to be affected by the independent variable and is observed and measured.

Question 13

Internal Validity (Maturation, History, Statistical Regression, Selection)

Answer 13

Internal validity refers to the degree to which a research study allows an investigator to conclude that observed variability in a dependent variable is due to the independent variable rather than to other factors. Maturation is one threat to internal validity and occurs when a physical or psychological process or event occurs as the result of the passage of time (e.g., increasing fatigue, decreasing motivation) and has a systematic effect on subjects’ status on the DV. History is a threat when an event that is external to the research study affects subjects’ performance on the DV in a systematic way. Statistical regression is a threat when subjects are selected to participate because of their extreme status on the DV or a measure that correlates with the DV and refers to the tendency of extreme scores to “regress to the mean” on retesting. Selection threatens internal validity when groups differ at the beginning of the study because of the way subjects were assigned to groups and is a potential threat whenever subjects are not randomly assigned to groups.

Question 14

Interval Recording/Event Sampling

Answer 14

Interval recording is a method of behavioral sampling that involves dividing a period of time into discrete intervals and recording whether the behavior occurs in each interval. It is particularly useful for behaviors that have no clear beginning or end. Event sampling is a method of behavioral sampling that is useful for behaviors that are rare or that leave a permanent product. It involves recording each occurrence of a behavior during a predefined or preselected event.

Question 15

LISREL

Answer 15

LISREL is a structural equation (causal) modeling technique that is used to verify a predefined causal model or theory. It is more complex than path analysis, and it allows two-way (non-recursive) paths and takes into account observed variables, the latent traits they are believed to measure, and the effects of measurement error.

Question 16

MANOVA (Multivariate Analysis of Variance)

Answer 16

The MANOVA is a form of the ANOVA that is used when a study includes one or more IVs and two or more DVs that are each measured on an interval or ratio scale. Use of the MANOVA helps reduce the experimentwise error rate and increases power by simultaneously analyzing the effects of the IV(s) on all of the DVs.

Question 17

Measures of Central Tendency (Mean, Median, Mode)

Answer 17

The mean, median, and mode are the most commonly used measures of central tendency. The mean is the arithmetic average of a set of scores, and it can be used when scores represent an interval or ratio scale. The median is the middle score in a distribution when scores have been ordered from lowest to highest. It is used with ordinal data (and with interval and ratio data when the distribution is skewed or contains one or a few outliers). Finally, the mode is the most frequently occurring score or category, and it is used as a measure of central tendency for nominal variables or variables that are being treated as nominal variables.

Question 18

Mixed (Split Plot) ANOVA

Answer 18

The mixed ANOVA is a type of factorial ANOVA that is used when a study includes at least one between-groups independent variable and one within-subjects independent variable.

Question 19

Mixed Designs

Answer 19

Mixed designs are a type of factorial design in which at least one IV is a between-groups variable and one IV is a within-subjects variable.

Question 20

Moderator and Mediator Variables

Answer 20

Moderator variables affect the strength or direction of the relationship between independent and dependent variables. If a treatment is more effective for reducing cigarette smoking for men than for women, gender is a moderator variable. Mediating variables explain or account for the relationship between independent and dependent variables. As an example, authoritative parenting may have positive effects on academic achievement because authoritative parenting leads to high self-efficacy beliefs (the mediator) which, in turn, leads to a high level of academic achievement.

Question 21

Multiple Regression/Multicollinearity

Answer 21

Multiple regression is a multivariate technique that is used for predicting a score on a continuous criterion based on performance on two or more continuous and/or discrete predictors. The output of multiple regression is a multiple correlation coefficient (R) and a multiple regression equation. Ideally, predictors included in a multiple regression equation will have low correlations with each other and high correlations with the criterion. High correlations between predictors is referred to as multicollinearity.

Question 22

Normal Curve/Areas Under The Normal Curve

Answer 22

A normal curve is a symmetrical bell-shaped distribution that is defined by a specific mathematical formula. When scores on a variable are normally distributed, it is possible to conclude that a specific number of observations fall within certain areas of the normal curve that are defined by the standard deviation: In a normal distribution, about 68% of observations fall between the scores that are plus and minus one standard deviation from the mean, about 95% between the scores that are plus and minus two standard deviations from the mean, and about 99% between the scores that are plus and minus three standard deviations from the mean.

Question 23

Null And Alternative Hypotheses

Answer 23

In experimental research, an investigator tests a verbal research hypothesis by simultaneously testing two competing statistical hypotheses. The first of these, the null hypothesis, is stated in a way that implies that the independent variable does not have an effect on the dependent variable. The second statistical hypothesis, the alternative hypothesis, states the opposite of the null hypothesis and is expressed in a way that implies that the independent variable does have an effect.

Question 24

One-Way ANOVA

Answer 24

The one-way ANOVA is a parametric statistical test used to compare the means of two or more groups when a study includes one IV and one DV that is measured on an interval or ratio scale. The one-way ANOVA yields an F-ratio that indicates if any group means are significantly different. The F-ratio represents a measure of treatment effects plus error divided by a measure of error only (MSB/MSW). When the treatment has had an effect, the F-ratio is larger than 1.0.

Question 25

Parametric and Non-Parametric Tests

Answer 25

Parametric tests are inferential statistical tests that are used when the data to be analyzed represent an interval or ratio scale and when certain assumptions about the population distribution(s) have been met - i.e., when scores on the variable of interest are normally distributed and when there is homoscedasticity (population variances are equal). An advantage of the parametric tests is that they are more “powerful” than the nonparametric tests. They include the Student’s t-test and the analysis of variance. Nonparametric tests are inferential statistical tests used to analyze nominal or ordinal data (or interval or ratio data when the assumptions for a parametric test have not been met). They include the chi-square test, the Mann-Whitney U test, and the Wilcoxon matched-pairs test.

Question 26

Path Analysis

Answer 26

Path analysis is a structural equation (causal) modeling technique that is used to verify a pre-defined causal model or theory. It involves translating the theory into a path diagram, collecting data on the variables of interest (the observed variables), and calculating and interpreting path coefficients.

Question 27

Probability Sampling

Answer 27

When using probability sampling, each element in the target population has a known chance of being selected for inclusion in the sample. Methods of probability sampling include simple random sampling, stratified random sampling, and cluster sampling. In contrast to simple random sampling and stratified random sampling (which involve selecting individuals from the population), cluster sampling involves selecting units or groups of individuals from the population (e.g., schools, hospitals, clinics.)

Question 28

Random Assignment

Answer 28

Random assignment involves randomly assigning subjects to treatment groups and is sometimes referred to as “randomization.” It is considered the “hallmark” of true experimental research because it enables an investigator to conclude that any observed effect of an IV on the DV is due to the IV rather than to error. (Random assignment must not be confused with random selection, which refers to randomly selecting subjects from the population.)

Question 29

Random Error

Answer 29

Random error is error that is unpredictable (random). Sampling error and measurement error are types of random error.

Question 30

Randomized Block ANOVA

Answer 30

The randomized block ANOVA is the appropriate statistical test when blocking has been used as a method for controlling an extraneous variable (i.e., when the extraneous variable is treated as an independent variable). It allows an investigator to statistically analyze the main and interaction effects of the extraneous variable.

Question 31

Regression Analysis/Least Squares Criterion

Answer 31

Regression analysis is used to predict a score on one criterion based on the person’s obtained score on one predictor. It involves identifying the location of the regression line (“line of best fit”) and using the equation for that line, the regression equation, to make predictions. The least squares criterion is used to locate the regression line so that the amount of error in prediction is minimized.

Question 32

Rejection and Retention Regions

Answer 32

The rejection region of a sampling distribution contains the sample values (e.g., means) that are unlikely to be obtained simply as the result of sampling error. When an inferential statistical test indicates that the obtained sample value falls in the rejection region, the null hypothesis is rejected and the alternative hypothesis is retained. The size of the rejection region is defined by alpha. The retention region is the region of a sampling distribution that contains the values that are likely to be obtained simply as the result of sampling error. When an inferential statistical test indicates that an obtained sample value is in the retention region, the null hypothesis is retained and the alternative hypothesis is rejected. The retention region is equal to one minus alpha.

Question 33

Sampling Distribution of the Mean/Standard Error of the mean

Answer 33

The sampling distribution of the mean is the distribution of sample means that would be obtained if an infinite number of equal-size samples were randomly selected from the population and the mean for each sample was calculated. The sampling distribution is normally-shaped, its mean is equal to the population mean, and its standard deviation (the standard error of the mean) is equal to the population standard deviation divided by the square root of the sample size. The sampling distribution is used in inferential statistics to determine how likely it is to obtain a particular sample mean given the population mean, the population standard deviation, the sample size, and the level of significance.

Question 34

Scales Of Measurement

Answer 34

The four scales of measurement are one way to categorize the various ways of measuring variables. From least to most “mathematically sophisticated,” the scales are nominal, ordinal, interval, and ratio. A nominal scale yields “frequency data” (the frequency of observations in each nominal category). Ordinal, interval, and ratio scales provide scale values or scores.

Question 35

Shared Variability

Answer 35

A correlation coefficient for two or more variables can be squared to obtain a measure of shared variability. For example, if the correlation between X and Y is .50, this means that 25% of variability in Y is shared with (or is accounted for by) variability in X.

Question 36

Single Subject Designs

Answer 36

Single-subject designs include at least one A (baseline) and one B (treatment) phase and include multiple measurements of the DV at regular intervals during each phase. The AB design includes a single baseline phase and a single treatment phase. The reversal designs include, at a minimum, two baseline phases and one treatment phase (e.g., an ABA or ABAB design), with the treatment being withdrawn (“reversed”) during the second and subsequent baseline phases. Use of the multiple-baseline design involves sequentially applying a treatment to different “baselines” (e.g., to different behaviors, settings, tasks, or subjects).

Question 37

Skewed Distributions

Answer 37

Skewed distributions are asymmetrical distributions in which the majority of scores are located on one side of the distribution. In a positively skewed distribution, most scores are in the low side of the distribution but a few scores are in the high (positive) side and the mean is greater than the median which, in turn, is greater than the mode. In a negatively skewed distribution, the majority of scores are in the high side of the distribution, but a few are in the low (negative) side and the mode is greater than the median, which is greater than the mean.

Question 38

Standard Deviation

Answer 38

The standard deviation is a measure of dispersion (variability) of scores around the mean of the distribution. It is the square root of the variance and is calculated by dividing the sum of the squared deviation scores by N (or N - 1) and taking the square root of the result.

Question 39

Statistical Power

Answer 39

Statistical power refers to the probability of rejecting a false null hypothesis. Power cannot be directly controlled but is increased by having a large sample, maximizing the effects of the IV, increasing the size of alpha, and reducing error.

Question 40

Systematic Error/Extraneous Variables

Answer 40

Systematic error is predictable error. Extraneous (confounding) variables are a source of systematic error that affects the relationship between independent and dependent variables.

Question 41

Trend Analysis

Answer 41

Trend analysis is a type of analysis of variance that is used to assess linear and nonlinear trends when the independent variable is quantitative.

Question 42

Type I And Type II Errors

Answer 42

A Type I error occurs when a true null hypothesis is rejected. The probability of making a Type I error is equal to alpha, which is set by the investigator prior to collecting or analyzing the data. A Type II error occurs when a false null hypothesis is retained. The probability of making a Type II error is equal to beta (which is usually unknown).

Question 43

Within-Subjects Designs

Answer 43

Within-subjects designs are experimental research designs in which each subject receives, at different times, each level of the IV (or combinations of the IVs) so that comparisons on the DV are made within subjects rather than between groups. The single-group time-series design is a type of within-subjects design.