Statistics and Research Design

Question 1

Alpha (Level of Significance)

Answer 1

Alpha determines the probability of rejecting the null hypothesis when it is true: (i.e. the probability of making a Type I error). The value of alpha is set by the experimenter prior to collecting or analyzing the data. In psychological research, alpha is commonly set at .01 or .05

Question 2

ANCOVA (Analysis of covariance)

Answer 2

The ANCOVA is a version of the ANOVA that is used to increase the efficiency of the analysis by statistically removing variability in the DV that is due to an extraneous variable. When using the ANCOVA, each person’s score on the DV is adjusted on the basis of his or her score on the extraneous variable

Question 3

Between-groups design

Answer 3

Between-groups designs are experimental research designs that allow a researcher to assess the effects of the different levels of one or more IVs by administering each level or combination of levels to a different group of subjects

Question 4

Central Limit Theorem

Answer 4

The Central Limit Theorem is derived from probability theory and predicts that the sampling distribution of the mean

(a) will approach a normal shape as the sample size increases, regardless of the shape of the population distribution of scores
b) has a mean equal to the population mean; and
c) has a standard deviation equal to the population standard deviation divided by the square root of the sample size. This standard deviation is referred to as the standard error of the mean

Question 5

Chi-Square Test (Single Sample & Multiple Sample)

Answer 5

The Chi-square test is a nonparametric statistical test that is used with normal data (or data that are being treated as nominal data) - i.e. when the data to be compared are frequencies in each category. The single-sample of chi-square test is used when the study includes one variable; the multiple-sample chi-square test when it includes two or more variables. (When counting variables for the chi-square test, independent and dependent variables are both included)

Question 6

Cluster Analysis

Answer 6

Cluster analysis is a multivariate technique that is used to group people or objects into a smaller number of mutually exclusive and exhaustive subgroups (clusters) based on their similarities (i.e. to group people or objects so that the identified subgroups have within-group homogeneity and between-group heterogeneity.

Question 7

Correlation Coefficient (Pearson r, Spearman Rho, Point Biserial, biserial, ETA)

Answer 7

The correlation coefficient is a numerical index of the relationship (Degree of association) between variables. The magnitude of the coefficient indicates the strength of the relationship; its sign indicates the direction (positive or negative). The Pearson r is used when data on both variables represent a continuous scale. Spearman rho is used when both variables are ranks, the point biserial coefficient is used when one variable is a trie dichotomy and the other is continuous, the biserial coefficient is used when one variable is an artificial dichotomy and the other is continuous, and eta is used when the variables are both continuous and have a nonlinear relationship

Question 8

Cross-Validation/Shrinkage

Answer 8

Cross-validation refers to validating a correlation coefficient (e.g. a criterion-related validity coefficient) on a new sample. Because the same chance factors operating in the original sample are not operating in the subsequent sample, the correlation coefficient tends to “shrink” on cross validation. In terms of the multiple correlation coefficient (R), shrinkage is greatest when the original sample is small and the number of predictors is large

Question 9

Discriminant Function Analysis

Answer 9

Discriminant function analysis is the appropriate multivariate technique when two or more continuous predictors will be used to predict or estimate a person’s status on a single discrete (nominal) criterion

Question 10

Effect Size

Answer 10

An effect size is measure of the magnitude of the relationship between independent and dependent variables and is useful for interpreting the relationship’s clinical or practical significance (e.g. for comparing the clinical effectiveness of two or more treatments). Several methods are used to calculate an effect size including Cohen’s d (which indicates the difference between two groups in terms of standard deviation units) and eta squared (which indicates the percent of variance in the dependent variable that is accounted for by variance in the independent variable)

Question 11

Experimental Research (True and quasi-experimental)

Answer 11

Experimental research involves conducting an empirical study to test hypotheses about the relationships between independent and dependent variables. A true experimental study permits greater control over experimental conditions, and its “hallmark” is random assignment to groups. A quasi-experimental study permits less control.

Question 12

Experimentwise Error Rate

Answer 12

The experimentwise error rate (also known as the familywise error rate) is the probability of making a Type 1 error. As the number of statistical comparisons in a study increases, the experimentwise error rate increases

Question 13

External Validity (Pretest Sensitization, Reactivity, Multiple Treatment Interference)

Answer 13

External validity refers to the degree to which a study’s results can be generalized to other people, settings, conditions, etc. Threats include pretest sensitization (Which occurs when pretesting affects how subjects react to the treatment), reactivity (which occurs when subjects respond differently to a treatment because they know they are participating in a research study), and multiple treatment interference (which occurs when subjects receive more than one level of an IV). Counterbalancing can be used to control multiple treatment interference and involves administering different levels of the IV to different groups of subjects in a different order.

Question 14

Factorial ANOVA

Answer 14

The factorial ANOVA is the appropriate statistical test when a study includes two or more IVs (i.e. when the study has used a factorial design) and a single DV that is measured on an interval or ratio scale. It is also referred to as a two-way ANOVA, three-way ANOVA, with the words two and three referring to the number of IVs.

Question 15

Factorial Design (Main and Interaction Effects)

Answer 15

Factorial designs are research designs that include two or more “factors” (independent variables). They permit the analysis of main and interaction effects:

A main effect is the effect of a single IV on the DV
An interaction- the effects of one IV at different levels of another IV

Question 16

Internal Validity

Answer 16

Refers to the degree to which a research study allows an investigator to conclude that observed variability in a dependent variable is due to the independent variable rather than to other factors.

Maturation is one threat to internal validity and occurs when a physical or psychological process or event occurs as the result of the passage of time (e.g. increasing fatigue, decreasing motivation) and has a systematic effect on the subjects status on the DV

History is a threat when an event that is external to the research study affects subjects performance on the DV in a systematic way.

Statistical regression is a threat when subjects are selected to participate because of their extreme status on the DV or a measure that correlates with the DV and refers to the tendency of extreme scores to “Regress to the mean” on retesting

Selection threatens internal validity when groups differ at the beginning of the study because of the way subjects were assigned to groups and is a potential threat whenever subjects are not randomly assigned to groups

Question 17

Interval Recording/Event Sampling

Answer 17

Interval recording is a method of behavioral sampling that involves dividing a period of time into discrete intervals and recording whether the behavior occurs in each interval. It is particularly useful for behaviors that have no clear beginning or end.

Event sampling is a method of behavioral sampling that is useful for behaviors that are rare and that leave a permanent product. It involves recording each occurrence of a behavior during a predefined or preselected event.

Question 18

LISREL

Answer 18

LISREL is a structural equation (causal) modeling technique that is used to verify a predefined causal model or theory. It is more complex than path analysis, and it allows two-way (non-recursive) paths and takes into account observed variables, the latent traits they are believed to measure, and the effects of measurement error

Question 19

MANOVA (Multivariate Analysis of Variance)

Answer 19

The MANOVA is a form of the ANOVA that is used when a study includes one or more IVs and two or more DVs that are each measured on an interval or ratio scale. Use of the MANOVA helps reduce the experimentwise error rate and increases power by simultaneously analyzing the effects of the IV(s) on all of the DVs

Question 20

Mixed (Split-plot) ANOVA

Answer 20

The mixed ANOVA is a type of factorial ANOVA that is used when a study includes at least one between-groups independent variable and one within-subjects independent variable

Question 21

Mixed designs

Answer 21

A type of factorial design in which at least one IV i a between-groups variable and one IV is a within-subjects variable

Question 22

Moderator and Mediator Variables

Answer 22

Moderator variables affect the strength or direction of the relationship between independent and dependent variables. If a treatment is more effective for reducing cigarette smoking for men than for women, gender is a moderating variable.

Mediating variables explain or account for the relationship between independent and dependent variables. As an example, authoritative parenting may have positive effects on academic achievement because authoritative parenting leads to high self-efficacy beliefs (the mediator) which, in turn, leads to a high level of academic achievement

Question 23

Multiple Regression/Multicollinearity

Answer 23

Multiple regression is a multivariate technique that is used for predicting a score on a continuous criterion based on performance on two or more continuous and/or discrete predictors. The output of multiple regression is a multiple correlation coefficient (R) and a multiple regression equation. Ideally, predictors included in a multiple regression equation will have low correlations with each other and high correlations with the criterion. High correlations between predictors is referred to as multicollinearity.

Question 24

Normal Curve/Areas under the normal curve

Answer 24

A normal curve is a symmetrical bell-shaped distribution that is defined by a specific mathematical formula. When scores on a variable are normally distributed, it is possible to conclude that a specific number of observations fall within certain areas of a normal curve that are defined by the standard deviation: In a normal distribution, about 68% of observations fall between the scores that are plus and minus two standard deviations from the mean, and about 99% between the scores that are plus and minus three standard deviations from the mean

Question 25

Null and alternative hypotheses

Answer 25

In experimental research, an investigator tests a verbal research hypothesis by simultaneously testing two competing statistical hypotheses. The first of these, the null hypothesis, is stated in a way that implies that the independent variable does not have an effort on the dependent variable. The second statistical hypothesis, the alternative hypothesis, states the opposite of the null hypothesis and is expressed in a way that implies that the indpendent variable does not have an effect.

Question 26

One-way ANOVA (Analysis of Variance)/F-Ratio

Answer 26

The one-way ANOVA is is a parametric statistical test used to compare the means of two or more groups when a study includes one IV and one DV and is measured on an interval or ratio scale. The one-way ANOVA yields an F-ratio that indicates if any group means are significantly different. The F-ratio represents a measure of treatment effects plus error divided by a measure of error only (MSB/MSW). When the treatment has had an effect, the F-ratio is larger than 1.0

Question 27

Parametric and Nonparametric Tests

Answer 27

Parametric tests are inferential statistical tests that are used when the data to be analyzed represent an interval or ratio scale and when certain assumptions about the population distribution(s) have been met- i.e. when scores on the variable of interest are normally distributed and when there is homoscedasticity (population variances are equal). An advantage of the parametric tests is that they are more “powerful” than the nonparametric tests. They include the Student’s t-test and the analysis of variance.

Nonparametric tests are inferential statistical tests used to analyze nominal or ordinal data (or interval or ratio data when the assumptions for a parametric test have not been met

Question 28

Path Analysis

Answer 28

Path analysis is a structural (causal) modeling technique that is used to verify a pre-defined causal model or theory. It involves translating the theory into a path diagram, collecting data on the variables of interest (the observed variables), and calculating and interpreting path coefficients

Question 29

Probability Sampling

Answer 29

When using probability sampling, each element in the target population has a known chance of being selected for inclusion in the sample. Methods of probability sampling include sample random sampling, stratified random sampling, and cluster sampling. In contrast to simple random sampling and stratified random sampling (which involves selecting individuals from the population), cluster sampling involves selecting units or groups of individuals from the population (e.g. schools, hospitals, clinics).

Question 30

Random Assignment

Answer 30

Involves randomly assigning subjects to treatment groups and is sometimes referred to as “randomization.” It is considered the “hallmark” of true experimental research because it enables an investigator ro conclude that any observed effect of an IV on the DV is due to the IV rather than to error. (Random assignment must not be confused with random selection, which refers to randomly selecting subjects from the population).

Question 31

Random Error

Answer 31

Random error is error that is unpredictable (random). Sampling error and measurement error are types of random error.

Question 32

Randomized Block ANOVA

Answer 32

The randomized block ANOVA is the appropriate statistical test when blocking has been used as a method for controlling an extraneous variable (i.e. when the extraneous variable is treated as an independent variable). It allows an investigator to statistically analyze the main and interaction effects of the extraneous variable.

Question 33

Regression Analysis/Least Squares Criterion

Answer 33

Regression analysis is used to predict a score on one criterion based on the person’s obtained score on one predictor. It involves identifying the location of the regression line (line of best fit) and using the equation for that line, the regression equation, to make predictions. The least squares criterion is used to locate the regression line so that the amount of error in prediction is minimized.

Question 34

Rejection and Retention Regions

Answer 34

The rejection region of a sampling distribution contains the sample values (e.g. means) that are unlikely to be obtained simply as the result of sampling error. When an inferential statistical test indicates that the obtained sample value falls in the rejection region, the null hypothesis is rejected and the alternative hypothesis is retained. The size of the rejection region is defined by alpha. The retention region is the region of a sampling distribution that contains the values that are likely to be obtained simply as the result of sampling error. When an inferential statistical test indicates that an obtained sample value is in the retention region, the null hypothesis is retained and the alternative hypothesis is rejected. The retention region is equal to one minus alpha.

Question 35

Sampling Distribution of the Mean/Standard Error of the Mean

Answer 35

The sampling distribution of the mean is the distribution of sample means that would be obtained if an infinite number of equal-size samples were randomly selected from the population and the mean for each sample was calculated. The sampling distribution is normally-shaped, its mean is equal to the population mean, and its standard deviation (the standard error of the mean) is equal to the population standard deviation divided by the square root of the sample size. The sampling distribution is used in inferential statistics to determine how likely it is to obtain a particular sample mean given the population mean, the population standard deviation, the sample size, and the level of significance

Question 36

Scales of Measurement

Answer 36

The four scales of measurement are one way to categorize the various ways of measuring variables. From least to most “mathematically sophisticated,” the scales are nominal, ordinal, interval, and ratio. A nominal scale yields “frequency data” (the frequency of observations in each nominal category). Ordinal, interval, and ratio scales provide scale values or scores.

Question 37

Shared Variability

Answer 37

A correlation coefficient for two or more variables can be squared to obtain a measure of shared variability. For example, if the correlation between X and Y is .50, this means that 25% of variability in Y is shared with (or is accounted for by) variability in X.

Question 38

Single Subject Designs (AB, Reversal, Multiple Baseline)

Answer 38

Single-subject designs include at least one A (baseline) and one B (treatment) phase and include multiple measurements of the DV at regular intervals during each phase. The AB design includes a single baseline phase and a single treatment phase. The reversal designs include, at a minimum, two baseline phases and one treatment phase )e.g. an ABA or ABAB design), with the treatment being withdrawn (“reversed”) during the second and subsequent baseline phases. Use of the multiple-baseline design involves sequentially applying a treatment to different “baselines” (e.g. to different behaviors, settings, tasks, or subjects)

Question 39

Skewed Distributions

Answer 39

Skewed distributions are asymmetrical distributions in which the majority of scores are located on one side of the distribution. In a positively skewed distribution, most scores are in the low side of the distribution but a few scores are in the high (positive) side and the mean is greater than the median which, in turn, is greater than the mode. In a negatively skewed distribution, the majority of scores are in the high side of the distribution, but a few are in the low (negative) side and the mode is greater than the median, which is greater than the mean

Question 40

Standard Deviation

Answer 40

The standard deviation is a measure of dispersion (variability) of scores around the mean of the distribution. It is the square root of the variance and is calculated by dividing the sum of the squared deviation scores by N (or N-1) and taking the square root of the result

Question 41

Statistical Power

Answer 41

Statistical power refers to the probability of rejecting a false null hypothesis. Power cannot be directly controlled but is increased by having a large sample, maximizing the effects of the IV, increasing the size of alpha, and reducing error

Question 42

Student’s t-Test (Single sample, independent samples, correlated samples)

Answer 42

The Student’s t-test is a parametric test that is used to compare two means. The single-sample t-test is used to compare a single obtained sample mean to a known or hypothesized population mean. A t-test for independent samples is used to compare means from two independent groups. The t-test for correlated samples is used to compare two sample means when subjects in the two groups are related in some way (e.g. because they were matched on an extraneous variable or because a single-group pretest/posttest design was used

Question 43

Systematic Error/Extraneous Variables

Answer 43

Systematic error is predictable error. Extraneous (confounding) variables are a source of systematic error that affects the relationship between independent and dependent variables.

Question 44

Trend Analysis

Answer 44

Trend analysis is a type of analysis of variance that is used to assess linear and nonlinear trends when the independent variable is quantitative

Question 45

Type I and Type II Errors

Answer 45

A Type 1 error occurs when a true null hypothesis is rejected. The probability of making a Type 1 error is equal to alpha, which is set by the investigator prior to collecting or analyzing the data. A type II error occurs when a false null hypothesis is retained. The probability of making a Type II error is equal to beta (which is usually unknown)

Question 46

Within-Subjects Design

Answer 46

Within-subjects designs are experimental research designs in which each subject receives, at different times, each level of the IV (or combinations of the IVs) so that comparisons on the DV are made within subjects rather than between groups. The single-group time-series design is a type of within-subjects design.