Quiz 4 Flashcards
Three General Assumptions of Parametric Statistical Tests
- Normality of sampling distributions/population residuals
- When you compare more than 1 population, the variances of the populations are equal
- Data from your population are independent
Parametric Tests
- statistical tests used to estimate a specific parameter value (e.g. t-tests)
Normality
Inferential statistics assume that our sample data are drawn from normal sampling distributions
How can we know about normality?
- If we have a large enough sample (typically more than 30), we have met the assumption
- If we have a small sample, we examine our sample to infer normality of the sampling distribution
- If you have multivariate data, examine each variable by itself to see if it is normally distributed (e.g. aX + bY) → a linear combination needs to be normal
- If we have a normally distributed sample data, it is likely that it is from a normally distributed population → sampling distribution would be normal
Skewness
- When a distribution is perfectly normal, the values of skewness and kurtosis are zero
- Positive skewness means that there is a pile up of cases on the left and a long right tail (skewed to the right)
- Negative skewness means that there is a pileup of cases to the right and a long left tail (skewed to the left)
When does the CLT not work/apply?
- When distributions have thick tails
- If your sample is small
The Central Limit Theorem
- The CLT is one of the most remarkable results of the theory of probability
- In its simplest form, theorem states that:
- The mean of a large
number of
independent
observations from
the same distribution
has, under certain
general conditions, an
approximate normal
distribution - Note: exception of distributions with heavy tails
Testing Normality in Single Variables
TB p. 183-191
- Is the sample size big enough to assume that the sampling distribution is normally distributed?
- Look at histogram of each continuous variable → starting at visual inspection of normality
- Perform the Kolmogrorov-Smirnov (K-S) test or the Shapiro-Wilk test
- Significant results would suggest that the data are NOT normally distributed
- Caveat: the power of the test depends on the sample size, and is often a moot point because in large samples without thick tails, we would assume normality anyways
- Shapiro-Wilk test is highly sensitive to even small deviations from normality in large samples
Look to skewness and kurtosis stats
For the formula for skewness
- Average of the z scores raised to the third power
–> Increases the influence of outliers by raising to the third power
- Converting raw scores into z scores
- If skewed to the right, we will get a positive skewness score
- If skewed to the left, we will get a negative value
- No skewness → Formula results in zero
Cutoffs: ± 2
If skewed to the right, we will get a _______ skewness score
positive
If skewed to the left, we will get a ______ value
negative
Formula for Kurtosis K4
- Kurtosis values above zero indicate a distribution that is too peaked with short thicks tails
- Kurtosis value below zero is platykurtic
Leptokurtic (thicker tails)→ positive kurtosis statistic
Platykurtic (thinner tails) → negative kurtosis statistic
Kurtosis value below zero is _________
platykurtic
Leptokurtic (thicker tails)→ ______ kurtosis statistic
positive
Platykurtic (thinner tails) → ________ kurtosis statistic
negative
Rule of thumb cutoffs for kurtosis
±7 be concerned
Significance Tests for Skewness and Kurtosis
Step 1: convert skewness and kurtosis scores into z scores
Step 2: compare z scores to critical values of ±1.96 for small samples and ±2.58 for large samples. If greater than the critical value, significant skewness/kurtosis
- More stringent for larger samples
What is the Big Deal if a Distribution Isn’t “Normal?”
- We could get inaccurate results from our analysis
- Mess with type I and type II error rates
- Meaning that the null could be true when our stats tell us it isn’t or vice versa
What to do if normality assumption is NOT met
- Data transformation
Appropriate when there is skewness in distribution
- Replacing the data with a function of all the data within that variable - Non-parametric tests
- Modern methods (e.g. bootstrapping)
Data Transformation
Most common → square root transformation
Most useful when data are skewed to the right
Pulls more extreme values closer to the middle
Bigger impact on bigger values
Square Root Transformation is most useful when _______
data are skewed to the right
When data are skewed left what transformation can be done?
When data are skewed to the left:
- Reflect scores and the do a square root transformation
- Subtract values from a large number to reflect
Log transformations
For extreme positive skew → reduces positive skew
Pulls in values to a greater degree than square root transformation
Inverse transformation
- Transforms data with extreme positive skew to normal
- 1 / (value of data)
- Need to add a constant to bring all values to non zero, but CAN have a negative as long as there’s no zero
- Table 6.1 in TB
To Transform … or Not
- The CLT: sampling distribution will be normal in samples >30 (unless sample distribution has a substantial skew; heavy tails)
- Transformations sometimes fail to restore normality and equality of variances
- They make the interpretation of results difficult, as findings are based on transformed data
Rank Order Tests (AKA Non-Parametric Tests)
Can be used in place of their parametric counterparts when it is questionable that the normality assumption holds
sometimes more powerful in detecting population differences when certain assumptions are not satisfied
Nonparametric tests cannot assess interactions
Modern Methods
Refers to approaches dealing with non normal data that require a lot of computing power
- E.g. bootstrapping methods
Bootstrapping
Goal is to observe the sampling distribution shape directly to allow us to do hypothesis testing
Uses sample data to estimate the sampling distribution itself
By drawing random samples with replacement from the sample data
Because sampling distribution is estimated directly, no need to assume normality
P-value can be calculated based on how rare it is to get the observed test-statistic value or more extreme values in the estimated sampling distribution (regardless of whether it is normal or not)
Rule of thumb for Bootstrapping
5,000 - 10,000 bootstrap samples
Can use bootstrapping to create a CI
Find central value (mean) of the data points and look at what falls at lower 2.5th percentile and upper 97.5th percentile and use them as upper and lower bounds for CI
The Three Assumptions of Parametric Statistics
- The sampling distribution(s) is(are) normally distributed
- Homogeneity (equality) of Variance
- Data from your population are independent
Homogeneity (equality) of Variance
The assumption that the dependent variable exhibits similar amounts of variance across the range of values for an independent variable
Assessing Homogeneity of Variance
Visual inspection of graphs
- Scatter plot, residual
plot
Levene’s Tests
Can become overly sensitive in large samples
Variance Ratio (Hartley’s
FMAX)
Variance Ratio (Hartley’s Fmax)
With 2 or more groups
VR = Large
variance/smallest
variance
If VR < 2, homogeneity of variance can be assumed
If the group sizes are roughly equal, hypothesis testing results are robust to the violation of homogeneity
Would still likely still get valid results even if
If the largest group size is smaller than 1.5 times the smallest group size → can use this concept
Leven’s Tests
- Tests if variance in
different groups are
the same- Significant = variances not equal - Non significant = variances are equal Null hypothesis is that there’s homogeneity of variance
Visual inspection of graphs for assessing homogeneity of variance
- Scatter plot, residual
plot
*Space between line of best fit and actual point
–> Deviation, error, residual
Addressing Homogeneity of Variance
- Using robust methods
- Bootstrapping
- Transforming an outcome variable
Why is independence of data important?
- general formula for test statistic involves two types of variability, one in numerator and one in denom
- Formula for test-statistic = explained variability/unexplained variability
- We want test statistic to be bigger such that we have greater explanatory power
BUT with dependent data, unexplained variability becomes artificially smaller
In the case of dependent data → increased Type I error rate
Measuring Relations Between Variables
We can see whether, as one variable deviates from its own mean, the other deviates in the same way from its own mean, the opposite way, or stays the same
This can be done by calculating the covariance
If there is a similar (or opposite) pattern, we say the two variables covary
Variance
measure of how much a group of scores deviates from the mean of a single variable
Average squared deviation from the mean
Covariance
Tells us by how much a group of scores on two variables differ from their respective means
Covariance Steps
Calculate the deviation (error) between the mean and each subject’ score for the first variable (x)
Calculate the deviation (error) between the mean and their score for the second variable (y)
Multiply these deviations (error) values
These multiplied numbers are called the cross product deviations
Add up these cross product deviations
Divide by N-1 → result is covariance
COVARIANCE IS THE AVERAGE _________
CROSS-PRODUCT DEVIATION
Limitations of Covariance
depends upon the units of measurement
E.g. the covariance of two variables measures in miles and dollars would be much smaller than if the variables were measures in feet and cents, even if the relationship was exactly the same
Solution to the limitations of covariance
Solution –> standardize it
Divide by the product of standard deviations of both variables
The standardized version of covariance is known as the correlation coefficient
Relatively unaffected by units of measurement
Correlation Coefficient
When x and y are both continuous, their correlation is called the Pearson Product Moment Correlation Coefficient
Correlation statistics are standardized and can range from -1 to 1
It can be used as a measure of effect size
The equation for Correlation is similar to z scores….
both a standardized statistic that can compare across samples
Convention for effect size in correlation
Convention
.1 = small effect
.3 = medium effect
.5 = large effect
Correlation and Causality
Correlation is a necessary but not sufficient criteria for causality
Possible directions of causality:
X → Y
X ← Y
A third factor leads to changes in both x and y
Correlation is by coincidence
Things needed to determine causality
Temporal precedence
Demonstrating empirical association between variables
Control for confounds
Types of Correlations
Pearson’s Correlation (r)
Spearman’s p (greek rho) (rs)
Kendall’s Tau (t)
Point Biserial Correlation (rpb)
Biserial Correlation (rb)
Phi Coefficient (φ)
Pearson’s Correlation (r)
For analyzing relationship between two continuous variables
Assumes normality, homogeneity of variance, independence of data, ANDlinear relationship
Spearman’s p (greek rho) (rs)
When one or both variables are ordinal
E.g. SAT score and high school class standing
Nonparametric alternative to pearson’s coefficient
Does not assume linear relationship
Kendall’s Tau (t)
Better for smaller samples
Possible to be helpful in cases of ranks?
Point Biserial Correlation (rpb)
One continuous and one dichotomous variable
Used for continuous and binary variable that is quantitatively dichotomous
Biserial Correlation (rb)
One continuous and one dichotomous variable
When the variable is not truly quantitatively dichotomous, but treated as such
E.g. pass/fail class grades (categories based on continuum)
Median split → create groups
Phi Coefficient (φ)
Two categorical variables
Procedure in Experimental research to Rule Out Confounding Variables
Random assignment
Coefficient of Determination, R2
By squaring the value of r, you get the proportion of variance in one variable shared by the other(s), R2
Can only take the value of 0-1, because it is a squared value (must be positive)
Spurious Relationship
That the two variables have no causal connection, yet it may be inferred that they do, due to an unknown confounding factor
I.e. ice cream sales and death by drowning → third confounding is temperature outside
Caveat for biserial and point biserial correlations
when group sizes for binary variable are unequal → correlation values can become smaller
