Methods Flashcards
What happens when the data provides sufficient evidence against the null hypothesis?
-reject the null hypothesis and adopt the alternative hypothesis
What happens when the data does not provide sufficient evidence against the null hypothesis?
-reject the alternative hypothesis
When the alternative hypothesis is rejected, does this make the null hypothesis true?
not necessarily
What is NHST?
Null Hypothesis Significance Testing
What is a problem with Null Hypothesis?
it is a hypothetical construct assuming that the difference between conditions is exactly 0.0000 which doesn’t exist in real world scenarios
Why should a non significant result never be interpreted as ‘no difference/no relationship between variables/means’?
A non-significant result only tells us that the effect is not large enough to be
detected with the given sample size
How do researchers set up their research with regards to hypothesis?
Researchers must set up their research so that the ‘desired’ outcome is to reject the null hypothesis
How are statistical significances NOT practical significance?
with a sufficiently large sample, very small effects can become statistically significant, although they may be unimportant for any practical purpose.
Why is All or Nothing thinking a problem with NHST?
p-values that only differ by a small amount could end up reaching completely opposite conclusions
what does ‘significance’ imply in statistics?
implies that something is unlikely to have occurred by chance and may therefore have a systematic cause
What is the significance threshold in psychology?
a=0.05
What is a criticism of the 5% significance threshold?
significance at a 5% threshold indicates limited
evidence that the data is not entirely random
What is a proposed alternative to NHST?
Effect Size
What is ‘Effect Size’?
Provides an estimate of the size of group differences or the effect of
treatment
What is the use of effect Size?
-Measure of how large an effect is, which the P and F value cannot tell
-Estimating sample size needed for sufficient statistical power
-Used when combining data across studies
What are three types of effect sizes?
-Group Difference Indices
-Strength of Association
-Risk Estimates
What is the difference in sample and population means?
A sample mean is a good approximation of a population mean that is normally unknown
How can Effect Size be calculated when population means are unknown?
Where M is ‘Sample Means’
Effect size= M1 -M2
What is a disadvantage of calculating Effect size using difference in means?
The measure is dependent on measurement scale
ie. 180cm-165cm= 15
1.8m-1.65m= 0.15
What do we need to know to calculate standard mean difference?
-population means OR sample means (M)
-Sigma
=M1-M2
————-
Sigma
How do measures of group difference indices differ?
Measures differ on how the population variance is estimated from the
data
What is the most commonly reported measure of group difference indices?
Cohen’s d
When is Hedge’s g used instead of Cohen’s d?
When two groups have different sample sizes and the sample sizes are below 20,
What is different in the Glass’ Delta compared to Cohen’s d and Hedge’s g?
it uses the standard deviation from the control group rather than the pooled standard deviation from both groups
When is Glass’ delta normally used?
when several treatments are compared to the control group
What measure of group difference is used for a paired sample t-test?
Cohen’s d
what is a paired samples t-test?
compares the means of two measurements taken from the same individual, object or related units ie. a measurement taken at two different times
Classifications of effect size:
d between 0.2 and 0.49 = small
d between 0.5 and 0.79 = medium
d of 0.8 and higher = large
What is partial eta squared used for?
Can be used for a factorial design (ANOVA). Measures linear and nonlinear association (in contrast to correlation)
What does Classical eta measure?
measures the proportion of the total variance
in a dependent variable that is associated with the variance of a given factor in an ANOVA model.
What does partial eta measure?
measure in which the effects of other independent variables and interactions are partialled out
Three Risk Estimates…
1- Relative Risk
2- Odds Ratio
3- Risk Difference
Smokers have an estimated RR (relative risk) equal to 23 to develop lung cancer. what does this mean?
smokers are 23 times more likely to develop lung cancer
Why might Odds Ratio be used?
When risk is small the odds ratio approximates the relative risk
What does a ‘Risk Difference’ indicate in a study investigating the chances of smokers vs non smokers developing lung cancer?
Difference between proportion of treatment group that contract the disease and proportion of controls that contract the disease. Can be used to estimate number of cases avoided by a treatment. Reflects overall probability of getting disease.
If the null hypothesis is true but is is rejected, what error is this?
Type 1 error
If the null hypothesis is false but it is accepted, what type of error is this?
Type 2 error
What is ‘Power’?
power is the probability of correctly rejecting the null hypothesis. Power determines how likely an effect is detected
What are factors that can influence ‘Power’?
-Effect Size (unreliable measures reduce effect size by inflating estimate of sigma)
-Alpha level (one tailed test has higher power than two tailed)
-Sample size
What is prospective power and how is it calculated?
Computed before the study’s data are collected.
three steps; hypothesis effect size; alpha level; planned sample size
How can you get an estimate of effect size?
-do a pilot experiment and compute effect size
-do a meta analysis and compute weighted effect size
-use Cohen’s estimates for small, medium, and large effect size
when is a power important?
A power of at least 80% is usually considered acceptable.
Underpowered (power<80%) studies are useless and unethical (waste of resources and people’s time)
What is an Observed Power?
Computed after study is completed.
Assumes effect size in the sample equals effect size in the population
Generally not very useful
What is an exception where the Observed Power is useful?
In a meta-analysis. It provides an indication as to which results to assign a higher weight
How can power be increased?
-Adding Participants
-Choose a less stringent significance level (usually not an option)
-Increase the hypothesised effect size
-Use as few groups as possible
-Use covariates variables
-Use a repeated measure design
-Use measures sensitive to change
How do you calculate Pearson’s R?
- Calculate covariance between the X and Y variables, and then standardize
- Convert the X and Y scores to z-scores (standard scores), then divide by n
What does Pearson’s r measure?
measuring a linear correlation between two variables
what is a linear correlation?
a statistical relationship between two variables where the data points tend to fall along a straight line when plotted on a graph
Difference between ‘Correlation’ and ‘Regression’?
Correlation: is there a relationship between 2 variables?
Regression: how well does one variable predict the other variable?
What is required when predicting regression?
Prediction requires calculating a line of best fit (an equation)
What is the dependent variable?
the variable that you are trying to predict
What is the independent variable?
the variable that you are trying to predict from
How many Predictor Variables in ‘Simple Linear Regression’ and ‘Multiple Regression’?
Simple linear regression = 1 predictor variable
Multiple regression = 1+ predictor variables
What is the regression equation for a straight line?
Y=a + bX
(where a= intercept and b=slope)
What to do when there is ‘error’ in regressions?
Line of Best Fit: find a regression line that provides the best prediction possible
i.e., a regression line that minimizes error.
How to calculate Line of Best Fit:
Step1: for each data point, calculate the deviation, then square it.
Step2: across the dataset, add up all deviations (→ sum of squared deviations).
Best fit: the equation that produced the smallest SSERROR.
How to calculate R:
Step1: convert X and Y into z-scores
Step2: multiply z(X) by z(Y)
Step3: add up
Step4: divide by n-1
How to calculate the equation of a line:
Format:
Y=a + bX
(where b is the slope, and a is the starting point)
Calculate B: b = r × (SD of Y / SD of X)
(where r is the correlation between X and Y)
Calculate A:
a = mean(Y) - b × mean(X)
How to calculate explained variance (the long way):
SSREGRESSION + SSERROR
(SSX) (SSRESIDUAL)
Sum of Squares Y (SStotal)= how much variance there is in Y in total
Sum of Squares X (SSregression)= variance X can explain
Sum of Squares Residual (SSerror)= how much variance is not explained
How to calculate Sum of Squares Y (sum of squares total):
Total variance in Y
Step1: calculate the difference (deviation) between each score and the mean
Step2: square the deviations
Step3: add up
How to calculate Sum of Squares X (Sum of Squares regression):
Step1: calculate the difference (deviation)
between the predicted and the observed score
Step2: square the deviations
Step3: add up
How to calculate unexplained variance:
SSTotal= SSregression (variance in Y) + SSerror (variance in X)
Easy way to work out explained variance:
-Calculate R2
What do you use to determine whether a model accounts for a statistically significant amount of variance?
F statistic
Step1: calculate Mean Squares (SS / df)
Step2: calculate F ratio
(MSregression/ MSresidual)
Describe ‘Correlation’:
relationship between 2 variables
* calculate r and R2
Describe a ‘Partial correlation’
relationship between 2 variables while accounting for another variable or variables
* calculate partial r and R2
Describe ‘simple linear regression’
predicting one variable from another variable
* calculate R and R2
Describe ‘multiple linear regression’
predicting one variable from 2+ other variables
* calculate multiple R and multiple R2
Two key concepts of linear regressions:
1) Least squares
2)Variance accounted for (R^2)
What is R^2 in simple linear regression?
R2 (coefficient of determination) is the amount of variance explained by that single predictor
What is Multiple R^2 in multiple regression?
Multiple R2 (coefficient of multiple determination) is the amount of variance explained by those multiple predictors
Multiple Linear regression equation:
Y=a+bx+bx+bx+bx+…….bx
-where a is the Y value when all predictor variables are zero
-where b is a partial regression coefficient and represent the change in Y associated with a 1 unit change in a particular x
What is ‘error’ in a linear regression?
variance in the model that is not explained
what does it mean when a regression has the ‘least squares’?
find regression line that provides the best prediction possible, i.e., a regression line that minimizes error
If X accounts for very little variance, is the predictor strong or not?
X is not a strong predictor
What is unique variance?
variance that can be attributed only to 1 variable
What is shared variance?
variance that can be attributed to 2+ variables
What are possibilities for the relationship between X and Y after controlling for Z (a third variable)?
- no change in correlation
- weaker (still significant) correlation
- stronger correlation
Theory driven approach to regressions:
you start with a specific hypothesis → use only the predictors necessary to test this hypothesis
Data driven approach to regressions:
you start with a broad hypothesis → use as many predictors as necessary to test this hypothesis or as indicated by previous literature
What is simultaneous regression?
all the variables are entered in together, irrespective of their absolute or relative importance
What is hierarchical regression?
you decide (you can enter variables in blocks, with your decisions being driven by previous research and hypotheses
What is stepwise (statistical regression)?
Forward regression:
your computer programme (e.g., SPSS) will find the single best predictor and enter it as the first variable; the variable that accounts for the highest proportion of the remaining variance is entered next and so on
Backward regression:
all variables are entered initially and the worst predictors (i.e., the predictors that account for the least variance) are removed in turn
