Methods Flashcards
1
Q
What happens when the data provides sufficient evidence against the null hypothesis?
A
-reject the null hypothesis and adopt the alternative hypothesis
2
Q
What happens when the data does not provide sufficient evidence against the null hypothesis?
A
-reject the alternative hypothesis
3
Q
When the alternative hypothesis is rejected, does this make the null hypothesis true?
A
not necessarily
4
Q
What is NHST?
A
Null Hypothesis Significance Testing
5
Q
What is a problem with Null Hypothesis?
A
it is a hypothetical construct assuming that the difference between conditions is exactly 0.0000 which doesn’t exist in real world scenarios
6
Q
Why should a non significant result never be interpreted as ‘no difference/no relationship between variables/means’?
A
A non-significant result only tells us that the effect is not large enough to be
detected with the given sample size
7
Q
How do researchers set up their research with regards to hypothesis?
A
Researchers must set up their research so that the ‘desired’ outcome is to reject the null hypothesis
8
Q
How are statistical significances NOT practical significance?
A
with a sufficiently large sample, very small effects can become statistically significant, although they may be unimportant for any practical purpose.
9
Q
Why is All or Nothing thinking a problem with NHST?
A
p-values that only differ by a small amount could end up reaching completely opposite conclusions
10
Q
what does ‘significance’ imply in statistics?
A
implies that something is unlikely to have occurred by chance and may therefore have a systematic cause
11
Q
What is the significance threshold in psychology?
A
a=0.05
12
Q
What is a criticism of the 5% significance threshold?
A
significance at a 5% threshold indicates limited
evidence that the data is not entirely random
13
Q
What is a proposed alternative to NHST?
A
Effect Size
14
Q
What is ‘Effect Size’?
A
Provides an estimate of the size of group differences or the effect of
treatment
15
Q
What is the use of effect Size?
A
-Measure of how large an effect is, which the P and F value cannot tell
-Estimating sample size needed for sufficient statistical power
-Used when combining data across studies
16
Q
What are three types of effect sizes?
A
-Group Difference Indices
-Strength of Association
-Risk Estimates
17
Q
What is the difference in sample and population means?
A
A sample mean is a good approximation of a population mean that is normally unknown
18
Q
How can Effect Size be calculated when population means are unknown?
A
Where M is ‘Sample Means’
Effect size= M1 -M2
19
Q
What is a disadvantage of calculating Effect size using difference in means?
A
The measure is dependent on measurement scale
ie. 180cm-165cm= 15
1.8m-1.65m= 0.15
20
Q
What do we need to know to calculate standard mean difference?
A
-population means OR sample means (M)
-Sigma
=M1-M2
————-
Sigma
21
Q
How do measures of group difference indices differ?
A
Measures differ on how the population variance is estimated from the
data
22
Q
What is the most commonly reported measure of group difference indices?
A
Cohen’s d
23
Q
When is Hedge’s g used instead of Cohen’s d?
A
When two groups have different sample sizes and the sample sizes are below 20,
24
Q
What is different in the Glass’ Delta compared to Cohen’s d and Hedge’s g?
A
it uses the standard deviation from the control group rather than the pooled standard deviation from both groups
25
When is Glass' delta normally used?
when several treatments are compared to the control group
26
What measure of group difference is used for a paired sample t-test?
Cohen's d
27
what is a paired samples t-test?
compares the means of two measurements taken from the same individual, object or related units ie. a measurement taken at two different times
28
Classifications of effect size:
d between 0.2 and 0.49 = small
d between 0.5 and 0.79 = medium
d of 0.8 and higher = large
29
What is partial eta squared used for?
Can be used for a factorial design (ANOVA). Measures linear and nonlinear association (in contrast to correlation)
30
What does Classical eta measure?
measures the proportion of the total variance
in a dependent variable that is associated with the variance of a given factor in an ANOVA model.
31
What does partial eta measure?
measure in which the effects of other independent variables and interactions are partialled out
32
Three Risk Estimates...
1- Relative Risk
2- Odds Ratio
3- Risk Difference
33
Smokers have an estimated RR (relative risk) equal to 23 to develop lung cancer. what does this mean?
smokers are 23 times more likely to develop lung cancer
34
Why might Odds Ratio be used?
When risk is small the odds ratio approximates the relative risk
35
What does a 'Risk Difference' indicate in a study investigating the chances of smokers vs non smokers developing lung cancer?
Difference between proportion of treatment group that contract the disease and proportion of controls that contract the disease. Can be used to estimate number of cases avoided by a treatment. Reflects overall probability of getting disease.
36
If the null hypothesis is true but is is rejected, what error is this?
Type 1 error
37
If the null hypothesis is false but it is accepted, what type of error is this?
Type 2 error
38
What is 'Power'?
power is the probability of correctly rejecting the null hypothesis. Power determines how likely an effect is detected
39
What are factors that can influence 'Power'?
-Effect Size (unreliable measures reduce effect size by inflating estimate of sigma)
-Alpha level (one tailed test has higher power than two tailed)
-Sample size
40
What is prospective power and how is it calculated?
Computed before the study’s data are collected.
three steps; hypothesis effect size; alpha level; planned sample size
41
How can you get an estimate of effect size?
-do a pilot experiment and compute effect size
-do a meta analysis and compute weighted effect size
-use Cohen's estimates for small, medium, and large effect size
42
when is a power important?
A power of at least 80% is usually considered acceptable.
Underpowered (power<80%) studies are useless and unethical (waste of resources and people’s time)
43
What is an Observed Power?
Computed after study is completed.
Assumes effect size in the sample equals effect size in the population
Generally not very useful
44
What is an exception where the Observed Power is useful?
In a meta-analysis. It provides an indication as to which results to assign a higher weight
45
How can power be increased?
-Adding Participants
-Choose a less stringent significance level (usually not an option)
-Increase the hypothesised effect size
-Use as few groups as possible
-Use covariates variables
-Use a repeated measure design
-Use measures sensitive to change
46
How do you calculate Pearson's R?
1. Calculate covariance between the X and Y variables, and then standardize
2. Convert the X and Y scores to z-scores (standard scores), then divide by n
47
What does Pearson's r measure?
measuring a linear correlation between two variables
48
what is a linear correlation?
a statistical relationship between two variables where the data points tend to fall along a straight line when plotted on a graph
49
Difference between 'Correlation' and 'Regression'?
Correlation: is there a relationship between 2 variables?
Regression: how well does one variable predict the other variable?
50
What is required when predicting regression?
Prediction requires calculating a line of best fit (an equation)
51
What is the dependent variable?
the variable that you are trying to predict
52
What is the independent variable?
the variable that you are trying to predict from
53
How many Predictor Variables in 'Simple Linear Regression' and 'Multiple Regression'?
Simple linear regression = 1 predictor variable
Multiple regression = 1+ predictor variables
54
What is the regression equation for a straight line?
Y=a + bX
(where a= intercept and b=slope)
55
What to do when there is 'error' in regressions?
Line of Best Fit: find a regression line that provides the best prediction possible
i.e., a regression line that minimizes error.
56
How to calculate Line of Best Fit:
Step1: for each data point, calculate the deviation, then square it.
Step2: across the dataset, add up all deviations (→ sum of squared deviations).
Best fit: the equation that produced the smallest SSERROR.
57
How to calculate R:
Step1: convert X and Y into z-scores
Step2: multiply z(X) by z(Y)
Step3: add up
Step4: divide by n-1
58
How to calculate the equation of a line:
Format:
Y=a + bX
(where b is the slope, and a is the starting point)
Calculate B: b = r × (SD of Y / SD of X)
(where r is the correlation between X and Y)
Calculate A:
a = mean(Y) - b × mean(X)
59
How to calculate explained variance (the long way):
SSREGRESSION + SSERROR
(SSX) (SSRESIDUAL)
Sum of Squares Y (SStotal)= how much variance there is in Y in total
Sum of Squares X (SSregression)= variance X can explain
Sum of Squares Residual (SSerror)= how much variance is not explained
60
How to calculate Sum of Squares Y (sum of squares total):
Total variance in Y
Step1: calculate the difference (deviation) between each score and the mean
Step2: square the deviations
Step3: add up
61
How to calculate Sum of Squares X (Sum of Squares regression):
Step1: calculate the difference (deviation)
between the predicted and the observed score
Step2: square the deviations
Step3: add up
62
How to calculate unexplained variance:
SSTotal= SSregression (variance in Y) + SSerror (variance in X)
63
Easy way to work out explained variance:
-Calculate R2
64
What do you use to determine whether a model accounts for a statistically significant amount of variance?
F statistic
Step1: calculate Mean Squares (SS / df)
Step2: calculate F ratio
(MSregression/ MSresidual)
65
Describe 'Correlation':
relationship between 2 variables
* calculate r and R2
66
Describe a 'Partial correlation'
relationship between 2 variables while accounting for another variable or variables
* calculate partial r and R2
67
Describe 'simple linear regression'
predicting one variable from another variable
* calculate R and R2
68
Describe 'multiple linear regression'
predicting one variable from 2+ other variables
* calculate multiple R and multiple R2
69
Two key concepts of linear regressions:
1) Least squares
2)Variance accounted for (R^2)
70
What is R^2 in simple linear regression?
R2 (coefficient of determination) is the amount of variance explained by that single predictor
71
What is Multiple R^2 in multiple regression?
Multiple R2 (coefficient of multiple determination) is the amount of variance explained by those multiple predictors
72
Multiple Linear regression equation:
Y=a+bx+bx+bx+bx+.......bx
-where a is the Y value when all predictor variables are zero
-where b is a partial regression coefficient and represent the change in Y associated with a 1 unit change in a particular x
73
What is 'error' in a linear regression?
variance in the model that is not explained
74
what does it mean when a regression has the 'least squares'?
find regression line that provides the best prediction possible, i.e., a regression line that minimizes error
75
If X accounts for very little variance, is the predictor strong or not?
X is not a strong predictor
76
What is unique variance?
variance that can be attributed only to 1 variable
77
What is shared variance?
variance that can be attributed to 2+ variables
78
What are possibilities for the relationship between X and Y after controlling for Z (a third variable)?
1. no change in correlation
2. weaker (still significant) correlation
3. stronger correlation
79
Theory driven approach to regressions:
you start with a specific hypothesis → use only the predictors necessary to test this hypothesis
80
Data driven approach to regressions:
you start with a broad hypothesis → use as many predictors as necessary to test this hypothesis or as indicated by previous literature
81
What is simultaneous regression?
all the variables are entered in together, irrespective of their absolute or relative importance
82
What is hierarchical regression?
you decide (you can enter variables in blocks, with your decisions being driven by previous research and hypotheses
83
What is stepwise (statistical regression)?
Forward regression:
your computer programme (e.g., SPSS) will find the single best predictor and enter it as the first variable; the variable that accounts for the highest proportion of the remaining variance is entered next and so on
Backward regression:
all variables are entered initially and the worst predictors (i.e., the predictors that account for the least variance) are removed in turn
84
What does an F value represent?
what the model can explain / what it cannot explain
85
What is Residual variance?
unexplained variance
86
What is regression variance?
variance explained by the regression
87
What is the Null hypothesis of a Model 2?
Change in R2 from Model 1 to Model 2 = 0
88
What is a useful way of dealing with nuisance or suppressor variables?
Regression
89
What is collinearity?
Two predictors are said to be collinear if they are highly correlated with one another (r>.75).
In other words, they may be measuring the same construct and it is difficult to estimate independent contributions of each variable
90
Collinearity solutions:
Unless you have a research question that specifically requires keeping both predictors, the best approach is to drop one of these predictors from your model.
Or conduct a PCA (principal component analysis).
91
What type of variable can be employed as a predictor in regression?
any type of variable can be employed as a predictor (i.e., independent) variable.
Researchers often want to use, or are forced to use, categorical variables
92
What must the dependent variable (Y) be in regression?
continuous and normally distributed
93
What is dummy coding?
To code category membership where k = number of categories, you need k - 1 dummy variables
-2 categories need 1 dummy variable
-3 categories need 2 dummy variables
94
When are linear regressions useful?
Linear regressions only provide a valid measure of the relationship between two variables when that relationship is linear (when it can be described by a straight line)
95
What is used for non-linear relationships?
Polynomials...a trick to get a linear method to model a non linear relationship
-non linear quadratic: modelled by entering square of predictor/independent variable
-non-linear cubic: modelled by entering cube of the predictor/independent variable
96
How can you test Non-Linear Relationships?
Hierarchal regression and:
1- enter predictor variable
2- then the square of it (f there is a significant change, then there is a significant non-linear (quadratic) component to
the relationship between the predictor and criterion
3- then the cube of it. ( if there is a significant change in R2 then there is a significant non-linear cubic component to the relationship between the predictor and criterion)
97
Alternative to mediation approach (Baron and Kenny):
An alternative is to estimate the indirect effect and its significance using the Sobel test (Sobel. 1982). If the Sobel test is significant, there is significant mediation
98
What question does an ANCOVA answer?
Can I control for a continuous variable before I
look at the effect of my categorical variable?
99
Which variable does the ANCOVA always look at first?
the continuous variable
100
what is an ANOVA?
ANOVA is multiple regression with only categorical predictors
101
what is an ANCOVA?
ANCOVA is an ANOVA (and hierarchical multiple regression), where 1 continuous variable is entered first into the model, to
“control for” that variable
102
What does controlling for a variable mean?
“partialling out” the influence of that variable (the covariate) on the outcome.
You adjust the means of the categorical
predictor to account for the influence of the covariate
103
How does an ANCOVA control for a variable?
You measured a continuous variable that covaries with the outcome, but which is not of interest. The ANCOVA first takes away any variance in Y that is due to this covariate, then proceeds with the rest of the analysis
104
When is an ANCOVA used?
* To test for differences between group means when we know that an extraneous variable affects the outcome variable.
* Used to adjust for known extraneous variables
105
Advantages of ANCOVA:
Reduces Error Variance
* By explaining some of the unexplained variance (SSR) the error variance in the
model can be reduced.
Greater Experimental Control:
* By controlling known extraneous variables, we gain greater insight into the effect of the predictor variable(s).
106
Assumptions of ANCOVA test:
-No collinearity between the two IV’s, i.e. no high correlation between the categorical IV and the covariate
-Homogeneity of slopes: The relationship between covariate and and DV has to be similar in all conditions of the categorical IV
107
what is an interaction effect?
The combined effect of two variables on another is known conceptually as moderation, and in statistical terms as an interaction effect
108
Why is it common to transform predictors using grand mean centring in ANCOVA?
The interaction term makes the bs for the main predictors uninterpretable in many situations.
109
What does centring variables refer to?
Centring refers to the process of transforming a variable into deviations around a fixed point
110
What is mediation?
Refers to a situation when the relationship between a predictor variable and outcome variable can be explained by their relationship
to a third variable (the mediator)
111
How is mediation tested?
Mediation is tested through three regression
models
1. Predicting the outcome from the predictor variable.
2. Predicting the mediator from the predictor variable.
3. Predicting the outcome from both the predictor variable and the mediator.
112
Four conditions of mediation:
1. The predictor must significantly predict the
outcome variable.
2. The predictor must significantly predict the
mediator.
3. The mediator must significantly predict the
outcome variable.
4. The predictor variable must predict the outcome
variable less strongly in model 3 than in model 1
113
Limitations of mediation approach (Baron and Kenny):
How much of a reduction in the relationship between the predictor and outcome is necessary to infer mediation?
* people tend to look for a change in significance, which can lead to the ‘all or
nothing’ thinking that p-values encourage
114
Alternative to mediation approach (Baron and Kenny):
An alternative is to estimate the indirect effect and its significance using the Sobel test (Sobel. 1982). If the Sobel test is significant, there is significant mediation
