Research Methods Flashcards
content validity
Evidence that the content of a test corresponds to the content of the construct it was designed to cover
Ecological validity
evidence that the results of a study, experiment or test can be applied, and allow inferences, to real-world conditions.
reliability
the ability of the measure to produce the same results under the same conditions.
test-retest reliabillity
The ability of a measure to produce consistent results when the same entities are tested at two different points in times.
Correlational research
observing what naturally goes on in the world without directly interfering with it.
Cross-sectional research
This term implies that data come from people at different age points with different people representing each age point
Experimental research
- One or more variable is systematically manipulated to see their effect (alone or in combination) on an outcome variable.
- Statements can be made about cause and effect
Systematic variation
differences in performance created by a specific experimental manipulation
unsystematic variation
Differences in performance created by unknown factors. (age, gender, IQ, Time of Day, Measurement error etc.)
Randomization
Minimizes unsystematic variation
Frequency distributions (AKA Histograms)
A graph plotting values of observations on the horizontal axis, with a bar showing how many times each value occurred in the data set.
The ‘Normal’ Distribution
- Bell shaped
- Symmetrical around the centre
Properties of frequency distributions
- Skew
- Kurtosis
Skew
- The symmetry of the distribution
- Positive skiew (scores bunched at low values with the tail pointing to high values)
- negative skew (scores bunched at high values with the tail pointing to low values)
Kurtosis
- the ‘heaviness of the tails
- leptokurtic = heavy tails
- Platykurtic = light tails
Deviance
- we can calculate the spread of scores by looking at how different each score is from the center of a distribution eg: the mean
Sum of squared errors (SS)
- indicates the total dispersion, or total deviance of scores from the mean
- it’s size is dependent on the number of scores in the data.
- More useful to work with the average dispersion, known as the variance
The sum of squares, variance, and standard deviation represent the same thing
- the ‘fit’ of the mean to the data
- the variability in the data
- how well the mean represents the observed data
- error
Population
- The collection of units (be they people, plankton, plants, cities, suicidal authors etc.) to which we want to generalize a set of findings or a statistical model.
Sample
a smaller (but hopefully representative) collection of units from a population used to determine truths about that population
calculating ‘error’
- a deviation is the difference between the mean and an actual data point
- deviations can be calculated by taking each score and subtracting the mean from it:
deviance=outcome(i)-model(i)
Sum of squared errors
- we could add the deviations to find out the total error
- deviations cancel out because some are positive and others negative
- therefore, we square each deviation
- if we add these squared deviations we get the Sum of Squared Errors (SS)
Mean squared error
Although the SS is a good measure of the accuracy of our model, it depends on the amount of data collected. To overcome this problem we use
The standard error
- SD tells us how well the mean represents the sample data
- but, if we want to estimate this parameter in the population, then we need to
why can’t we prove certainty in stats
- because it’s inferential statistics
- it’s based on probability
Type I error
- occurs when we believe that there is a genuine effect in our population, when in fact there isn’t
- the probability is the a-level (usually .05)
Type II error
- occurs when we believe that there is no effect in the population when, in reality, there is.
- The probability is the B-level (often 0.2)
regression has no IV, DV
Predictor = IV
Outcome = DV
misconceptions around p-values No1
A significant result means that the effect is important =
no, because significance depends on sample size
misconceptions around p-values No 2
A non-significant result means that the null hypothesis is true = no, a non-significant result tells us only that the effect is not big enough to be found (given our sample size), it doesn’t tell es that the effect size is zero.
misconceptions around p-values No 3
A significant result means that the null hypothesis is false? = no, it is logically not possible to conclude this
Researcher degrees of freedom
A scientist has many decisions to make when designing and analysing a study
Continuous DV over categorical DV
ANOVA
Noir
Nominal
Ordinal
Interval
Ratio
Measurements of error
Deviances
how to get rid of a 0 deviance
square it
Standard deviation
- estimate of error
crombachs alpha
tests internal validity
random allocation
- attempt to control for individual difference
- each person has an equal chance
Matched pairs
-is a within subjects
- doesnt test same subject but matches on characteristics
mean squared error
refers to variance
mean
- the one number that bests represents a normal distribution
- best represents central tendency
- it needs to be thought of as a model not a number
Degrees of Freedom
The number of scores that are free to vary
Variance
SS/df
within group variance
the estimate of error
test statistic for and ANOVA
F statistic
Big F
is significant and is ratio divided by error
levels
the divides with the IV’s
ANOVA
- tests the mean differences between levels
- controls for type I error
Cheffe’s
- Post-hoc
- the most conservative (least likely to make a Type I error)
- keeps a large estimate of error
when to do post-hoc
After
- 3 or more levels
- IFF
- main effect is significant
factorial designs
- have multiple IV’s (Factors)
interaction
- the effect of one IV depends upon the level of another IV
- are more important to interpret than main effect
- should be interpreted first
- IV-DV-IV
how to check interaction
- visual inspection, if the lines are parallel there is no interaction
- the differences between cell means (if the differences are the same there may be no interaction)
how many hypothesis for a two-way ANOVA
- 3
- one for the IV A on the DV
- IV B on the DV
- Interactional effect on the DV
If there is a significant interaction
- use a simple effect analysis
Independent design
- different entities in all conditions
Repeated measures design
- the same entities in all conditions
Mixed design
- different entities in all conditions of at least one IV, the same entities in all conditions of at least one other IV
- SPANOVA
you can have a significant interaction without having
- significant main effect
ANOVA (Between)
main effect
ANOVA (Within)
Error
F
- total variability
=MS between/MS Within
all statistical statements
Statistic, degrees of freedom, value, significance, effect size
tests of between-Subjects effects
- the amount of stats needed depends on factor levels
repeated measures attempt to control for
- individual differences
when accounting for differences in statistical equations
- you must make changes to both numerator and denominator
Advantages of repeated measures
- Sensitivity (unsystematic variance is reduced, more sensitive to experimental effects)
- Economy (requires less participants)
disadvantage of repeated measure
- practice effect
- fatigue
can you use post hoc for repeated measure?
- no, you need to use contrast (pre-planned comparison)
Covariance
the amount to which scores vary together
- needs to be measured before everything else
how to tell if it is an ANCOVA
- there is no interaction
when you have a significant covariate
-make an adjustment to the means
- compare the estimated margin means
what N is needed to reach normality
30
