Research Methods Beth 2 Flashcards
What are the three things to consider when choosing an appropriate statistical test?
1) Are we testing a difference or relationship?
2) What type of experimental design has been used/Are we looking for an association or correlation?
3) What level of measurement was used?
What are the 8 statistical tests?
Sign test.
Chi-squared test.
Wilcoxon.
Mann-Whitney.
Related t-test.
Unrelated t-test.
Spearman’s rho.
Pearson’s r
If we are testing the difference, used independent groups, and collected nominal level data, what statistical test do we use?
Chi-squared test
If we are testing the difference, used independent groups, and collected at least ordinal level data, what statistical test do we use?
Mann-Whitney
If we are testing the difference, used independent groups, and collected at least interval level data, what statistical test do we use?
Unrelated t-test
If we are testing the difference, used repeated measures or matched pairs, and collected nominal level data, what statistical test do we use?
Sign test
Which statistical test do we need to be able to calculate?
Sign test
If we are testing the difference, used repeated measures or matched pairs, and collected at least ordinal level data, what statistical test do we use?
Wilcoxon
If we are testing the difference, used repeated measures or matched pairs, and collected at least interval level data, what statistical test do we use?
Related t-test
If we are looking for an association and collected nominal level data, what statistical test do we use?
Chi-squared test
If we are testing the relationship, are looking for a correlation, and collected at least ordinal level data, what statistical test do we use?
Spearman’s rho
If we are testing the relationship, are looking for a correlation, and collected at least interval level data, what statistical test do we use?
Pearson’s r
What do inferential statistics allow us to do?
Conclude whether any difference or relationship found is statistically significant
At the end of statistical testing, what does the researcher do?
Support one hypothesis and reject the other
What is significance?
When the research findings are sufficiently strong to enable a researcher to reject the null hypothesis and accept the alternative hypothesis
What does a low statistical significance mean?
There is a high probability the results are due to chance so testing the target population would yield different results
What does a high statistical significance mean?
There is a low probability the results are due to chance so testing the target population would yield similar results
What is probability?
A numerical measure of the likelihood that certain events will occur
What is the significance level?
A margin of error that a researcher is willing to accept in their research
What is the most commonly selected significance level?
5%
What is a more stringent level of significance?
1%
What is a more lenient level of significance?
10%
Using significance levels, what does it mean if the results are statistically significant?
The likelihood of the results coming about by chance is equal to or less than 5%/1%/10%
Which statistical tests are non-parametric?
Spearman’s rho, Wilcoxon, Mann-Whitney, Chi-squared test, Sign test
(the top 2 rows)
Which statistical tests are parametric?
Pearson’s r, Related t-test, Unrelated t-test
(the bottom row)
When do we use the more stringent level of significance?
Often for medical research
What are levels of measurement?
How the psychologist has measured their dependent variable - nominal, ordinal, interval
What is nominal data?
Data which appears in categories (often not numerical) - often referred to as frequency data. Data is discrete (cannot be in more than 1 category)
What is ordinal data?
Data is ordered but the intervals between each value are unequal, can be in rank order, often comes from the researcher creating scales themselves
What is interval data?
Data can be ordered and the intervals between each value are equal e.g. time, length, temperature (already known scales)
What is the order of the levels of measurement from least to most sophisticated?
Nominal, ordinal, interval
Which is the only statistical test which tests for an association?
Chi-squared test - can only be nominal level data
Is an association a relationship?
No but on the statistical tests table it looks like it is
What is a statistical test (how do they work)?
They are formulae
What level of measurement is ‘number of x’? And why?
Can argue ordinal or interval but make sure to explain why.
Ordinal: x may not be of equal difficulty
Interval: there is equal distance between numbers
What measure of central tendency is used for nominal data?
Mode
What measure of central tendency is used for nominal data?
Median
What measure of central tendency is used for ordinal data?
Mean
What measure of dispersion is used for interval data?
Range
What measure of dispersion is used for ordinal data?
Range
What measure of dispersion is used for interval data?
Standard Deviation
If there are outliers, what measure of central tendency do we use for interval data?
Median
How do we answer the question: Why is x the best measure of central tendency/dispersion for this data?
Because it’s nominal/ordinal/interval data
What are 3 reasons to use the sign test?
Testing for a difference, repeated measures or matched pairs design, nominal level data
What are the steps for calculating the sign test?
Convert data into a summary table (condition A is always the column after participant).
Work out the sign of difference (A-B).
Exclude any with an = sign.
Add up total number for - and + (separately).
S is the number of the least frequent sign.
Compare to the critical values
What do you need to know for a critical value table for the sign test?
Whether it was a directional or non-directional hypothesis, the significance level, N (the number of participants excluding any =)
What is a two-tailed hypothesis?
Non-directional
What is a one-tailed hypothesis?
Directional
How do we know if the difference is significant for the sign test?
If S is less than or equal to the critical value, it’s significant
Why is statistical testing used in research?
Researchers use statistical tests to determine the likelihood that the effect/difference/relationship they have found occurred by chance or not
What is the hypothesis (not the null) called when it isn’t an experiment? i.e. looking for a relationship/correlation
‘Alternate’ - rather than saying experimental, these are still directional/non-directional
What is content analysis?
A form of indirect observation (observing artefacts rather than people)
What are examples of artefacts for a content analysis?
Books, magazines, TV programmes, films, diaries, songs, interviews, presentation, discussions
(can be already produced or produced specifically for the investigation)
What is the process of content analysis?
Collect data/material, read/watch/listen it to familiarise themselves with it, choose coding units, code using them (coding is tallying)
What statistical test do we use after a content analysis?
Chi-squared
What type of data does content analysis give us?
Quantitative
What type of data does thematic analysis give us?
Qualitative
Is thematic analysis subjective or objective?
Subjective
What is the process of thematic analysis?
Transcribe the material, read, get a feel for the themes that keep cropping up, these become the themes, select every quote for the themes
What are the 4 strengths of content analysis?
Ethical.
High ecological validity.
Reliable.
Flexible
What are the 3 strengths of thematic analysis?
Ethical.
High ecological validity.
Flexible
What are the limitations of content analysis?
Possible observer bias.
Cultural bias.
Sample may be biased.
No cause and effect
What are the limitations of thematic analysis?
Possible observer bias.
Cultural bias.
Unreliable.
Sample may be biased.
No cause and effect
What is reliability?
How consistent the findings are/measurement is. How much we can depend on it.
If it is measuring the same thing for the same person, each time it should produce the same results
What are the 2 ways of assessing reliability?
Test-retest and inter-observer reliability
What is the test-retest method?
Study repeated with the same participants to see if simialr results are obtained. It is necessary to consider the amount of time between each test, ideal is 2 weeks
What is inter-observer reliability?
Whether all the researchers involved are gathering consistent observations/ratings
How do we check if similar results are obtained in assessing reliability?
We compare them using a correlation, there must be a coefficient of 0.8 or above
What are some of the most reliable research methods?
Lab experiments, structured interviews, questionnaires
What are some of the least reliable research methods?
Natural experiments, case studies, unstructured interviews
What are some of the most valid research methods?
Field experiments (ecological), unstructured interviews, covert observations
What are some of the least valid research methods?
Lab experiments, structured interviews, questionnaires, overt observations
How can reliability be improved in general?
Standardisation.
Pilot studies
How can reliability be improved for experiments?
Standardisation.
Take an average
How can reliability be improved for questionnaires/psychological tests?
Careful wording
Closed questions
How can reliability be improved for interviews?
Standardisation.
Train interviewers.
Careful wording
How can reliability be improved for observational techniques (including content analysis)?
Operationalise.
Cover all possibilities.
Train observers
Which statistical test do we use for reliability or validity?
Spearman’s rho
What is validity?
Whether the findings are genuine/accurate/meaningful/truthful.
Whether the researcher is measuring what they claim to be measuring and if those findings can be generalised beyond the research setting
What are the 2 types of validity?
Internal and external
What are the 2 types of external validity?
Ecological and temporal
What is internal validity?
Whether we are measuring what we set out to measure inside the study, whether there are extraneous variables like demand characteristics and investigator effects
What is external validity?
Whether we can generalise the findings to outside of the research settings
What is ecological validity?
The extent to which the findings can be generalised to other settings
What is temporal validity?
The extent to which the findings can be obtained at other times or in other historical eras
What are the 2 ways of assessing validity?
Face and concurrent validity
What is face validity?
Getting an expert to look at the test, measure or scale used to confirm that it does measure what it claims to measure
What is concurrent validity?
(used for self-invented tests) Compare (via correlation) a test, measure or scale with an already established/recognised one by getting participants to do both tests.
Time period not needed, but often wait 2 weeks
How can validity be improved in general?
Single blind procedure.
Double blind procedure.
Randomisation
How can validity be improved for experiments?
Standardisation.
Single blind procedure.
Control group.
Use tasks people use in everyday situations
How can validity be improved for questionnaires/psychological tests?
Allow anonymity.
Allow self-completion.
Only include questions that measure what is supposed to be measured
How can validity be improved for interviews?
Unstructured to build rapport.
Train interviewers.
Only include questions that measure what is supposed to be measured
How can validity be improved for observational techniques?
Use overt.
Use naturalistic.
Use participant.
Operationalise.
Train observers
How can validity be improved for qualitative methods (case studies, unstructured interviews)?
Use direct quotes.
Triangulation (using several methods)
What are the 2 types of error?
Type 1 and Type 2
How can you tell what the risk of making a type 1 error?
It’s the same as the level of significance
What is a type 1 error called?
An error of optimists, false positive
What do we say has happened for a type 1 error?
The researcher rejected the null hypothesis and accepted the experimental/alternate hypothesis, and they were incorrect
Why do type 1 errors occur?
The researcher has been too lenient, the significance level was too easy to reach/leniant
What is a type 2 error called?
An error of pessimists, false negative
What do we say has happened for a type 2 error?
The researcher accepted the null hypothesis and rejected the experimental/alternate hypothesis, and they were incorrect
Why do type 2 errors occur?
The researcher has been too cautious, the significance level was too hard to reach/stringent
Why is the 5% significance level the generally accepted level?
It is the perfect balance between making a type 1 and type 2 error, it minimises the chance/risk of either, it’s the perfect compromise
What are the 3 conditions you need to meet in order to use a parametric test?
Data must be interval level.
Population data must be normally distributed.
Homogeneity of variance
What do you do if you can’t use a parametric test?
Use the test above it in the table (ordinal level)
Why are parametric tests preferred?
They are more powerful/robust as they are more likely to detect significance
What is homogeneity of variance?
The standard deviations must be similar/the same
