Research methods and statistics 1 (year one) Flashcards
What is a hard science?
a science that is objective and measurable e.g chemistry
What must a scientifically sound experiment consist of?
- operational definitions
- suitable sample size
- control
What is the empirical approach?
science, evidence-based
When did Wilhelm Wundt open the first psychology lab?
1879
What is introspection?
Paying attention to and analysing your own thought processes
What is the order of the scientific method?
- observation-theory-hypothesis-research-research data
What is a theory and what must it consist of?
- general principals for outlining or understanding
- must include empirical investigation, prediction, explanation
- must be falsifiable
Who promoted falsification?
- karl popper (1934)
What does a good theory consist of?
- testable hypothesis
- guiding research and organising empirical evidence
- be supported or refuted
What is a hypothesis?
- a theory based prediction
To be scientifically testable what must a hypothesis be?
- clearly defined
- non-circular
- deal with observable phenomena
What are examples of famous studies that are not scientifically sound?
Asch (1951), Zimbardo (1971)
What are some of the methodological flaws of the Stanford prison experiment?
- researcher bias, small sample size, not representative sample, most guards were not violent, worst guard based behaviour on “cool hand luke”
What do we need to infer causation?
- correlation/co-variation
- time-order relationship (cause has to come before effect)
- eliminate other possible causes
- What is the independent groups design?
Groups that are made up of different people
- What do independent groups look at?
difference in performance between subjects
What is an independent variable?
- IV = variable we manipulate
What is a dependent variable?
- DV = variable we measure
What are some advantages to independent groups?
- no fatigue or boredom
- ## no carry over learning
What is a natural groups design
- IV not manipulated as it is already naturally occuring
What does the within groups design measure?
- repeatedly measure the same people on the same DV
- controls for individual differences
- ppts may do all conditions at the same time or different times
What does power refer to?
- the probability that you will find a statistically significant difference when it actually exists
What is error variance?
- variation caused by individual differences
- reducing error variance makes a significant result more likely
What are advantages of within-subjects designs?
- individual differences not a problem
- more powerful
- fewer ppts
- more convenient
What are the 4 levels of data?
- nominal
- ordinal
- ratio
- interval
What is nominal data?
names, categories
What is ordinal data?
data is ordered (e.g on a likert scale)
What is interval data?
data points have a similar interval between them (e.g height)
What is ratio data?
same as interval, but x has a zero value
What is qualitative research?
no set hypothesis
explores opinions, experiences etc
just from asking people e.g interviews
What is quantitative research?
numerical data analysed with maths based methods
via surveys, tasks etc
Give examples for each data level
- Nominal examples male/ female, smoker/non-smoker - Ordinal examples shoe size, position in race, subjective opinion (likert scales) - Interval examples voltage, temperature - Ratio examples (CANNOT go below 0) Height, weight, test scores
What are descriptive statistics?
- describing data
- see what data “looks like”
- looks at central tendancy and measures of dispersion
- only tells us about our sample, not population
What are inferential statistics?
- use sample to make inferences about the population
- help us reach conclusions beyond our data
- What does the statistic you use depend on?
2. What are the two main types of descriptive stats?
- data level
- distribution of data
- measures of central tendancy
- measures of dispersion
- measures of central tendancy
What are measures of central tendancy?
- this is how “most” people behave
- measures : mean, median, mode
- What is the mode?
- What is the median?
- How do you calculate the median?
- What is the mean?
- most common value or score
- mostly used with nominal data
- most common value or score
- central value in a data set ordered from lowest to highest
- mostly used with ordinal level data/ skewed data
- central value in a data set ordered from lowest to highest
- add up the two scores in the middle and divide by
2
- add up the two scores in the middle and divide by
- add up all the scores and divide by the number of values
- What does a histogram show?
- What is normal distribution?
- What is skewed data?
- the frequency of the data
- scores average around the middle, very few extreme scores
- bell-shaped curve
- median
- scores average around the middle, very few extreme scores
- mean is not central
- extreme scores affect the mean
- not normally distribution
- mean is not central
- How do we get around skewed data?
2. What is the 5% trimmed mean?
- remove extreme results
- take off 5% of scores from each end
equation = %required x no. scores
100
What is variance and standard deviation?
- average distance of scores from the mean
What are the measures of dispersion?
- range, interquartile range, variance and standard deviation
What is the range?
- difference between largest and smallest score
What is the IQ range?
- difference between middle 50% of scores
How do we calculate the IQ range?
- want to find the range of the middle 50 % of scores (second and third quartile)
- % required x no. scores
100 - answer rounded gives how many scores to take from top and bottom, which you find the range from
What is variance?
- deviation of scores from the meaan
- subtract the mean from each score, then find the average
- add up and square all scores, take them away from sum of scores squared divided by n, divided by n-1
- n = how many values there are
What is the standard deviation?
- square root of the variance
What descriptive statistics should I consider for each data level?
Nominal = frequencies/ %/ mode
Ordinal = median (+range)
Interval/ ratio skewed = median (+range)
ND = mean (+/-SD)
What data is chi-square used for and what does it assess for?
nominal data level
- association between categorical variables
what is the equation for expected frequencies?
(values taken from observed frequencies) (row total)x(column total)
N
what are p values?
evaluate how well the data in your sample supports the null hypothesis
what do high and low p values mean?
o High p value : data are likely with a true null
o Low p value : data are unlikely with a true null
at what value is the p-value said to be significant?
below .05
what is the alpha level?
level at which we accept result to be significant
what is the effect size for 2x2?
phi
what do different phi values represent?
- Φ .1 = small
- Φ .3 = medium
- Φ .5 = large
what are type 1 and type 2 errors?
- Type 1 : rejection of true null hypothesis (false positive)
- Type 2 : accepting a null hypothesis (false negative)
what is a Bonferroni correction?
- Change the alpha level to prevent type 1 errors
o Divide alpha level by number of tests that will be conducted
what is p hacking and how is it done?
: method of manipulating data to achieve significant results
Multiple analysis
Omitting other info
Controlling for variables
Analyse part way through then gather more data until a significant result is found
Changing DV
what is the null hypothesis and alternative hypothesis?
- Null hypothesis = statement of no difference
o True until there is evidence against it - Alternative hypothesis = statement of difference or association
What are one-tailed and two-tailed hypothesis?
- One tailed hypotheses = state which direction the effect will be in (e.g those that subscribe to Zoella will be more likely to choose the unhealthy snack
- Two tailed hypotheses = no direction stated
what do box plots show?
show medians, ranges, IQ ranges, skewness etc
Range = upper adjacent value – lower adjacent value
Uneven whiskers = skewness
how do you find the IQ range from a box plot?
IQ Range = upper hinge – lower hinge (whole box)
What is positive/negative skew?
- Deviation from symmetry
- Show a big difference between means, medians, and mode
- Extreme scores affecting the mean
- Positively skewed : scores greater than the mean skewing
- Negatively skewed : scores lower than the mean skewing
what is leptokurtic/paltykurtic distribution?
- Refers to extent to which scores cluster at the tails of the distribution – changes pointiness
- Positive kurtosis : leptokurtic distribution
- Negative kurtosis : platykurtic
Flatter than normal
what is the boundary for skewness?
more than twice the standard error
what is indicated when there is no overlap between two confidence intervals?
: difference between parameters is significant
what are some disadvantages of the between-subjects design?
- Between subjects : independent, looks at performance between subjects/groups
Disadvantages
High sample size
Individual differences
what are confounding and situation variables?
- Confounding variable : extraneous variable that influences results
- Situation variables : variables in condition that could confound, e.g environment, temperature, time of day
what are expectancy effects?
- Expectancy effects : expecting an effect can cause that effect e.g expecting a substance rather than a placebo may lead to experiencing some of the effects
what are the three types of balancing/matching for between subjects?
- Balancing and matching techniques
1. Random allocation
2. Matched group design
3. Natural group design
explain random allocation design?
RANDOM ALLOCATION DESIGN
- Participants randomly assigned to groups
- Controls for participant variables
- Sample size should be larger
explain matched group design?
MATCHED GROUP DESIGN
- Matches participants based on a certain characteristic (sometimes DV)
explain the within-subjects design and its disadvantages
WITHIN SUBJECTS DESIGN
- Repeatedly measure the same people on the same DV
Disadvantages
o Boredom/ fatigue
o Order/practice effects
o Individual differences
o Time consuming conditions
o Can’t use for experiments where task cannot be repeated (e.g first impressions)
o Can’t be used if there’s differential transfer (effects of one condition affect performance in some conditions (e.g using cannabis then placebo)
what is differential transfer?
effects of one condition affect performance in some conditions (e.g using cannabis then placebo)
state and explain different order/practice effects
Learning
Fatigue
Habituation (leads to reduced response)
Sensitisation (leads to greater response)
Contrast (may lead to less effort if initially rewarded
Adaptation (e.g low light levels, drug effects)
what is incomplete within-subjects design
- Each condition given to each participant once
- Order of administration varied
- Practice effects balanced
what are the main counterbalancing methods for incomplete within subjects?
o All possible orders
o Selected orders
describe all possible orders counterbalancing
Have to calculate the factorial based on levels of IV (result of multiplying number by all numbers less than it)
Used on 3-4 conditions or less
describe selected orders counterbalancing
Based on Latin square
Used for more than 3 conditions
Each condition occurs once in each position
Each condition precedes/ follows each other condition only once
what are the main counterbalancing methods for complete within subjects?
- Each condition administers several times (different orders each time)
- Practice effects balanced for each participant
- 2 main counter balancing methods
o Block randomisation
o The ABBA design
describe block randomisation and ABBA design
- BLOCK RANDOMISATION
Consists of all conditions
Participants complete the conditions several times, each time in a different order - ABBA DESIGN
Presents one random sequence of conditions, then the opposite sequence
explain observation without intervention and advantages/diadvantages
o Naturalistic observation o Behaviour occurs naturally, experimenter is a passive recorder ADVANTAGES High external validity Can investigate complex social situations Useful for developing theories DISADVANTAGES Time consuming/ expensive Description, not causation Not useful for specific hypotheses
explain participant observation and advantages/disadvantages
- PARTICIPANT OBSERVATION o Undisguised • Researcher part of group • In depth interviews/ observations Advantages No ethical problems Natural setting Openly record data Disadvantages Behaviour may change due to presence o Disguised • Those observed are unaware • Prevents observer influence Advantages Access to particular groups Natural setting Disadvantages Ethical issues Problems recording data Researcher bias Interaction : researcher may change the observeds behaviour
explain structured observations
- Cause an event or set up a situation
- Observe specific behaviour in a particular setting
- No attempt to control for other variables
- Uses behavioural checklist or code using mutually exclusive categories
- Same procedures across other observers
explain field experiments
- Well controlled in natural setting
- Manipulate IV to observe effect on behaviour
explain interobserver reliability
- Consistency in measuring between observers
- Correlations can be used to check reliability
explain observer influence
- Reactivity : participant modifies behaviour when they know they are being observed
Socially normative behaviour to gain approval
Demand characteristics : change behaviour depending on what the expected objective of the research is - Controlling reactivity : unobtrusive measurement
Disguised participant observation
Adaptation : habituation, desensitisation
Indirect measurement : physical traces, archival data - Expectancy effects with observer bias : knowledge of hypothesis/ previous research
Can be controlled by blind observers
explain observation with intervention
Precipitate an uncommon/ difficult to observe event
Gain access to closed event/situation
Establish comparison by adding/manipulating IVs
Control antecedent events/ behaviour
Vary qualities of a stimulus event
3 kinds : participant observation, structured observation, field experiments
Define correlational analysis
- Assess relationships between variables
define correlation coefficients
- tells us about the strength of the association : range from -1 to +1
- negative values : negative correlation (-1 = positive)
- positive value : positive correlation (+1 = positive)
define positive and negative correlations
positive correlation
- as one variable increases so does the other
negative correlation
- as one variable increases the other decreases
what are other explanations for correlations?
- other explanations for correlations: 3rd variable, chance
what are the two main inferential tests for correlation data?
- spearman rank correlation
- ordinal level data
- skewed ratio/ interval level data- pearson product-moment correlation
- normally distributed interval/ratio level data
- pearson product-moment correlation
What data levels are spearman rank correlations used for?
- spearman rank correlation
- ordinal level data
- skewed ratio/ interval level data
what data levels are pearson product-moment correlations used for?
- pearson product-moment correlation
- normally distributed interval/ratio level data
explain how spearman’s correlations are determined
- uses the ranks of the data, not the actual data
- not influenced by skewed data
- when we have two values that would get the same rank we add together the ranks and divide by how many tied scores there are
describe how you would report results for spearman’s correlation
- shows a positive/negative/no correlation
- spearman’s : rs
- degrees of freedom = n-2
- order = rs (degrees of freedom) = correlation coefficient, p
explain one tailed and two tailed tests
- one-tailed : direction is stated
- we can halve the two-tailed p value to find one-tailed
- alpha values : .025
define alpha values
- alpha value : level at which effect is significant
- typically .05, so p values below .05 are significant
define degrees of freedom
- degrees of freedom : the number of observations in the data that are free to vary when estimating parameters
what is the symbol for pearsons?
-pearson’s symbol = r
what are correlation matrices?
- present lots of variables in a table - correlation matrix
- APA format
explain surveys
- predetermined questions
- includes questionnaires and structured interviews
- can be given online, mail etc
give advantages/disadvantages of mail surveys
- mail : + convenient
- response rate/bias
give advantages/disadvantages of internet surveys
- internet : + efficient/cheap \+ convenient \+ large/diverse sample - representativeness - ethics
give advantages/disadvantages of phone surveys
- phone : + some questions easier to ask
+ large, diverse sample
- sample bias
- interviewer bias
give advantages/disadvantages of group surveys
- group : + captive audience
+ large amount of data quickly
- privacy/anonymity
- pressure
give advantages/disadvantages of interview surveys
- interview + same questions/order
+ quantitative analysis
- interview bias/ social context
give advantages/disadvantages of personal surveys
- personally : + convenient/ large sample
+ good response rate
- representativeness/demand characteristics/ questionnaire fatigue
explain psychometric tests
- ability/ aptitude (e.g numerical/verbal reasoning
- personal qualities (personality/attitudes)
when were surveys first used?
- 605-1905 : Chinese civil service exams used to recruit officials
describe the army alpha and beta tests
- 1917 : army alpha and beta tests developed by Robert yerkes
- evaluated intellectual/ emotional functioning
- tested verbal/ numerical ability, (e.g following directions)
- also tested capability of serving, job classification, leadership potential
- beta test - non verbal equivalent
- allowed intelligence classification as superior, average, inferior
- highest to lowest score : white Americans, north/west European immigrants, south/ east European immigrants, black Americans - test was very amercio/eurocentric (e.g what is crisco, celebrities) and required cultural knowledge
- actually measured level of education / acculturation
describe the woodworth personal data sheet
- woodworth personal data sheet
- world war 1 by US army
- a test of emotional stability (susceptibility to shell shock)
- first personality test
describe the stanford-binet IQ test
- Stanford-binet IQ test
- used to assess for learning disabilities
- used today for clinical/ neurological assessment and educational placement
what is the procedure for designing a questionnaire?
- need a topic, then draft, then reexamine/ revise, then do pilot study, then edit and specify procedures for administering
what should questionnaire questions be?
- questions must be simple
- dont use double barralled questions
- avoid using loaded/ guiding questions
- avoid negative wording (e.g do you think students shouldnt pay tuition fees
describe open ended questions
- open ended questions
+ detailed answers
+/- quick to design, long analysis time
+ participant led
- subjective interpretation - partially open-ended (multiple answers given
describe close ended questions
- closed questions (e.g likert scales, true/false)
- guessing
- unsubtle
- complex to design, quick to mark
- theory led
- questionnaire fallacy : people will find a box to tick, even if their opinion is not represented
describe rating scales
- e.g yes or no, agree or disagree, graphic rating scale
- likert scale : labelled statements of a varying strength (e.g strongly agree to strongly disagree
- each measure given a score (positive question : strongly agree = 5, negatie question : strongly agree = 1 - semantic differential scale : connotative meaning between bipolar adjectives, and rating is placed on a scale inbetween
explain order effects in questionnaire bias
- order effects/ priming : detailed questions at first may influecne later general questions
- thinking about how the answer to one question while answering another
- counter balance questions and randomising can help
explain demand characteristics in questionnaire bias
- demand characteristics : answer in a certain way to sabotage/ give “beneficial” answers/ look more desirable
explain acquiescence in questionnaire bias
acquiescent : always agreeing/ disagreeing, even if it contradicts previous answers
- use a mix of positive/negative questions to overcome
explain extreme/ neutral responses in questionnaire bias
- extreme/ neutral responses : may not be concentrating, sabotaging etc
- raw data may need to be disregarded
explain cultural bias in questionnaire bias
cultural bias : language could be misunderstood, multiple interpretations of words, differing opinions between cultures, social desirability differs
explain attitude questionnaires
- assumptions : attitudes can be verbally expressed
- statements will have the same meaning for all participants
- attitudes can be quantified
- problems : consistency, social desirability, ambivalence, normative response bias (use a lie scale)
- implicity : do the statements express what they should clearly or not?
explain word association tests
- used in a clinical setting to determine complexes/ deficiencies used to predict things such as drug use
- advantages : quick/ easy to administer
- predict prospective drug use
- self scoring improves validity - disadvantages : colloquialism - cant make standardised procedures
- tests may not be implicit
explain implicit tests
- implicit cognitive tasks : infer attitude/ beliefs from performance on different tasks
- often use reaction times
- e.g IAT
- automatic association between concepts, used for attitudes towards age/gender/ race etc
- now computer based
- categorised target concepts with an attitude as quickly as possible
- faster association = stronger correlation - disadvantages : cultural values vs beliefs/ attitudes
- ecological validity
- may not act that way
explain reliability in psychometric tests
- reliability : internal (all items measure the same thing)
external (consistent across time and setting)
- test-retest reliability and split-half reliability
explain classical test analysis in psychometric tests
- classical test analysis : assumes observed score (X) is made up of true score (T) and random error score (E) : X=T+E
- random error : reading errors, social desirability bias, mood, tiredness
- systematic error : characteristic of test e.g “how often do you go to the cinema” would be influenced by factors like wealth
explain validity in psychometric tests
- validity : content validity (covers all behaviour/ aspects)
construct validity (measures theoretical construct)
criterion- orientated validity (correlates with establishes measure)
explain standardisation in psychometric tests
- standardisation: standardised instructions and procedures
explain established population norms in psychometric tests
established population norms : should be able to compare results to an appropriate established tests/theories
describe the two types of questionnaires
- knowledge based : ability, aptitude, achievement e.g intelligence tests, clinical assessment instruments
- person based : personality, mood, attitude to assess differences between people
describe the two types of response references
- normative reference testing : scores compared to norm e.g mean/ median split
- criterion reference testing : scores compared to pre-determined criteria e.g determine if someone is at risk
- restrictive : doesnt take in to account individual differences or non-clinical samples
define regression
- focuses on predicting variance in an outcome (criterion or response variable) from predictors (IV)
- creates a statistical model to find out whether model is a good fit for data and find whether there is a significant association/ direction of association
give the linear relationship formula
- linear relationship formula = Y = bX + a
- Y : criterion/response variable
- b: slope of the line (based on Pearson’s r)
- X: predictor variable (years of experience)
- a: constant or intercept
- calculates line of best fit for the observed data which can be used to make predictions for unobserved values
give the regression equation
- Yi = (B0 + B1Xi) + ei
explain bivariate linear regression
- two variables
- X is predictor variable (IV/explanatory variable)
- Y is criterion variable (response/outcome/criterion/DV)
give the assumptions of regression
- normally distributed continuous outcome
- independent data
- interval/ ratio predictors
- nominal predictors with two categories (dichotomous)
define R square
- R square/ adjusted R square
- how close data is to fitted regression line
- proportion of variance explained by the model
- presented as a percentage
- coefficient of determination
define ANOVA
- ANOVA
- measure of model fit : tells us how well regression fits the data
define beta coefficient
- beta coefficient
- number of SDs the criterion variable will change as a result of one SD change in the predictor variable
what do we need to interpret a regression?
- we need to :
- assess model fit (f value)
- know how effective model is - R squared value
- know whether an association is significant and direction - beta value
explain how to report regression data
- Example :
- a bivariate regression was conducted to investigate the association between years of experience and salary. The regression model predicted approximately 70% of variance in salary, adjusted R^2 = .70, F(1,8) = 22.34, P= .001.
There was a positive association between years of experience and salary 𝜷 = .86, p = .001
- a bivariate regression was conducted to investigate the association between years of experience and salary. The regression model predicted approximately 70% of variance in salary, adjusted R^2 = .70, F(1,8) = 22.34, P= .001.
Explain why multiple regressions are used
- can increase the amount of variance explained by a model by including additional variables
Define correlational analysis
- Assess relationships between variables
define correlation coefficients
- tells us about the strength of the association : range from -1 to +1
- negative values : negative correlation (-1 = positive)
- positive value : positive correlation (+1 = positive)
define positive and negative correlations
positive correlation
- as one variable increases so does the other
negative correlation
- as one variable increases the other decreases
what are other explanations for correlations?
- other explanations for correlations : 3rd variable, chance
what are the two main inferential tests for correlation data?
pearson product moment correlations
spearman rank
What data levels are spearman rank correlations used for?
- spearman rank correlation
- ordinal level data
- skewed ratio/ interval level data
what data levels are pearson product-moment correlations used for?
- pearson product-moment correlation
- normally distributed interval/ratio level data
explain how spearman’s correlations are determined
- uses the ranks of the data, not the actual data
- not influenced by skewed data
- when we have two values that would get the same rank we add together the ranks and divide by how many tied scores there are
describe how you would report results for spearman’s correlation
- shows a positive/negative/no correlation
- spearman’s : rs
- degrees of freedom = n-2
- order = rs (degrees of freedom) = correlation coefficient, p
explain one tailed and two tailed tests
- one-tailed : direction is stated
- we can halve the two-tailed p value to find one-tailed
- alpha values : .025
define alpha values
- alpha value : level at which effect is significant
- typically .05, so p values below .05 are significant
define degrees of freedom
- degrees of freedom : the number of observations in the data that are free to vary when estimating parameters
what is the symbol for pearsons?
pearson’s symbol = r
what are correlation matrices?
- present lots of variables in a table - correlation matrix
- APA format