Research Methods⚗️ Flashcards
BPS code of human research ethics in ethical reasoning
Respect, scientific integrity, social responsibility and minimise harm:
Assess potential risk benefit of research
Clarify conditions but should use judgement to avoid bad outcome
Consider choices and consequences and decide on course of action
Avoid being accountable for unexpected outcome or one you are unhappy with
BPS code of ethics-accountability
Respect autonomy, dignity of people
Submits application for ethical approval
Academics review
Identify problems and give feedback
Revise applications and re-submit
BPS code of ethics-research malpractice
Fabrication or deliberate manipulation of data
Plagiarism
Incorrect data processing
Dubious analysis practice
BPS-code of ethics and conduct
For people who practice psychology
Respect, competence, responsibility and integrity
Health and care professions council (HCPC)
Regulate psychologists, approve training, take action when fail to meet standards
Caution, set conditions, suspend and strike
Other ethical codes
UK government Universal ethical code for scientists
Sheffield uni ethics policy
Department of psychology ethics policy
BPS code of ethics-consent
Clear info to participant (aims, methods, data storage)
Scientific integrity
Risks and benefits
Social responsibility
Coercion
Research as ‘free therapy’ or financial gains
Free to withdraw without reason or consequence
Friends and family may feel coerced to take part
Info sheet and consent form
Anonymity and confidentiality
Anonymity - no info collected to identify participant. Anonymous data does not have to conform to GDPR
confidentiality- info could identify but is kept confidential. Must comply with GDPR
GDPR
general data protection regulation
Personal data allowing identification is protected
Participants must be informed cannot access, delete or transfer their data or withdraw after
How to work with non anonymous data
Fully informed of how and why data is processed
Ensure happy to use anonymised data
Qualitative data can be anonymised in interview transcripts (pseudonyms)
Handling data
Interviews recorded, should know how stored/who has access
Aware recordings will be deleted once been transcribed to be confidential
Storing data
Consent forms separate from participant lists and data
No participant numbers on consent forms
Store all data in password protected drive (University drive)
Accountability-protection of researcher
Qualitative work a physical risk
Carry phone and tell people your location
Risk of being challenged if sensitive topic
Less mistakes if follow ethics
Accountability-participant trust and debriefing
Trust- risk of exposure, trust researcher to respect privacy, recordings not available to those beyond the research
Debrief- still happy to use data, know who to contact to withdraw, should leave in same or better mood
Accountability- minimise harm
Frame interview questions, signpost to support
Never go outside expertise or offer counselling
Suggest withdrawal if distressed. Avoid making decision for them
Can withhold aims
Stages of a write up for qualitative and quantitative
Qualitative- intro, methodology, analysis, conclusion, discussion
Quantitative- intro, method, results, conclusion, discussion
Epistemology
What is knowledge, belief, truth
Determine what is fact
Epistemological perspectives
POSITIVISM- one reality uncovered through observation (quantitative)
POST POSITIVISM-some objectivity, research in a social context. What people say about their experience
SOCIAL CONSTRUCTIVISM-range of valid views
RELATIVISM -reality relative to history and culture (qualitative)
Reflexivity
Researcher reflects on own attitudes, values and experiences
Social and political context
Qualitative research
INDUCTIVE, start with observations and form theory
Find general themes, no single answer
Focus on context (words) make predictions
Literature can inform
Subjective representations of reality, humans have viewpoints and should acknowledge reflexivity
Good research question
Match question to relativist approach, extract unique meaning of the world
Question guides the method and reveals epistemological position
Lend to qualitative methodology e.g. focus group
Well constructed, answerable and specific
Existing literature as context, reviews help define the question
What does the choice of research question influence
The method you use Epistemological position (usually, relativist, positive)
Interviews
Exploratory, acknowledge diversity of experience
Skilled interviewer brings out topics
Allow flexibility but follow topics
Follow up questions and probes to explore for long detailed answers
Meaningful interesting and answerable
Display question for focus group, order of questions and wording is important
Draft, pilot and revise
Role of interviewer
Questions not neutral, interviewer is integral to data and influences it
Data is coproduced between interviewer and participant
Interview prep and practicalities
Consider length, amount of participants
Run a pilot and practice
Prepare interview schedule
Quiet room, safety and confidentiality
Check equipment, toilets, fire exits etc
Introduce purpose of research, relaxed and friendly
Check consent
Learn questions off by heart if possible
Active listening, respond and end well
Iterative process of research question in qualitative
Identify what you need to know
Derive research questions from literature without answering question that has been answered
Question may become more focused or develop different direction from literature reviews
Grounded theory-can delay literature review until after data collection
When do literature reviews happen
Development of research question
During planning stages (rationale)
End of project (literature not considered, check for relevant studies not published since)
What are literature reviews used for
Find what has already been done
Understand kinds of questions people in topic have asked
Get to grips with issues relevant to area of interest
Construct account about research
Funnel from bigger general issues to sub themes and areas
Transcription types
Orthographic- verbatim postscript
Non orthographic-paralinguistics, extra linguistics Jeff
PLAYSCRIPT AND JEFFERSONIAN (Systematic representation of language in written form)
playscript
Orthographic
Hesitations, false starts
Used for analysis of the meaning of speech such as thematic or phenomenological analysis
Jeffersonian
Non orthographic
Reflects interview as social interaction, more systematic
Used for conversation analysis, time consuming and may be less widely applicable
Ontology and epistemology
Ontology- is there a reality or not(realism, relativism)
Epistemology- how do we know if there is a reality (positivism, social constructionism)
Critical realism
Perception of reality comes from within and without
Biological mechanisms drive perception common to all living things, people agree on some truths
Thematic analysis
Extract common topics and themes
Use of quotes to illustrate key aspects in wider themes
Themes-Coherent and meaningful concepts in data with minimal overlap allowing categorisation
Creating themes process
Familiarise with data (re-read, initial observation)
Generate initial codes (short version)
Search for higher order themes
Reviewing themes (overlap, reorder, check themes in relation to codes and define nature of each theme and relationship between in thematic map)
Define and name themes (how fits overall data)
Write up (consolidate narrative, select extracts and show how reflects end product and context of literature)
Common errrors of thematic analysis
Extracts with no analysis, must be interpreted
Using interview questions as themes or to describe responses
Weak analysis, themes not coherent or reflect data
Mismatch between data and analysis
Must make theoretical position clear
Data and theory driven thematic analysis
Data- avoid starting with predetermined theory, be open minded but still have some reflexivity
Theory- apply or test theory, establish holistic sense of experiences, key points and how interviewer shapes data
Sample variation
Any sample from population will vary
Can be measured with SD (unsystematic variation)
Wide error bars
More unsystematic variation, difficult to find effects based on conditions in the study
Standard error
Estimate of deviation between sample and population mean
Errors can occur when sample taken from wrong population (sampling error) -measure of unsystematic variation
95% CI error bars estimate the effects we should find in a population
Error and effect
If more error (unsystematic variation) than effect (systematic variation)
Unlikely to find an effect in the population or is hidden by variation
Not significant effect
Effect due to differences in sample, not experimental manipulation
Standard error positive value and negative
Positive value- more effect than error
P value
How likely to see effect in sample of no effect in population (null)
If smaller than 5% then is significant, effect is likely to exist in the population
Inferential statistics
Generalise from a small sample to population (not necessarily qualitative)
Have not tested whole population so don’t know population mean and standard deviation
Small samples will not be representative of population/unsystematic
Effect size
Measure of how important the difference/effect is that was found in the sample
Partial eta squared
Np2 shows total variance explained by treatment effect (systematic variation)
value as a proportion
The differences between scores attributed to experimental manipulation, the rest is due to other factors
1 way anovas
(Like a t test) with more than 2 conditions
To test whether there is a difference between the levels of the factor
If significant- there are differences, can explore data in post hoc tests
Anova IVs conditions names
IVs- factors
Conditions-levels
Equation F ratio
Systematic and unsystematic variance (the EFFECT, between groups variance) ÷
Unsystematic variance alone (the ERROR, within groups variance)
Removes the effect =giving an F RATIO
Treatment effect
Comprises of the effect (difference from manipulation)
But also error (unsystematic variation)
The F ratio results and what they mean
If F greater than 1- more effect than error
If F smaller than 1- more error than effect
Rounding up numbers
All values rounded to two decimal places apart from p value
What does it mean if anova is significant
There are differences between groups and was an effect of treatment
-now need to find out which groups differ by doing separate t tests on the groups (post hoc tests)
Writing up an independent anova
A one way between participants anova shows there was a significant effect of… F (effect df, error df)= [F value] , p [ p value ], np2 [np2 value ]
The effect demonstrates that [np2 value as %] of the variance in errors can be attributed to…
This suggests that…had an effect on…
Post hoc tests
To see where there are differences between the levels (omnibus test)
1 way independent ANOVA- independent t test
1 way repeated measures ANOVA- paired t test
Post hoc t tests and bonferroni correction
Establish whether significant difference between levels
Alpha cut off at 5% prevents type 1 error but needs to be Bonferroni corrected for the levels
0.05 ÷ amount of levels to keep it at 5%
Reporting post hoc test
Bonferroni corrected post hoc tests revealed that…
(mean, SD) compared to (mean, SD) ; t (df)= [t value] , p = [p value]
These results suggest…
(Use sig 2 tailed )
Maunchley’s test of sphericity
Want to be not significant- assumption of equal variance so can do a post hoc test
IF SIGNIFICANT- Read GREENHOUSE GEISSER for the dfs and the significance (p)
If Greenhouse geisser still not significant then cannot carry out post hoc tests
If Maunchley significant and has outliers, normal distribution-NON PARAMETRIC
ONLY FOR REPEATED MEASURES
Non parametric anova
When violate expectations of a parametric ANOVA
If LEVENES/MAUNCHLEYS TEST IS SIGNIFICANT=non parametric
Rank scores not raw scores, manages variations of distributions, outliers and sphericity
Kruskall Wallis-non parametric
Between participants
H (df)=H value, p=p value
Mann Whitney post hoc U value, p value
Use medians
Reporting Kruskall Wallis
As the data were not normally distributed, a Kruskall-Wallis one way ANOVA was performed for…
The results showed significant difference between…
H (df)= (H value), p (asymp sig)
Bonferroni corrected post hoc Mann-Whitney U tests revealed more.. (median) than…(median); U (u value), p (sig) and more..than…
This suggests
Friedman test-non parametric
Within participants
x2 (df)=chi square, p=p value
Wilcoxon post hoc Z value, p value
Use medians
Two way Factorial ANOVA
More than one factor (IVs)
Differences between more than 2 conditions or groups
Tests the interaction between the factors, tells us whether impact of IV is the same regardless of other IV or whether DV varies as a result of an IV
What does Factorial anova produce for each factor mean
MAIN EFFECT for each factor-Marginal mean (based on means for each type in that IV)
INTERACTION between factors (IVs)
If interaction, lines will be crossed over on the graph
Factorial ANOVA how to write up main effects of IVs and the interaction between the IVs
F (effect df, error df)= F value, p value, np2 value
Look to significance to see if main effects are significant
Following up a significant interaction
Factorial anova
Bonferroni corrected t tests (simple effects test)
Run independent t test to test differences between the levels of 1 factor (learning style) in each level of the other factor
Run on conditions involved in interactions e.g.2 levels in each factor (4 simple effects tests)
Reporting factorial ANOVA simple tests
Bonferroni corrected simple effects tests showed that visual style outperformed [auditory] (mean, SD) when [visualisation was used] t (df)=t value, p=p value
[auditory style using auditory] performed better (mean, SD) than those with[ auditory and visualisation] (mean,SD); t (df)=t value, p=p value
THEN SWITCH THEM ROUND
Cohen’s d
For tests with 2 comparisons
Measure of the overlap between 2 means in standard deviations
High overlap in graph =small Cohen’s d and small effect size
(The magnitude of the F ratio)
Why does 1 way repeated measures ANOVA not take individual differences into account?
There is only one group of participants, sampling error is included but individual differences between participants are not included as unsystematic variance
What does a main effect of the factors mean without an interaction?
No impact of either factor on the DV
and no impact of factor 1 compared to 2 on DV
What does an interaction mean
Effect of Factors on DV when they interact together
Assumptions of non parametric anova
Continuous DV
Normal distribution (but is okay if there are equal sample sizes)
No outliers
Equal variance
Non parametric omnibus test
Non parametric t tests carried out if p value is significant
Bonferroni corrected
Stainar Kvale’s criteria
Knowledgable, clear, structured, gentle, sensitive, open, steering, critical, remembers, interprets
Graphing non parametric data
Error bars with mean and standard error are not appropriate
Use bar graph based on median without error bars
LEVENES test
If not significant then do non parametric (Friedman)
Deductive and inductive
Deductive- theory driven
Inductive-data driven
How ANOVA assesses impact of experimental conditions
Size of error (unsystematic variance) in relation to effect (systematic variation)=F ratio
Partial eta square indicates proportion of variance
Assumptions of a 1 way anova
Continuous DV No outliers (bar and whisker) Normal distribution (Histograms) Equal variance in each group (levene’s test INDEPENDENT Maunchley’s REPEATED MEASURES) if non significant then is no significant differences between error variances and assumptions)
Post hoc tests for parametric and non parametric
Parametric- M, SD
Non parametric- Mdn
Non parametric anova and the tests and omnibus tests used
Between participant-Mann Whitney U- Kruskall Wallis H
Within participant-wilcoxon Z signed rank -Friedman test X2
Simple effects tests for factorial Anova
Bonferroni corrected t tests test differences between levels of 1 factor in each level of the other factor
What assumption can be violated but still have parametric anova
Can still do parametric anova IF sample sizes are equal
Types of one to one interviews and questions
Structured
Semi structured
Unstructured
Closed Open Leading Non leading Probes Main questions (open, non leading and descriptive)
Coding
Generate short relevant labels
Semantic and conceptual understanding
Codes must be specific and work independently of the data, must successfully evoke relevant features of the data
Writing up narrative
Integral part, weave together coherent story
Contextualise analysis in relation to existing literature
Interpreting 95% confidence interval graph
Between-summaries for groups of cases
Within-summaries for separate variables
If confidence intervals overlap: any effects in sample are unlikely to be found in the population
Longer bars- wider range of results, also shows the trends
