Research methods C Flashcards
Types of rationale (4)
From previous researches methodological problems
Considering different theories to explain past research
Replication
Unique theory based on general observations
Types of design (3)
Simple comparisons - one IV, two conditions
One way designs - one IV, three or more conditions
Factorial designs - more than one IV, multiple simultaneous experiments
Example of factorial design
Effect of rain (IV 1) and wind (IV 2) on perceived pleasantness (DV)
How many interactions and conditions are there with two factors
One interactions and 4 conditions
How many interactions are there with 3 factors
4 interactions, one being all of them together
What does an ANOVA do
Is an analysis of the variance (SD^2), it determines if two or more groups are from the same population of scores
What do ANOVAs compare… and if they are similar?
The within group error variance to the between group error variance
… if they are similar then the groups are from the same population
When might ANOVAs be significant
If the between group variance is substantially larger than the within group variance
Why do within group designs tend to be more sensitive to ANOVAs than between group designs
Has smaller error variance so more likely for the between group variance (caused by IV) to be significantly larger than within group variance
(assuming minimal carry-over effects)
Sources of variance (2)
Error - different people producing different scores
Effect of variables / different conditions
F-ratio =…
Between group variance / within group variance
General rule for F-ratios showing significance
If they Re greater than 1, most likely significant
How to read F-ratios
The larger the more significantly the between group variance is larger than the within group variance
What a significant ANOVA tells us and what it doesnt
At least one group is significantly different from at least one other
Doesn’t tell us which groups
How to determine which groups significantly differ after a significant ANOVA
Comparing means And SDs indicates which ones
But post-hoc comparisons to find simple effects does so statistically
Problem with post-hoc comparisons
Familywise error rates
How to carry out post hoc comparisons
Use the appropriate t-test to compare each separate pairs of conditions
Type 1 error
Concluded significant when it is not
Went to find wolf but there wasn’t one
Type II error
Conclude no significance when there is
Didn’t go to wolf but there was
Explain family wise errors
Chance of type I errors = p, if comparing 3 groups then 3 comparisons needed… what would usually be 1/20 (.05) is now 3/20 (.15), meaning significance threshold too high
P =.25 threshold if 5 comparisons (**though threshold would still say .05!!)
How to avoid family wise errors
Use bonferroni correction - divide .05 by number of comparisons to make new criterion for significance
parametric assumptions (3)
normally distributed
Homogeneity of variance
Interval or ratio data
If one is not met, a non parametric test is needed
Limitations of non parametric tests (2)
Provide limited information (no means or variance)
Greater chance of type II errors as less sensitive
How to report non parametric tests (3)
State that (and how) assumptions for parametric tests were violated Report medians and range information (maybe IQR) Report test statistics
How to rank data
Order scores and number them in size order. If more than one of the same number, calculate their mean rank
Non parametric equivalent to a one way independent ANOVA
Kruskal-Wallis test
Features of Kruskal-Wallis test (4)
Uses independent groups
Compares 3 or more groups (different participants each group)
Sample sizes do not have to be equal
Uses ranked data
How Kruskal-Wallis test works
Rank all scores across all groups
Determine if rank for one condition is systematically higher or lower than another condition
If so, then a real difference. If non significant then ranks from each sample will be mingled together
How to do a post-hoc comparison after a significant Kruskal-Wallis test
Do a Mann-Whitney U test (non parametric equivalent to independent t-test), using a Bonferroni correction
Non parametric equivalent to a one way repeated measures ANOVA
Friedman’s test
Features of Friedmans test (3)
Compares 3 or more related conditions
Uses rank differences
Uses repeated measures design (same participants for each condition)
Hypothesis for Friedman’s test
Ranks of differences between a pair of conditions are systematically higher / lower than the ranks of differences between another pair of conditions
How to do a post-hoc comparison after a significant Friedman’s test
Conduct a Wilcoxon test (non parametric equivalent of a repeated measures t-test) on each pair of conditions, using a bonferroni correction
What is a mixed ANOVA?
At least one factor is within-participants and one factor between participants
Example of a mixed ANOVA
Two groups in different conditions (e.g. self compassion and control intervention), being tested before and after intervention.
How many independent variables can an ANOVA have?
One
What is levels of treatment
Number of groups in one factor
How many 1) factors 2) levels of treatment and 3) conditions does a 2 x 3 ANOVA have?
1) 2
2) 2 and 3
3) 6
Sources of variance in a two factor ANOVA (4)
Error variance
Main effect 1
Main effect 2
Interaction of factors
How does it get exponentially more complicated with studies with more factors
Number of interactions increases exponentially
Number of interactions in ANOVA = …
2^k - k - 1
k = number of factors
How to determine interactions from a graph
If the lines of factors are parallel then there is no interaction. The less parallel, the greater the interaction. Though significance of interactions cannot be determined by eye
What is an interaction
Variation in scores not due to error variance or the main effects
How many 1) main effects, 2) interactions and 3) sources of variance does a 2 x 2 x 4 ANOVA have?
1) 3
2) 4
3) 8
What is a simple effect
Difference between two conditions of one factor in a single condition of the other factor
Example of a simple effect in ANOVA (laptop vs hand written) x (review vs no review of notes)
With participant reviewing notes, the laptop was significantly less effective than hand written
Maximum number of simple effects in a 2 x 2 ANOVA
4
What to consider regarding research ethics? (4)
Provide consent form
Confidentiality / Anonymity
Stress optional, no pressure to participate
Debrief before or after study
Underlying assumptions of qualitative research
Descriptions / interpretations that move away from the realism / objectivity of quantitative, focusing on experience and subjectivity
Theoretical approach (generally) of qualitative research
An inductive approach whereby theories are shaped after the data is looked at, rather than having preconceptions
Explain how qualitative research has high reflexivity
Researchers interpretations is more embedded into the outcome of results
Characteristics of qualitative research (4)
High realism (naturalistic setting)
Low reliability
Loose structure
High reflexivity
Ontology
Theories about the nature of reality
Epistemology
Theories about the nature of knowledge
Types of qualitative researcher (3)
Miner (digs for information)
Traveler (observing and following information)
Gardener (?)
Ontological assumptions that can underpin research (3)
Realism, critical realism and reletivism
Epistemological assumptions that can underpin research
Positivism (objective, measurable, inert)
Social constructionism (culture shapes reality, knowledge is active)
Contextualism (social world has valuable knowledge in a critical realist reality)
Reflexivity in research
How researchers involvement i the study influences and informs it
Methods for being reflexive in qualitative research (3)
Diaries, memos, supervisory discussions etc
Describe interviews
Contrived conversations with a purpose
Types of interview (4)
Structured (strict Q and A) Semi-structured (some flexibility) Loosely structured (largely flexible, with some aim) Unstructured (does not exist)
Features of a good interview question (4)
Not leading / influencing the answer
Encourages long, elaborated answers
Not ambiguous
Cross-checked with others and piloted
Ethical consideration when conducting qualitative research online
Some websites do not allow research involvement in their terms and conditions
How to be sensitive in qualitative research (3)
Do not make participant uncomfortable or offend them
Use distress protocols to respond well
Maintain boundaries of researcher and participant
How to be sensitive when reporting qualitative research (2)
Consider whether participants would / wound not benefit from seeing report
Consider social / political impact of report before submitting
Methods of transcription (2)
Post-modern - real words, no correction, include pauses.
Jeffersonian - textual devices of how speech is said
Reliability
The accuracy of replication / repetition of a study… consistency of results
How can reliability be reduced
Differing interpretations between researchers
Validity
It measures what it intends to
How can validity be reduced
Differing cultural interpretations
Lack of standardized questions
Generalisability
Applications to other contexts
How does quantitative research reduce generalisability
Focus on detail and controlling all variables
Measures of quality in quantitative research (3)
Reliability
Validity
Generalisability
Measures of quality in qualitative research (4)
Sensitivity
Commitment and rigor
Transparency
Impact
Examples of sensitivity in qualitative research (3)
Inductive approach
Considering participant and cultural perspectives
Sensitive to ethical issues
Commitment and rigor in qualitative research
Faithfulness in participants provided information, methodological competence
Transparency in qualitative research and how to achieve it
Documenting and demonstrating the path to interpretation
Use an audit trail outlining process in detail, explaining rationality
Types of impact of qualitative research (3)
Inform current practices
Advance theoretical approaches
Used for political benefit
Focus groups
Individuals (best from 3-6) discuss a topic so researchers further understand their knowledge / viewpoints
Advantages of focus groups (2)
Sensitive to points of consensus and disparity on the topic
Ideas are developed from listening to others
Limitations of focus groups (4)
Recruitment can be difficult, finding a time everyone is free
Interactions hard to guide, easily side-tracked
Extroverts can dominate, losing introvert perspective
Groups must share similarities
How to better guide focus group interactions, and benefits of doing so (3)
Prompt methods - use activities (such as videos or hypotheticals) as a catalyst for discussion, makes it less personal / more indirect and participant led
Story completion
Projective text where participants complete a story stem
Benefits of story completion tasks (3)
Useful for socially sensitive and ambiguous issues as can express views indirectly
Explore a range of assumptions at once
Sharing views implicitly means participants’ lack of introspective awareness can be overcome
Qualitative surveys
Predetermined yet open-ended questions given to participant for written answers
Benefits of qualitative surveys (4)
Less reliant on researchers skills
Participants can feel more freedom in response
Often easier to articulate in written form
Suites broad and specific topics of interest
Solicited diaries and example of use
Diary writing for researchers with predefined guidelines, e.g. for food consumption
Benefit and limitation of solicited diaries
Can be more stringent than other methods
Participants may not fill it out as asked, can lie… however technology makes lying and forgetting harder
Features of media data qualitative analysis
Ubiquitous, easily accessible
Highlights reoccurring messages in culture
Taps into our mediated lives
A focus on sampling strategy and justifications
Disadvantage of media data qualitative analysis
Cannot dig further than content displayed… unless on a forum
Thematic analysis
Process of identifying meaningful patterns in data
Benefits of thematic analysis (4)
Good for novice qualitative researchers, giving breadth
Researcher actively involved in generation of themes
Flexible in method, topic, theoretical approach, interpretation etc
Can disseminate to general population
Inductive analysis
Themes generated without predefining a research question, coding focusing on anything ‘noteworthy’
Deductive analysis
Themes relate to predefined theoretical positions, organizing data into patterns, coding with a focus of finding examples related to the theory / question
Theme
Recurrent idea / statement that generate a pattern that adds meaning to experience… these are actively constructed by researcher
Stages of thematic analysis (5)
Familiarization with the data, highlighting items
Initial coding, summarizing noteworthy items
Developing themes from the codes
Reviewing themes and organizing them into a model
Presenting results
Disadvantages of thematic analysis (4)
Often just focuses on the obvious
Difficult to identify inconsistencies in data
Ignores tone of voice and the non-verbal
Can be time intensive
Personal reflexivity
Researchers influence on research from own interpretation and dissemination (etc)
Epistemological reflexivity
Influence of researcher’s assumptions about the nature of knowledge
Critical language awareness (reflexivity)
Awareness of the power of words to carry a certain view / assumption… researcher must make all of this intentional
What method and analysis (2) to use when gathering experiential information
Use interviews
Analyse with IPA and TA
What method (3) and analysis (2) to use when gathering understanding and perceptions information
Use focus groups, interviews and online data
Analyse with TA and ground theory
What method (3) and analysis (3) to use when gathering information about peoples’ practices
Use interviews, focus groups and online data
Analyse with TA, ground theory and DA
What method (4) and analysis (2) to use when gathering information about influencing factors
Use interviews, focus groups, surveys, diaries
Analyse with TA, ground theory
What method (2) and analysis (2) to use when gathering information about how people are represented
Use media and online sources
Analyse with TA and pattern-based DA
What method (3) and analysis (2) to use when gathering information about how people construct something
Use focus groups, interviews and online data
Analyse with TA and pattern-based DA
What method (3) and analysis (2) to use when gathering information about peoples’ language practices
Naturalistic data, interviews and focus groups
Analyse with discursive psychology and conversation analysis
What method and analysis to use when gathering information about stories
Use interviews
Analyse with narrative analysis
Interpretive Phenomenological Analysis (IPA) features (6)
Inductive approach Open-ended questions to participants Dynamic interpretive endeavour Analyses in depth, creating thick data Uses small, purpose samples Investigates meaning of participant's experience
Underlying assumptions of Interpretive Phenomenological Analysis (IPA) (2)
Experience is an existential import, based on idea that humans are constantly searching for meaning
Assumes agency of individual
Elements of Interpretive Phenomenological Analysis (IPA) (3)
Phenomenology - examining life experience
Symbolic interactionism - the self emerges from the meaning of social interaction
Hermeneutics - interpretation of experiences / messages
Double hermaneutics
1) participants interpret their experience
2) researchers try to understand and interpret it
How IPA differs from most of psychology
Most are nomothetic, meaning making claims at a group level
IPA is idiographic, meaning it interprets phenomenon comprehension by a particular people / individual… more value to personal experience
Sampling process for IPA
Use a small sample, and consider richness of data to determine if more participants are needed… though time for analysis and availability of sample are also considered
Potential data collection methods for IPA (4)
Semi structured interviews (usually best) Email exchange Postal questionnaires Online forums (Q/A essential)
Thematic analysis differences compared to IPA (3)
Used across epistemological and ontological spectrum
More flexible in every regard
Larger sample used for TA (breadth), as not idiographic
Differences in IPA methodology compacted to TA (3)
Initial noting is between TA’s data familiarisation and coding
Coding, after noting, is done idiographically
Generally more themes, akin to sub themes in TA, as value of each experience is increased
Example of both TA and IPA being used in same study
Interviews about depression, use TA for the whole set (breadth) and IPA for those who have experienced depression for over 10 years (depth)
Limitations of IPA (4)
Over reliance on language, non-verbal unaccounted for
Inability of participants to articulate experience (double hermeneutics)
Not always interpretive / reflexive… poor research skills
Maybe not phenomenological enough
Discourses and examples (2)
Inter-related sets of text that bring an object into being (pockets of knowledge)
E.g.) dissemination, reception
Discourse analysis
Systematic study of texts to ascertain the constructive effects of discourse
How do discourses possess meaning
Through connection with other texts, no meaning without context… meaning ascribed by analyser
Features of discourse analysis (4)
Social constructionist view of social world
Reflexivity particularly essential (increased researcher role)
Goes beyond data, relating it to a theory
A linguistic approach, seeing how choice of words constructs the social object
Assumptions underlying discourse analysis (2)
Language is purposeful, having an impact on reality, e.g. The law is not impartial due to language choice
People use discursive practices, an identifiable choice of words that construct a certain social object
What ‘social object’ means for discourse analysis
Choice if description and associations that implicitly creates social idea… though this is fluid and ever changing
Identifying linguistic features in discourse analysis and example
Focus on grammatical and pragmatic features
E.g. In politics, change verbs into nouns for powerful ascertains
Interpretive repertoires (discourse analysis) and example
Used to construct alternative, often contradictory versions of events in different contexts
E.g. In public, scientists speak empirically but in private they describe facts as more fluid
Discursive psychology
DA considered too contrived, this is in a more naturalistic setting. How discursive resources are used to achieve social objectives
Foucauldian discourse analysis
Investigates what kind of objects are constructed and its implications to subjectivity, selfhood and power relations
Data sources that we can use DA
Almost any, including diaries, speeches, posters, interviews… as long as there is discourse
Limitations of discourse analysis (2)
Embedded in social constructionism ISO less flexible… more a feature than a limitation
Heavily reflexive, so easy for unskilled to conduct it poorly
Spectre of subjectivity
Acknowledging interwoven role of researcher and socio-historical context in the research process
Subjectivity is accepted in qualitative research
Are there such thing as emergent themes
No, though often assumed they spontaneously arise, they are actually generated by researchers interpretation
Quasi-experimental method
Groups allocated based on naturally occurring phenomena, yet still has control conditions (half-experimental)
When are post hoc tests unnecessary, and why
When a factor has two levels (2 x 2 ANOVA) since the only possible effect is between those two
