Research methods and statistics 2 (year one) Flashcards
What are some advantages of qualitative research?
■ Reduce, reuse, recycle ■ Spontaneity ■ Exploratory, theory formation ■ Systematic and transparent ■ Flexible and open to change ■ Simultaneous data collection and analysis
Explain positivism
- Realist perspective
- Causal knowledge
- Deductive reasoning
- what qualitiatve research is built on
explain post-positivism
■ Criticisms of quantitative methods. – Bias in experimentation and data analysis – Reductionism and determinism – Data do not ‘speak for themselves’ ■ Lived experience and context ■ Words over numbers ■ Inductive reasoning
explain phenomenology and give advantages/ disadvantages
Captures the individual’s interpretation of a particular phenomenon.
Interpretive approach remains close to the raw data.
Can miss important precursors, consequences and factors associated with the phenomenon itself.
explain ethnography and give advantages/ disadvantages
- Examines characteristics that define us as being part of a particular cultural group, and how members of that group ascribe meaning to everyday life.
- Unlike phenomenology, ethnography captures the wider perspective.
- Does not capture the dynamic interactions between individuals within a group.
Explain grounded theory and give advantages/ disadvantages
Generates inductive theory that is fundamentally grounded in the data.
Bridges principles of quantitative and qualitative methods.
Fails to acknowledge the researcher’s role in constructing and interpreting data.
explain unobtrusive data
- generally, data which exist before research begins
e. g., Physical traces, archival data
define obtrusive data
Obtrusive: generated for the current research
e.g., Focus groups
- Field study or ethnography
Qualitative interviews
Define and explain qualitative interviewing
■ One to one semi-structured interviews are the most common form of qualitative data collection
■ A focussed conversation between a researcher and (usually) a single participant, where the researcher has a set agenda regarding issues to be discussed
■ Researcher has a role in encouraging participants to provide rich detail
■ More personal form of research than questionnaires
Explain why qualitative interviewing is useful
■ Generate rich and detailed data
■ Particularly useful for getting the story behind people’s experiences– the personal meaning and value
■ Arguably, easier for respondent than questionnaires, Mutually beneficial? e.g., Bereavement research (Bennett, 2005)
■ Chance for the researcher to develop rapport
■ Flexible, interactive and gives participant control over what they say (rather than researcher choosing IV and DV.)
Give some disadvantages of qualitative interviewing
■ Time consuming and effortful
■ Demands on interviewees time can make recruitment difficult
■ Relies on self-report– accurate information?
■ Cannot be generalised to the wider population
■ Interviewer effect/ bias
Give some solutions/ positives to criticisms of qualitative interviews
■ Time consuming and effortful : rich data
■ Demands on interviewees time can make recruitment difficult : typically small sample
■ Relies on self-report– accurate information? : participants truth often most important
■ Cannot be generalised to the wider population : but not the aim
■ Interviewer effect/ bias : but can consider this when planning and reflexivity also a core feature
Give some steps for planning a qualitative interview
■ Decide on your topic of investigation ■ Consider and account for ethical issues ■ Identify your target group ■ What resources do you need? ■ Decide on the format of your interview ■ Design your interview guide/schedule ■ Prepare by piloting the interview guide ■ When ready, start collecting data
Give some ethical considerations for qualitative interview research
- Researcher - participant relationship
- Data interpretation (e.g., risk of misappropriating data)
- Legal requirements of disclosure
- Data management e.g., storage, participant anonymity
- Potential harm e.g., emotionally difficult topics, vulnerability
Describe some ways of collecting qualitative interview data
specific responses
Unstructured: Very open and interactive, with questions created around a broad theme as interview proceeds
In-depth, semi-structured: Researcher guides discussion using an interview guide or ‘schedule’ containing a series of flexible questions and prompts
Describe the type of sampling used in qualitative research
■ Typically a smaller sample
– As many people as you need to find out what you need to know
■ Recruitment can be challenging
■ Sampling is important but varies depending on methodology
– e.g., Theoretical sampling in Grounded Theory
– Homogeneity in Interpretive Phenomenological Analysis (IPA), whereas maximum variation and use of extreme or ‘deviant’ cases in Grounded Theory
Describe what makes an effective interview question
■ Effective interview questions are open ended; neutral; sensitive; and easy to understand (Britten, 1999)
■ Better to avoid questions that are closed; biased or leading; confrontational; overly complicated or long; unrelated or nosey
what are some useful characteristics of an interview guide?
- Prompts can work really well
– e.g. What was that like for you? Could you tell me more about that? What do you mean by ___? Could you give me an example? How so?
■ Embed questions into longer sentences and encourage stories
Give some key ways of developing an interviewer-interviewee dynamic
■ A key ingredient to getting rich data
■ Developing interview skills through practice
– Value of conducting a pilot interview to get a sense of how your interview guide works and to examine your role as a researcher
■ Break the ice and develop rapport – first few minutes are crucial
■ Be yourself, but don’t give your views
■ Probe, but don’t cross examine. Draw out, but don’t be overbearing
■ Be clear about your purpose, how the information will be used etc.
Define reflexivity
■ “The process of a continual internal dialogue and critical self-evaluation of the researcher’s positionality as well as active acknowledgement and explicit recognition that this position may affect the research process and outcome” (Berger, 2015, p.220).
■ About acknowledging how aspects of our identities (such as race, class, and gender), histories etc. and how they affect data, analyses, and conclusions
■ Good practice to keep a reflexive journal, logging reflections on potential influence as a researcher
– Sensitises the interviewer to their prejudices, assumptions and subjectivities
What is a focus group?
- A group of individuals selected and assembled by researchers to discuss and comment on, from personal experience, the topic that is the subject of the research” (Powell et al, 1996: p,499)
- Facilitated by researcher (often helped by another person)
- Getting people to think about, discuss/debate an issue (or set of issues) – related to the research question
- Key is group interaction not just many interviews happening at once.
Explain when focus groups are used
- To access attitudes, feeling, beliefs and experiences of a group rather than just individuals (interviews)
- To generate discussion/debate about your research question/ area of research
- When you think that individuals may not be able to provide adequate responses on your research question
- When people may not feel ‘safe’ talking in individual interviews
Explain interpretivism
- social reality can only be understood through social constructions such as language, consciousness and shared meanings.
- Does not predefine variables, but explores human sense-making in naturalistic settings.
Give some benefits of focus groups
- Annecdotes
- Room for discussion
- Allows challenging of view points
Give some of the limitations of focus groups
- Can be conflict
- Hard to maintain anonymity
- Hard to control
- Group may not be fully representative : not everyone in group may equally contribute
- Difficult to arrange
What is the ideal number for a focus group?
- 6 and 10
Describe and explain photovoice methods
- Originally developed by Wang & Burris (1997)
- Qualitative method used in community-based participatory research
- It uses interviews and group discussions, and photographs that participants are invited to take
- Participants use photography to document their experiences and ‘tell stories’ about the photographs which identify and represent issues of importance to them
Give the three types of data collection methods for photovoice
- Focus group discussions
- Semi-structured interviews
- photographs
Explain Freire’s approach to critical consciousness
Visual image is an important instrument for:
) Becoming conscious of the contradictions that govern the world
Advocating for positive social change
The production of knowledge is in the hands of participants, who determine what to represent in a photograph
Through collective discussions, people share the meanings of the photographs they have taken
Describe the phases of a photovoice study
Phase 1: Photographic training and initial focus group discussion
Phase 2: Taking the photographs
Phase 3: Follow-up interview
Phase 4: Second focus group discussion
Phase 5: Summarising the participants’ narratives of the key photographs
Phase 6: Disseminate the findings (Photo-exhibition
what issues can be explored through photovoice?
Feelings ( social inclusion, loneliness in the community…)
Experiences (of living with mental health issues, of being a homeless, of living with chronic pain, diabetes, with intellectual disabilities…)
Features of the environment (e.g. how the local environment influences people’s diets, physical activity, access to green spaces
Give some benefits of using photovoice
- Encouraged engagement between participants and stakeholders
- Photovoice invited participants to successfully engage with the photographs, with the collective discussions offering an opportunity to critically think as a group about strengths and issues of their community
- Creative tool to reach key stakeholders (e.g. policy makers)
Explain some challenges of photovoice methods
Ethical aspects related to taking photographs
Important to ask written permission before taking photos of individuals
Some people may not want their photograph taken, and will have individual reasons for this. People often feel protective of their communities
Photo ownership: asking written permission to use participants’ photos in dissemination of results
Factors that can prevent participants taking photographs that they want to take
Photographing negative social concepts (e.g. social isolation) in comparison with more ‘tangible’ aspects (e.g. rubbish in the street)
Define and explain ethnography
- the study of culture used by a range of disciplines including: sociology, anthropology, psychology, applied educational research, organisational studies, human geography, programme evaluation, and business studies (Hammersley & Atkinson, 2007).
- The defining characteristic of Ethnography is participant observation, with the researcher participating / observing the daily lives and events being studied over an extended period of time (Hammersley & Atkinson, 2003).
Other characteristics of ethnographic research include:
Using multiple methods of data collection
Placing emphasis on context
Focusing on what people do, as well as what they say they do (Savage, 2000)
When should ethnography be used?
- When aiming to: “describe how cultural groups work and explore the beliefs, language, behaviours and issues such as power, resistance and dominance” (Creswell, 2007, p. 70).
- Useful when developing an understanding of complex relationships from multiple perspectives (Kendall et al., 2009).
Explain the method for ethnography and what it its reliant on
- Ethnographic research has flexibility built within its aims (Agar, 1996; Fox, 2004; Hammersley & Atkinson, 2003).
- Instead of planning precisely what will be done at each stage, it responds to the research findings as they unfold and narrows its focus though on-going analysis
- While this flexibility allows for the research to develop in light of ongoing analysis it also holds the danger of methodological arbitrariness that is not always easily justifiable to those outside of the project (Flick, 2009b).
- Very reliant on:
Gatekeepers
Snowballing
Networking
Explain multi-sited ethnography
The ‘object’ being followed can consist of The people an artefact a metaphor a story a biography a conflict (Marcus, 1998).
It is this conceptual link that differentiates MSE from comparative studies (Hannerz, 2003).
Give the types of data collection for ethnography
Observation/ field notes Photos Documents Informal interviews Formal interviews
Give some benefits of ethnography
- It draws on the strengths of each of the types of data collection utilised while in turn compensating for some of their weaknesses
- It is possible to see the interlinking between different aspects of the culture under study: looking at the wider processes, relationships, connections and interdependency of the actors in the setting rather than looking at one person’s perspective in isolation
- The researcher can establish rapport and trust with the participants, which in turn can facilitate honest, uninhibited discussion
Give some of the challenges of ethnography
Time consuming:
Observations
Field notes
Quantity of data
Give some challenges of ethnography
- More subjective than other Qual methods
- Greater potential to invite ethical issues associated with invasion of privacy and informed consent
- One of the main criticisms of traditional ethnography at a more theoretical level is that it depicts the people it studies as being in a void, uninfluenced by time and broader social constructs
Name the types of qualitative analysis
Thematic analysis, Grounded theory, Interpretative Phenomenological Analysis (IPA) Framework analysis Narrative Analysis Content Analysis Discourse analysis Conversation analysis.
Explain t-tests
- One of the most widely used stats tests.
- There are generally two types of t-tests. Those used for within-subjects designs and those used for between-subject designs.
- Sometimes go under different names : paired/unpaired, dependent/independent samples, related/unrelated samples
Give the t-test for within and between subjects
Within subject : paired samples, dependent samples, related samples
Between subjects : unpaired samples, independent samples, unrelated samples
Explain when t-tests are used
- T-tests are used when we have two different conditions or groups we want to compare.
- Within-subject t-tests used for within subject designs
- Between-subject t-tests used for between subject designs.
- Data must be Interval or Ratio AND be normally distributed.
- Between-subject t-tests have an additional assumption: homogeneity of variance.
Give the equation for paired samples t-test
t= mean of difference scores
SD/√𝑁
Describe effect sizes for t-tests
The effect size for t-tests is called ‘Cohens D’.
As a guide a d value of 0.2 is a small effect, 0.5 a medium effect size and 0.8 and greater suggests a large effect size.
SPSS does not provide Cohen’s D.
We need to calculate this by hand
Give the equation for cohen’s D
𝑑= 𝑚𝑒𝑎𝑛 difference
SD 𝑜𝑓 𝑚𝑒𝑎𝑛 𝑑𝑖𝑓𝑓𝑒𝑟𝑒nce
Give a template for writing up paired samples t-tests
• A paired samples t-test was conducted to assess the difference in reaction time between congruent and incongruent conditions. There was a significant difference between congruent and incongruent conditions, t (11)= 11.06, p
Describe and explain homogeneity of variance
■ One additional assumption of independent samples t-test is homogeneity of variance.
■ Basically, to have homogeneity of variance, each of our conditions should have similar variances.
Explain how we find homogeneity of variance
- We use a statistical test called ‘Levene’s test’ . It tests the null hypothesis that the population variances are equal.
- So if our Levene’s test is significant that means we do not have homogeneity of variance and so the assumption is violated and this is a problem.
- You do not report Levene’s tests in a report. You just report the results of the t-test.
- Leads to an increased likelihood we will find a significant effect when there isn’t one (Type 1 error).
Explain the welch test
- Luckily there’s an easy fix if we don’t have homogeneity of variance.
- When SPSS calculates an independent samples t-test it conducts two t-tests. One t-test is for when ‘equal variances are assumed’.
- This is used when you have homogeneity of variance.
- The other t-test is called a Welch test. This is used when you do not have homogeneity of variances when ‘equal variances are not assumed’.
- Some statisticians suggest we should always report Welch test regardless of homogeneity of variance
Give the equation for cohen’s d
D = t (square root of n1+2 over n1n2
Give an example of writing up an independent samples t-test
An independent samples t-test was conducted to assess the difference in reaction time between congruent and incongruent conditions. There was a significant difference between congruent and incongruent conditions, t (28)= 3.07, p=.01, d= 0.43, with those within the congruent condition having significantly lower reaction times (8.75±1.77) than those within the incongruent condition (10.86±1.99).
Explain when the sign test is used
- Looks at differences between two related (within-subject) samples of scores.
- Can be used when you have ordinal data or when data are interval/ratio but are non-parametric.
- Not typically used (Wilcoxon is better) but is a simple non-parametric test for you to begin understanding the concepts
Explain how the sign test works and some advantages and disadvantages
- The sign test looks at the amount of positive differences and compares this to the amount of negative differences.
- It takes into account the sign of the difference not the magnitude of the difference.
- This makes it a pretty poor test
- Although it can be useful for very small sample sizes (6 or less)
Explain the Wilcoxon signed ranks test
- Wilcoxon signed ranks test looks at differences between two related samples. But it is based on ranking the data rather than just the sign of the difference.
- Used when data is ordinal or interval/ratio but is non parametric. Can only be used when there are 2 within-subject conditions.
Explain how to rank
- Scores are ordered from lowest to highest.
- The lowest value is given a rank of 1, the second lowest a rank of 2 etc. etc.
- Sometimes though there may be values that are the same. In this instance what the ranks would be (if the numbers were different) are added together and divided by how many values are the same. These values all get the same rank. If you have to do it the next value (which is different) is ranked as normal. (See the example on the last slide).
Describe how to interpret effect sizes
- r= .10 small
- r= .30 medium
- r= .50 large
Give an example of a Wilcoxon signed rank write up
A Wilcoxon signed rank test revealed there to be a significant difference in the number of words recalled between the control (no mnemonics) condition and the experimental (mnemonics) condition, z=-2.27, p=.02, r=.36, with improved performance when mnemonics were used (Mdn= 9, range=3) than when they were not used (Mdn=7, range=6).
Explain when to use Mann Whitney U
- Can be used when data is ordinal or when interval/ratio data is non-parametric.
- Looks at the difference between two un-related (between subjects) samples of scores.
- Like the Wilcoxon test the Mann-Whitney U test is based on differences (in this case total ) ranks of the scores.
Explain how mann-whitney U is calculated
- These scores are ranked as if they’re one group but are kept separate so that total ranks for each group can be calculated.
- The smallest total is then used to ascertain significance
Give an example mann-whitney U write up
NOTE: effect size calculated in the same way as for Wilcoxon
In a report:
A Mann-Whitney U test revealed there to be a significant difference in problem-solving ability between cats and members of the Trump administration, U= 53.50, p=.01, r= .45, with cats (Mdn= 14.50, range= 19) performing better than the Trump administration (Mdn= 8.50, range=8).
Describe friedman test and when it is used
- Friedman test looks at differences between 3 or more related (within) samples of scores. It is based on ranking the data and comparing the mean rank of each condition.
- Can be used when data is ordinal or when data are interval/ratio but are not normally distributed
Give the effect sizes for kendall’s W
‘Kendall’s W’. This is an effect size statistic that is useful for Friedman’s.
The rule of thumb for this test statistics is the same as for Cohens D.
0.1 = Small
0.5= Medium
0.8= Large
Define omnibus tests
- Friedman and Kruskal-Wallis are both ‘Omnibus tests’.
- These are tests which look to see if there are differences between conditions overall (i.e. between more than 2 groups).
- But it does not tell us exactly where the difference lies.
Explain how omnibus tests allow us to compare more than 2 conditions
- If we want to compare more than 2 conditions we would need to do more than one test.
- What is a problem with this?
- We have an increased likelihood of making a type 1 error.
- So omnibus tests reduce the familywise error rate.
- Instead, we can use omnibus tests.
- It looks to see if there is an overall effect of the IV on the DV but does not report differences between individual conditions.
Explain Post-hoc tests/ Pairwise comparisons
- Post-hoc tests and pairwise comparisons assess differences between conditions following an omnibus test.
- But ONLY if the omnibus test yields a significant result.
- These terms are generally used interchangeably.
- Planned comparisons specifically refer to tests you had planned to conduct and pairwise comparisons for which a test is conducted for every pair of conditions.
- Post-hoc tests generally refer to tests conducted after an omnibus test that you had not planned to do.
- Despite these separate definitions people tend to use them interchangeably
Give an example write up of the friedman test
There was a significant effect of drink type on subjective intoxication, 𝑋2(2)= 50.77, p
Explain when Kruskal-wallis is used
- Can be used when we have 3 or more unrelated (between) subject conditions.
- When data is ordinal or non-parametric.
Give the test statistic for Kruskal-wallis
h
Explain how to calculate the effect size for Kruskal-wallis
- Eta squared = 𝑛2
* 𝑛2= 𝐻𝑁−1
Describe eta squared and give the suggested cut offs
- Eta squared tells us something very specific about our results.
- It tells us how much variance (in the form of a percentage) in our results is accounted for by our IV.
- So in our example 60% of the variance in disgustingness ratings is accounted for by how much participants had drank while selecting it.
- Some suggested cut offs for eta squared are:
- Small= .01
- Medium= .06
- Large= .14
Give an example write up for Kruskal-Wallis
There was a significant effect of the amount of alcohol consumed on the disgustingness of the food that participants consumed, H(3)= 23.40, p
Describe Gustav Fechner’s work
Experimental psychology. Quantitative methods. Linking physical sensations to the mind. Large group studies impractical. Psychophysics today still uses single-case designs.
Describe Ebbinghaus’ work
Experimental study of memory.
Nonsense syllables.
Learning curve and Forgetting curve.
Used only one subject – himself.
Describe broca’s work
Language processing, speech, comprehension.
Patient had loss of speech but not comprehension.
Identified area of the brain important for speech production.
Broca’s area.
Give an overview of nomothetic and idiographic research
Nomothetic: Groups/classes. Universal laws. Predict average behaviour. Idiographic: The individual. Unique.
Describe Allport’s findings
Gordon Allport - founding figure of personality psychology.
Emphasised uniqueness of the individual.
“If you want to know something about a person, why not first ask him?” (Allport, 1953)
Nomothetic approach is inadequate, study of the individual is important.
The clinician’s goal “…. is not to predict the aggregate, but to foretell what any one man will do …… universal and group norms are useful, but they do not go the whole distance.”
Explain single case research
Developing broad psychological theories.
E.g. Amnesia
General understanding of hippocampus function.
Different parts of the brain - different kinds of memories.
Define a case study
Case Studies: Intensive study of a unit / system. Richly detailed information from a variety of sources. Exploratory. Rare/unusual conditions
Give some advantages of case studies
Good source of ideas for research.
Opportunity for innovation especially in clinical samples.
Rare phenomena.
E.g. Luria (1968) – The Mind of a Mnemonist.
Give some disadvantages of case studies
Objectivity: Confirmation bias. Cause and effect: Co-variation/correlation. Time-order relationship. Eliminate possible alternative causes Generalisation: Representativeness? Is the phenomena the same in other people? Low internal and external validity
Give skinners example of single-case experiments
School of experimental psychological research.
Most influential psychologist of the 20th century.
Behaviourist – operant conditioning NOT classical.
Behaviourism evolved from single case experimental research designs.
Training animals using operant reinforcement.
Project Pigeon
Describe applied behaviour analysis
ABAB Design (Reversal): Baseline – intervention – baseline - intervention. Clear change when intervention is added or removed = effective treatment. Some interventions can’t be reversed: E.g. Practical or ethical reasons.
Give the key qualities of ABAB designs
Continuous assessment. Baseline assessment. Standard for improvement. Stability of performance/behaviour. Evaluation of change from baseline. Different phases: High internal validity. Testing different interventions.
Explain how data is evaluated in ABAB designs
Mean / average values.
Level analysis.
Slope / trend analysis.
Latency.
