Week 2 Lecture 2: Literature Reviews and Sampling Flashcards
Reasons to conduct a literature review
(6)
Find out:
- unanswered questions/gaps/lacunae
- what is known
- relevant concepts and theories
- research methods and strategies used
- controversies
- inconsistent findings
Lacunae
= gap
What does peer-review ensure?
(2)
- worthy of being publishment
- no glaring errors of context (methodological) or style (typo etc.)
Types of literature review (10)
- narrative
- systematic
- living systematic
- meta-analysis
- conceptual
- rapid
- scoping
- critical
- expert
- state-of-the-art
Narrative literature review
= ‘tells a story’ about findings based on what you think is important to review
(+) good if expert (ignore junk)
(-) bad if not expert (emit important info)
(-) biased
Systematic literature review
= review of the peer-reviewed literature and sometimes ‘grey’ literature (gov docs, etc.)
(+) thorough, (systematic)
(+) less biased than narrative
(-) time-consuming
(-) unmanageable amount of data
Grey Literature
= information produced by government agencies, academic institutions, and also the for-profit sector that is not typically made available by commercial publishers
Meta-analysis literature review
= comparing statistics of different studies
or
comparing different literature reviews
Living systematic review
= conducted during an event to add to studies as they come out
Rapid reviews
= fast, not completely systematic, literature reviews
Conceptual reviews
= looking at theories/concepts
Scoping reviews
= review breadth of info/study types/focii
Critical reviews
= critical analysis of forms of research in a field
Expert reviews
= done by an expert
State-of-the-art Review (SoTA)
= provides a time-based overview of the current state of knowledge about a phenomenon and suggest directions for future research.
SoTA
= state-of-the-art review, timeline of knowledge + suggestions of future research directions
Cooper’s Taxonomy of literature reviews
use goodnotes tape on taxonomy diagram (6)
Method of classification of literature reviews according to:
1. Focus
2. Goal
3. Organisation
4. Perspective
5. Audience
6. Coverage
fagpoc
Cooper’s Taxonomy of literature reviews Focus catagories (4)
Research outcomes, research methods, theories, applications
Cooper’s Taxonomy of literature reviews Goal catagories
Integration, criticism, central issues
Cooper’s Taxonomy of literature reviews Organisation catagories
Historical, conceptual, methodological
Cooper’s Taxonomy of literature reviews Perspective catagories
Neutral representation, espousal of position
Cooper’s Taxonomy of literature reviews Audience catagories
Specialised scholars, general scholars, practitioners and polititians, general public
Cooper’s Taxonomy of literature reviews Coverage categories
Exhaustive, exhaustive and selective, representative, central/pivotal
Criteria for good literature reviews
8
- justify criteria for inclusion and exclusion
- distinguish what has been done from what needs to be done
- place topic/problem in broader scholarly context and in own historical context
- acquire and enhance subject vocabularly (learn the jargon and add to it) by articulating related variables and phenomena
- synthesize and develop a new perspective on the literature
- identify the methodological issues, ideas and techniques related to the topic
- provide a rationale for the practical and scholarly significance of the study
- write in coherent manner with clear structure
Exploratory research
= conducted to investigate a problem that is not clearly defined
Explanatory research
= seeks to explain why something occurs
Experimental research
= studies conducted with a scientific approach using two sets of variables
Quasi-experimental research
= estimate the causal impact of an intervention on target population without random assignment
Correlational research
= measures the strength and direction of a relationship between variables
Interventional studies
= tests an intervention on a target population
Observational studies
= collects information from participants or looks at data that was already collected, independant variable is not controlled by researcher
Cross-sectional studies
= observational studies that analyze data from a population at a single point in time
Longitudinal study
= observational study of the same variables in target population over long periods of time
Qualitative research
= gathers and analyse non-numerical data in order to gain an understanding of individuals’ social reality, including understanding their attitudes, beliefs, and motivation.
Quantitative research
= collection of measurable data and the application of statistical, mathematical, or computational techniques for analysis
Mixed methods research
= a mix of qualitative and quantitative research
Ontology
= what is true or real; the philosophical study of being
Epistemology
= what can be known and how you can know it; the theory of knowledge
Methodology
= overall theoretical rationale and the principles that define how a research question, set of methods and data are embedded within a perspective; the theory of methods
Methods
= the tools and techniques that you use for gathering and/or analysing data
Factors that influence choosing a research design (4)
- type of question
- stage of research of topic
- analysis of risks to validity and reliability
- available resources
Sampling frame
think of orange and blue bullseye: target population (determined by sampling criteria) –> accessible population (available to researcher) –> sample (selected with sampling method) –> study element (subject or participant)
= the list of popuation members (or units) from which the sample is drawn
- should contain a complete list of every element in a target population
- non-randomly chosen
- elements not in sampling frame have 0 probability of selection
- generalisations can be made only to actual population defined by sampling frame
Sampling scheme/strategy
= method of selecting sampling units from the sampling frame
Sampling frame error
= when sample elements are not listed or accurately represented in a sampling frame
Probability sample
= one in which each element of the population has a known non-zero probability of selection
Types of probability sampling
- simple random sampling
- systematic random sampling
- stratified random sampling
- cluster random sampling
Simple random sampling
= each unit in population has an equal and independant chance of selection and each combination of elements has an equal probability of selection
* usually use random number tables to select elements from an ordered list
(+) free from bias
(-) difficult to obtain
(-) can get unexpected results not representative of population, may be difficult to spot –> increasing sample size solves problem
Systematic random sampling
(implicit stratification)
= dividing sampling frame into intervals, randomly select a starting point and select one element from each interval
* non-equal chance of selection: selection of one member depends on previous
e.g. every 5th unit
(+) more representative sample
(+) eliminate bias
(+) same error rate as simple random sample if list is random
(-) error if periodicity in list matches sampling interval
Stratified sampling and pros
= population broken down into categories, random sample taken from each category
(+) increases precision of choosing sample
(+) can show tendencies within each category
(+) ensures adequate representation of subgroups of interest
Random cluster sampling and pros and cons
= population divided into groups/clusters, certain number of clusters randomly sampled using simple, systematic or stratified cluster sampling
* sampling unit is cluster not individual
* better to sample large no. clusters and smaller random no. within clusters
(+) cheaper and faster than random sample
(+) can show regional variation
(-) not a genuine random sample
(-) bias especially if only a few clusters are sampled
(-) high error if clusters are different from each other
Pure cluster sampling
= whole cluster is sampled
Stratified cluster sampling
= define clusters, group clusters into strata of clusters (similar clusters in a stratum), randomly pick clusters from each strata of clusters, sample participants within sampled clusters (all participants or simple random sample)
Multi-stage cluster sampling
= several stages of stratified cluster sampling
e.g. large national probability samples:
1. whole country divided into geographic clusters, metropolitan and rural
2. some cities selected with certainty
3. other areas formed into strata of areas
4. clusters selected randomly from strata
5. clusters defined within this, process is repeated until blocks or telephone exchanges
6. households and individuals selected with ‘random samples’ through calling and seeing who asnwers
The problem of non-response
destroys generalisability by generalising to people who are willing to respond to surveys
- 90% is good, serious problem with 50% or less but difficult to get above 60%
- multiple call backs to reduce non-response bias
Sampling error
= the probability that any one sample is not completely representative of the population from which it is drawn (iin probability sampling)
- cannot be eliminated but can be minimised
Sampling error effects
Internal validity: possibility of rejecting a true hypothesis (type 1 error) or accepting a false hypothesis (type 2 error)
Generalisability/external validity: of results to wider population of interest
Type 1 internal validity error
Rejecting a true hypothesis
Type 2 internal validity error
Accepting a false hypothesis
Proportional stratified random sampling
= taking samples from stratified groups, in proportion to the population
(+) representative of population
Disproportionate stratified random sampling
·= taking equal or minimum numbers from each group, not proportional to occurence in population
(+) ensures adequate numbers in each stratum for meaningful statistical comparisons
(-) not representative of population
Non-probability sampling
= some elements of population cannot be selected (0 probability) or probabilities of selection are unknown
- not directed at ensuring findings can be statistically generalised to a whole population
- used to examine specific phenomena or specific people’s experiences
- often reach “hard to get at” populations that cannot be found through screeing general populations
- often small samples <30, not large enough for statistical calculations
Quota sampling
= disproportionate stratified random sampling
- non-representative
Criterion sampling
= all cases that meet a set of criteria are selected
Maximum variation sampling
= select cases that provide wide variations in the experience or process being examined
e.g. people who recover very quickly vs very slowly –> insights into barriers and enablers of recovery
Homogenous sampling
= selected to minimise variation and maximise homogeneity to describe an experience or process in as much depth or detail as possible
(+) decrease sense of threat amongst participants (focus groups)
e.g. selecting only women survivors of rape
Heterogenous sampling
= allows for discussing amongst parties with divergent or similar views from demographically distinct populations
e.g. how ppl from different age groups view climate change
Extreme or deviant case sampling
= cases selected that are unsual or have distinctive characteristics that illustarte the process being examined
aim is to elicit rich and detailed info that provides a new perspective on more typical cases
Typical case sampling
= case is selected because it is not in any way atypical, extreme, deviant or intensely unsual
- often used with large units of analysis to demonstrate general process
- mostly useful if report is read by people unfamiliar with area of research
Critical case sampling
= cases selected that illustrate processes where processes would be thought least likely
Snowball or chain sampling
= initial respindents suggest other participants
useful when ppl being studied are:
- well-networked
- difficult to approach directly
- when focus of study is social networks
(-) bias
theoretical sampling
= the process of data collection to generate theory
- aims to represent concepts not people, construct a theoretical explanation by specifying conditions and processes that give rise to variations
- units of analysis are concepts
- representativeness is of the theoretical complexity of phenomenon
- sample design must be flexible and adjusted continuously
continue sampling until saturation:
* no new or relevant data emerge regarding category, or
* category is well-developed in terms of properties and dimensions
* relationships among categories are well established
Sample size determination
= the mathematical estimation of the number of subjects/units to be included in a study
optimisation requred:
* to allow appropriate analsyis
* to provide desired level of accuracy
* to allow validity to the significance test
