Final Exam Flashcards
sampling plan
specifies in advance how participants are to be selected and how many to include
to obtain an accessible sample to make an inference on target population
Target population
who you are trying to make an inference about
accessible population
aggregate of cases that conform to designated criteria and that are accessible for the study
sampling
process of selecting cases to represent an entire population, to permit inferences about the population
elements
basic units about which data are collected, usually humans
two goals of sampling plans
- representativeness of the general population
2. adequate size
if the population is not representative, what type of validity is threatened
external and construct
strata/stratum
subpopulations, mutually exclusive segment of a population defined by one or more characteristics
Ex: high school degree or w/o
multistage sampling
samples are selected in multiple phases
First Phase - large units are selected (i.e. hospital)
Next stage, smaller units are sampled (patients)
bias
systematic overrepresentation or underrepresentation of a population subgroup on a characteristic relevant to the research question
what drives sampling plan strategy
feasibility Ethics Desired rigor Convenience Costs
probability sampling
- Samples are randomly selected
- Everyone in population has an equal chance of being selected
- Used to control sampling bias
- Useful when focus is on population diversity
- Used when researcher needs to ensure accuracy
- Finding correct target population is not simple
(ex: public health initiatives)
non-probability sampling
Samples are selected based on researcher’s judgment
Not everyone has equal chance to participate
Sampling bias is not a primary concern
Useful in environment that shares similar traits
Does not help representation of entire population in terms of accuracy
Finding target population is very simple
when do you use probability sampling (3)
- reduce sampling bias
- when population is diverse
- to create an accurate sample
relationship between selecting a representative sample and sample size
Probability of selecting an unrepresentative sample decreases as size of sample increases
most basic type of probability sampling
simple random
stratified random sampling
population is divided into 2 or more homogenous strata form which elements are selected at random
proportionate stratified random sampling
participants are selected in proportion to the size of population stratum (i.e. race)
cluster sampling
involves selecting groups rather than selecting individuals as the first stage of a multistage approach
systematic sampling
involves selecting every kth person from a list
k = Divide N (population size) by n (sample size)
non probability convenience sampling
using the most conveniently available people as participants
Weakest form of sampling but most commonly used
snowball sampling
variant of convenience sampling → early sample members (seeds) are asked to refer other people who meet the eligibility criteria
quota sampling
non probability sampling
one in which the researcher identifies population strata and determines how many participants are needed from each stratum
consecutive sampling
recruiting all the people from an accessible population who meet eligibility criteria over a specific time interval or for a specified sample size
judgmental/provisional sampling
uses researcher’s knowledge about the population to make decisions
5 threats to statistical conclusion validity
Low statistical power Effect size - small, moderate effects need a larger sample size Heterogeneity of the population Cooperation Attrition
5 steps in sampling
Identification of the population
Specification of the eligibility criteria
Specify the sampling plan: decide method of drawing the sample and how large it will be (i.e. w/power analysis)
Recruit the sample: screening instruments
Generalizing from samples
which studies do not need a power analysis
Descriptive and exploratory studies, and non randomized trials
5 things required for sample size calculations
Significant level desired (a)
Power level of test desired (1 - beta)
Desired sample size (n)
Effect size desired (d) or Cohen’s d
magnitude b/w variables: small effect
.20
magnitude b/w variables: moderate
0.50
magnitude b/w variables: large effect
.8
Type I error
rejection of null hypothesis H0 when it is true
Concluding a relationship exists when it fact it does not
False positive
type II error
accepting the H0 when it is false
Concluding no relationships exists when it fact it does
False negative
how do you avoid type I error
by setting alpha at level they are comfortable with usually .05 or .01
how to control type II
by setting power (1 - beta) at 80% or 20% risk of committing a type II error
counterfactual
Expressing what has not happened but could, would or might under differing conditions
Ex: if researcher is doing an intervention - need to think what could, would or might happen under a different situation if intervention wasn’t done - what hasn’t happened (natural course of condition over time or intervention/condition that would influence outcomes)
3 criteria for causality
Temporal - IV before DV (cause before effect)
Relationship b/w IV and DV
No confounders
what type of study design enhances causality
experiemental, strongest - strongly controlled, minimizes bias
true experiments need what 3 things
intervention (manipulation)
control condition
randomization
blinding (masking)
concealing whether participant in intervention or control - conceal form participant, providers, data collectors, data analyst
single blinded
one group of participants does not what group is randomized, intervention or control
double blinded
those receiving intervention and those delivering intervention don’t know which group participants are in
placebo effects
changes in the outcome attribute to the placebo condition b/c of participants expectations of benefits or harm
complete randomization
no restrictions, allocate each person as they enroll into a study on a random basis - should only be used for studies of 200 or more
simple randomization
starting with a known sample size and then pre specifying proportion of subjects who will be randomly assigned to different tx conditions
gold standard for randomization
have someone unconnected w/ enrollment perform the treatment allocation
steps in RCT (6)
Screen for eligibility of the study Obtain informed consent Collect baseline data Randomly assign to condition Administer control or intervention Collect outcome data
basic experimental design
two groups and 1 intervention and 1 control group w/ outcome measure
Pre test post test design
you measure outcome twice: before and after intervention
post test only design
data on outcome are only collected once - after randomization and completion of the intervention
factorial design
manipulating 2 or more things
I.e. weight gain of infants - touch therapy, music therapy and control group
Can look at interventions separately and together
Look at interaction effect
crossover design
involves expsoing the same people to more that one condition
Must randomly assign participants to different orderings of treatment
concerns with cross over design and how to mitigate
Must be wary of carry over effects -when people are exposed to 2 different conditions, they may be influenced in the second condition by their experience in the first one
Can mitigate with a washout period - no treatment
hawthorne effect
caused by people’s expectations/knowledge of being in the study appears to affect peoples behavior
quasi-experiments
controlled trials without randomization, control group or both
Non equivalent control group design:
involves 2 groups of participants for whom outcomes are measured before and after intervention
Weaker because it cannot be assumed that the experimental and comparison groups are initially equivalent
time series design
data are collected over an extended period during which an intervention is introduced
Extended time period strengthens ability to attribute change to intervention
ex: Ex: rapid response teams were implemented in acute care units → administrators want to examine effects on patient outcomes –> Compare mortality rate before implementation and 3 months after
major strength of quasi-experimental studies
practical, mimics real world
limitation of quasi experimental
Could be other explanations for what happened (i.e. population is different) - rival hypotheses
descriptive correlation studies
to describe relationship among variables rather than to support inferences of causality
univariate descriptive studies
studies involves multiple variables but the primary purpose is to describe status of each, not to study correlation
Prevalence studies
done to estimate prevalence rate of some condition at a particular point in time
Cross sectional designs
incidence studies
estimate frequency of new cases
Need longitudinal designs to estimate incidence
retrospective design
ones in which a phenomenon existing in the present is linked to phenomena that occurred in the past
Begin with DV and then examines whether it is correlated with one or more previously occurring IV
prospective non experimental design
cohort design - researcher start w/ a presumed cause and then go forward in time to the presumed effect
nominal
mutually exclusive categories/groups but no hierarchy
involves assigning numbers to classify characteristics into categories
ordinal
ranked, sorted groups (highest to lowest), involves sorting people based on their relative ranking on an attribute
Doesn’t tell us about how greater one level is from another
I.e. education level
interval
occurs when researchers can assume equivalent distance between rank ordering on an attribute
Ex: temperature scale
ratio
interval level data which has a true zero (absence of a factor), the intervals between objects and the absolute magnitude of the attribute because there is rational meaningful zero
I.e. speed - 0 = not moving, person’s weight
+ skewed
longer tail points to the right
- skewed
tail points to the left
Unimodal distribution
has only one peak - a value with high frequency
Multimodal distribution
two or more peaks
mode
most frequently occurring score value in a distribution
median
point in a distribution above and which 50% of cases call - the midpoint
Usually reported if the data is skewed
mean
sum of all scores divided by the number of scores = average
Affected by every score
which is most stable - median, mode or mean
mean b/c it accounts for every data point
range
highest - lowest
Subtract lowest data point from the highest
variance
spread/dispersal of the data
Heterogenous or homogenous
standard deviation
average variance from mean, based on every score
More stable because it’s based on every score
inferential stats
Allows researchers to draw conclusion about a population, given data from a sample and permits inferences about whether results are likely to be found in a population
sampling error
tendency for statistics to fluctuate from one sample to another; the challenge is how to decide whether estimates are good population parameters
sampling distribution of the mean
theoretical distribution of a test statistic (i.e. mean) from an infinite number of samples as data points
standard error of the mean
standard deviation (SD) of a sampling distribution of the mean (i.e. estimated from the sample’s SD and the sample size)
the smaller the SEM…
the less variable the sample means → more accurate a single mean as estimate of the population value
what are the 2 forms of statistical inference
estimation of parameters and hypothesis testing
who are you estimating for in an estimation of parameters
the entire population
what are the 2 forms of parameter estimation
point estimation and interval estimation
point estimation
calculating a one value/single statistic (i.e. a sample mean) to estimate the population parameter (i.e. population mean)
interval estimation
calculating a range of values the parameter (i.e. population mean) has a specific probability of being located
- interval at which we are confident it lies between
confidence limits
range of values for the population and probability of being right with a certain degree of confidence (95% or 99%)
binomial distribution
probability distribution of the number of successes in a sequence of independent yes/no trials each of which yields success with a specified probability
hypothesis testing
For a particular sample or group in research context
Provides objective criteria for deciding whether hypotheses are supported by data
how to avoid type I error
significance (a or p-value) of 0.05 or 0.01
how to avoid type II error
controlled by setting power (1-B) at 80%
increase sample size
bivariate tests
analysis of 2 variables to assess empirical relationship b/w them
what does a signifcance level of .05 mean
we accept the risk that out of 100 samples drawn from a population, a true null hypothesis would be rejected 5 times
non signifcant result
relationship is from chance fluctuations (accept H0)
significant result
means that the H0 is improbable and thus statistically significant (reject H0 and accept HA)
two tailed tests
hypothesis testing in which both ends of the sampling distribution are used to define the region of improbable values
5% of sampling distribution is equally split b/w 2 tails: 2.5% on each side
one tailed test
critical region of improbable values is entirely in 1 tail of the distribution - corresponding to direction of they hypothesis set by the researcher
5% of the sampling distribution is on 1 side
must know direction of hypothesis in advance! can allow detection of small differenes
parametric tests/stats
involve estimation of a parameter, require measurements on at least an interval scale and involve several assumptions
non-parametric tests
Does not estimate parameters
Uses a rank ordering procedure
Uses variables/data on a nominal or ordinal scale
when do you use a test for independent groups
when comparisons involve different people (i.e. men and women), and study uses a b/w subjects design
when do you use a test for dependent groups
within subjects design, single group of people
compare same participants over time or various conditions - related to one another
if you run a large number of tests on the same data, what type of error increases
type I (false +)
chi-square test
A statistical test used to determine if group differences in a cross-tabs (or proportion of categories in 2 group variables) differ from one another
pearson’s r - correlation coefficient
Designates magnitude of relationship (strength and direction) b/w two variables measured on at least an interval scale, can be used b/w groups and within group situations
multivariate analysis
Statistical procedures for analyzing inter-relationships among 3 or more variables
simple linear regression
Regression analysis is used to predict outcomes 1 IV (x) is used to predict a DV (y)
what does Y’ = a +bX stand for
basic linear regression eq Y’ = predicted value of DV y A = intercept constant B = regression coefficient X = actual value of IV
MLR equation
Y’ = a + b1X1 + b2X2 Y’ = predicted value of variable Y (DV) a = intercept constant b1 = regression coefficient for variable X1 X1 = actual value of variable X1 b2 = regression coefficient for variable X2 X2 = actual value of variable X2
simultaneous multiple regression
enters all predictors into the regression eq at the same time
hierarchical multiple regression
involves entering predictors into eq in a series of steps
stepwise multiple regression
Sequentially add predictors based on ordering the IVs according to their predictor power, evaluate fit at each step
elimination multiple regression
Reverse of the forward stepwise - place all IV predictors in up front and remove them 1 at a time if they do not contribute to overall eq, evaluate fit at each step
what is R and what does it show?
Multiple correlation coefficient
shows strength of relationship between several IV and a DV but not the direction
ANCOVA
Analysis of covariance
Extension of ANOVA by removing effect of extraneous variables (CoV) before testing whether mean group differences are statistically significant
what two things are used in interpreting mLR results
z scores and beta weights
MANOVA
multivariate analysis of variance
Tests for sig differences in 2 or more groups on 2 or more (interval or ratio level) DV outcomes simultaneously (i.e. SBP and DBP)
adjusted means
r/t to ANCOVA
dependent variable after removing the effects of covariates
MANCOVA
- Multivariate analysis of covariance
allows for control of confounding variables when there are 2 or more outcome variables
logistic regression
used to predict categorical outcomes
Predicts a categorical DV (i.e. compliance) based on relationships b/w 2+ IV predictors w/ any level of measurement
odds ratio
ratio of the odds of an event in one group to the odds of an event in another group
what does an OR of 1.0 represent
an OR of 1.0 indicates no differences b/w groups
mixed methods research
planned integration of qualitative and quantitative data within single studies or a coordinated series of studies
meta inference
conclusion generated by integrating inferences from the results of the qual and quant strands of an MM study
3 advantages of MM studies
Complementarity
Practicality
Enhanced validity
4 disadvantages of MM research
Requires a researcher to be competent in both methods - methodologically bilingual
All members of the team need this dual method expert skill set
Expensive
Mixed methods studies may be of longer duration than single method studies
what is the central feature of MM research
Integration
how many research questions are involved in MM studies
at least 2
what does + sign indicate in MM study
indicates convergent design
Purpose is to obtain different but complementary data bout the central phenomenon under study
Qual and quant are collected simultaneously,
what does → indicate in a MM design
sequential
how do you identify the priority of an MM study
capital letters
which MM designs have qual first
exploratory
which MM studies have quant first
Explanatory designs
what does this mean: QUAN + QUAL
Qual and quant are collected simultaneously, with equal priority
what is the intent of exploratory MM studies
Intent is to use rich info to develop a quant feature like a new measure, survey, intervention or digital tool
is the sample size larger in quant or qual of MM studies
quant
MM sampling: identical
occurs when same people are in both strands of the study
MM sampling: parallel
samples in two strands are completely different, but likely drawn from same population
concurrent or sequential designs
MM sampling: nested
participants in the qual strand are a subset of the participants in the quant strand
MM sampling: multilevel
involves selecting samples from different levels of hierarchy
Different but related populations (hospital admins, clinical staff, patients)
inference transferability
degree to which mixed methods conclusions can be applied to other similar people, contexts, settings and time periods
inference quality
incorporates notions of both internal validity and statistical conclusions validity within quant framework and credibility within qual framework
Refers to the believability and accuracy of the inductively and deductively derived conclusions from an MM study
meta inference
conclusions are generated by integrating inferences obtained from the results of qualitative and quantitative strands of a mixed methods study
quantitizing
qual data is converted into numeric codes that can be analyzed quantitatively
qualitizing
transform quant data into qual information
metasyntheses
Interpretative translation of abstract phenomena produced by integrating findings from multiple qualitative studies
qualitative evidence syntheses
SR of qual evidence focused on particular aspects of an intervention, phenomenon, or program, i.e. barriers to participation, satisfaction w/ treatment
Mixed study reviews
integrate findings from qual and quant studies from mixed methods studies
Meta analyses
systematic reviews of quant studies - especially those that focus on intervention and use statistical integration
Use each study to develop a common metric = effect size
effect size
averaged across each study yielding aggregated info about not only the existence of a relationship b/w variables but also estimate of magnitude
scoping review
preliminary investigation that clarifies the range and nature of evidence base
Addresses broad questions and uses flexible procedures and typically does not formally evaluate evidence quality
Can suggest strategies for full systematic review and can indicate whether statistical integration is feasible
Used to identify areas of further research
rapid review
done within a period of a few weeks, do not involve statistical integration and involve a less rigorous search for available evidence
Often used to inform emergent decisions facing clinicians in health care settings
narrative lit review
generic review that identifies and reviews published literature on a topic
Approx time frame = 1-4 weeks
living systematic review
updated as new research becomes available and are published as online only evidence summaries in rapid formats
overview of reviews/umbrella review
reviews in which the unit of analysis is another review
two types of next generation systematic reviews
Individual patient level meta analysis
Network meta analysis
integrative reviews
Broad, not as tightly defined as SR
Span theory and philosophy and include quant designs
systematic review
a rigorous synthesis of research findings on an RQ, using systematic sampling, data collection and data analysis procedures and a formal protocol, quant or qual, considered strongest form of evidence
advantages of meta-analyses
objectivity
enhances power
precision - draws conclusions about intervention’s effect with specified probability the results are accurate
PROSPERO
international prospective register of systematic reviews
Grey literature
studies with more limited distribution = dissertations, conference presentations
publication bias
tendency for published studies to overrepresent statistically significant findings
risk of bias
refers to the likelihood of an inaccuracy in the estimate of a causal effect = threat to internal validity
fixed effects model
assumed that a single true effect size underlies all study results and that observed estimates vary only as a function of chance
random effects model
assumes that each study estimates different yet related true effects and that estimates are normally distributed around a mean effect size
if there is little heterogeneity, what kind of results do models yield
nearly identical results
if there is higher heterogeneity…
analyses will yield different estimates of the average effect size
would want to use the random effects model which is more stable
forest plots
graph the estimated effect size for each study and the 95% CI around each estimate
I^2 test
adjusts for the number of studies in the analysis
(0-100%, 50% = moderate heterogeneity)
sensitivity analysis
test of how sensitive the results of an analysis are to changes in the way the analysis was done
GRADE
two part process - quality of evidence about an intervention’s effect is graded for each outcome –> recommendation is made about using /not using intervention with strength of recommendation (strong or weak)
summary of findings table
show the results of the meta analysis for each outcome, number of participants and studies on which the effect size was based and then the quality of evidence score
what are 2 types of qualitative SRs
aggregative and interpretive
aggregative qual SRs
involving pooling of findings across qual studies in the review
which is more structure: aggregative or interpretive
aggregative
interpretative qual SRs
emphasize creation of integrated conceptualizations and theories by interpreting and reconfiguring findings from qual studies
not that structured
meta-synthesis
Interpretive translation produced by integrating findings from multiple qualitative studies
frequency effect size
indicates the magnitude of a finding - number of reports with unduplicated info that contain a given finding
intensity effect size
indicates concentration of findings within each report
metadata anlaysis
study of results of reported research in a specific substantive area of investigation by means of analyzing the processed data
metamethod
study of methodologic approaches and rigor of the studies included in the metasynthesis
metatheory
analysis of theoretical underpinnings on which studies are grounded
statistical conclusion validity
concerns the validity of inferences that there truly is an empirical relationship/correlation b/w presumed cause and effect
internal validity
concerns the validity of inferences that, given an empirical relationship exists, it is the independent variable, rather than something else that cause the outcome
construct validity
validity of inferences from observed persons, settings and cause and effect operations included in the study to the constructs that these instances might represent
Degree to which an intervention is a good representation of the underlying construct that was theorized as having the potential to cause beneficial outcomes
external valdiity
concerns whether inferences about observed relationships will hold over variations in persons, setting or time
Relates to generalizability o f inferences
threats to internal validity
temporal ambiguity selection bias history maturation mortality/attrition testing effect
threats to external validity
represetantiveness and selection effects
interaction effect b/w relationsihps and people
interaction effect b/w causal effects and tx variation
threats to statistical conclusion validity
low stat power
restriction of range (homogeneity)
unreliable tx implementation
threats to construct validity
reactivity to the study hawthorne effect researcher expectancies novelty effects compensatory effects tx diffusion/contamination
effect size
expresses the strength of relationships among research variables
If there is strong correlation b/w IV and DV - may need a small sample size
t-test
testing for difference b/w 2 group means
independent t-test
two independent groups
experimental vs. control or pre-and post scores for a group of the same people
paired t-test
2 measurements from same person over different time or paired participants together
1 way ANOVA
sum of squares b/w groups and within groups for 3+ groups on 1 IV
2 way ANOVA
3+groups and 2 IV
RM ANOVA
repeated measures
3+ groups over repeated times
why do you use a post hoc test
used to examine where group differences are occuring
type 1 or type 2: think there is no change when there really is
type 2
when to use MM (4)
New or poorly understood concepts
One approach enhanced by second sources of data
One approach alone is not effective
Quantitative results confusing to interpret
limtiations of descriptive correlation studies
self-selection of groups
strengths of correlational research
can get a large amount of data about a problem
central limit theorem
when samples are large, the distribution of the sample means tends to be normally distributed.
beta weight
indication of the relative importance of predictors because scores are standardized.
how long does a scoping review take and how many reserachers
2-8 weeks
2
