General Flashcards
examples of interventional study designs
N-of-1 randomised trials
systematic reviews
randomised controlled trials
examples of observational designs
cohort studies
case-control studies
cross-sectional studies
ecological studies
examples of descriptive designs
case series
case reports
functions of good research design
- enable a comparison
- allow the comparison to be quantified
- determine the temporal sequence of the risk
- identify the risk factor for the disease
- minimise third variable effects: bias, confounding
features of cohort studies
- exposure measurement occurs before outcome ascertainment; subjects are recruited based on their exposure
- controls do not have the exposure
- exposure is measured at or near the beginning of the study
- all groups are followed through time
- the outcome is measured when it occurs; cases arise during the study
in a cohort study, how might exposure be measured?
questionnaire, blood and tissue samples, pre-existing records
in a cohort study, how might outcomes be ascertained?
- follow-up: is time consuming and expensive, can lead to false diagnoses
- pre-existing records: e.g. registries, medical records
can risk be directly calculated in cohort studies?
Yes: absolute risk, attributable risk (risk difference), relative risk (risk ratio), survival curves
advantages of cohort study designs?
- for rare exposures and common outcomes
- rigorous epidemiological design (able to directly measure incidence)
- provide temporal sequence
disadvantages of cohort study design?
- expensive
- time consuming
- case and controls may differ on important outcome predictors “need to have good case and control selections so that confounding isn’t coming into the sample selection: once you select your sample, you can’t go back and repeat because the timeframes are long”
- can be susceptible to bias and confounding
What are the types of bias present in cohort studies?
- confounding
- sampling bias (selection bias: a sample is collected in such a way that some members of the population are less likely to be included than others)
- migration bias (type of sampling, and selection bias: excluding subjects who have recently moved into or out of a study area)
- measurement bias (measurement of exposure or outcome is not similar between groups of patients studied)
- misclassification bias (measurement error)
(effect modification)
How do prospective and retrospective cohort study designs differ?
Re when the cohort is assembled, in the past for retro, present for prospective
Advantages of prospective cohort studies
- able to collect life and demographic data not available on medical records
- able to set up a standardised way of measuring exposure and degree of exposure to risk factors
disadvantages of prospective cohort studies
- long duration
- expensive
- loss to follow up
- cannot be used for rare diseases
- inefficient because many more subjects need to be enrolled
advantages of retrospective cohort studies
- more efficient than prospective cohort study because data is already collected
- cheaper than prospective cohort study because there is no need for long follow up
- faster because patient outcomes have already been collected
disadvantages of retrospective cohort studies
- long duration
- expensive (less expensive than prospective)
- loss to follow up
- reliance on records; cannot examine a patient characteristic not already recorded
- measurement of exposure and degree of exposure may not be standardised; may not get a standard exposure measurement across different sites, or from patient to patient
- problem with time-dependent exposures; difficult to find temporal sequence
Define relative risk/risk ratio
- ratio of risk in exposed to risk in unexposed persons
- “how many times more likely are exposed persons to get the disease relative to non-exposed persons?”
- relative risk = Risk in exposed / risk in unexposed where a risk ratio of >1 means the exposure increases the risk of disease while, =1 doesn’t change the risk of disease
What are attributable and relative risk calculated from?
the incidence/absolute risk of an outcome in an exposed and unexposed group
4 components of reporting risk
exposed group, unexposed group, the relative risk and the outcome e.g. “Oestrogen users have 1.27 times the risk of developing breast cancer compared to oestrogen non-users
Calculate Population-attributable risk (PAR)
PAR = attributable risk x prevalence of the exposure in the population
or PAR = incidence rate in population - incidence rate in unexposed
Define Population-Attributable fraction (PAF)
PAF = (population attributable risk / incidence in total population) x 100 or what fraction of disease in a population is attributable to exposure to a risk factor PAF = (incidence in total population - incidence in unexposed)/ incidence in total population x 100
Prognosis
prediction of the course of disease following its onset
Define prognostic factors
patient characteristics that are associated with the outcome of the disease
Differences between risk and prognosis
- risk factor relates risk factor to disease, prognostic factor relates disease to outcome
- risk factors deal with healthy people, prognosis deals with sick people
- risk factors the outcome is usually disease onset, prognosis the outcome is the consequence of disease e.g. suffering, death, disability, complications
- risk factors usually for low probability events, prognosis factors are for relatively frequent events
- factors may be different; variable associated with an increased risk are not necessarily the same as those marking the worse prognosis
What is the difference between clinical course and natural history?
clinical course = evolution or prognosis of a disease that has come under medical care and has been treated in a variety of ways that affect subsequent course of events
natural history = prognosis of a disease without medical intervention
important features of prognosis studies
patient selection, time of entry into the study, follow-up, outcomes
What are the ways prognosis can be reported?
absolute risk, relative risk, odds ratio, 5-year survival, case-fatality, disease-specific mortality, response, remission, reoccurrence
How can you be sure a finding isn’t due to chance / or is causal?
When confounding variables are absent, where there are no selection and measurement biases, with p values and hypothesis testing
How do you tell if a finding isn’t due to chance/ or is causal?
hypothesis testing and p values
What is random error?
poor precision by chance alone
What is systematic error?
measurements differ from the truth in a systematic way - a degree of error is always inevitable e.g. blood pressure
When does selection bias occur? What does it result in?
occurs when comparisons are made between two groups that are different in ways other than the main factor under study
the difference between the two groups affects the outcomes of the study
What is the problem with using volunteers in studies?
Volunteers are generally more health conscious compared to the rest of the population (selection bias)
Types of selection biases?
- volunteers
- low response rates - control group, might not be as interested
- ascertainment or detection bias - an individual’s chance of being diagnosed with a particular disease is relating to whether they’ve been exposed (not detecting the exposure correctly)
- healthy worker effect; those able to work are healthier than the broader population (excluding disability, elderly, severely ill)
- loss to follow up
How to control selection bias in case-control studies?
strong case definition, objective ways to assess this
appropriate control selection
high participation rate in both groups
clearly defined inclusion and exclusion criteria
How to control for selection bias in cohort studies?
maximise retention of the cohort (exposed and non-exposed)
where/if possible follow up those who dropped out
clearly defined inclusion and exclusion criteria
When does measurement bias occur?
if the methods of measurement are not similar among the groups of patients; either measurement of the exposure or outcome
What are types of measurement bias?
- recall bias - differences in accuracy of recall of memory e.g. someone inaccurately memory, mothers of children with leukaemia (was mum exposed to X-ray when pregnant mother might be more likely to recall that event than a mother with children without leukaemia)
- reporting bias - incomplete medical notes, some information considered unimportant
- interviewer or observer bias; errors due to individuals taking the measurements
Ways to control measurement bias?
- clear, precise and objective definition
- appropriate choice of measurement definition
- quality control
- is the measurement accurate and precise
What are special biases associated with screening tests?
- Lead-time bias: (Huntington’s disease) overestimation of the survival time due to the backward shift in the starting point for measuring survival that arises from early detection procedures
- length-time bias: (slow vs fast growing cancer) a systematic error due to the inappropriate selection of long-duration cases
- compliance bias: because screening requires a patient to present and be regularly screening the type of patient presenting for screening is generally more compliant than those who are not presenting at screening
What is channelling bias?
type of selection bias. Drugs with similar therapeutic benefits are prescribed to groups with different prognoses: e.g. a new drug may be prescribed to a group with pre-existing morbidity, so a disease state may be incorrectly attributed to use of the drug.
What is data completeness bias?
type of measurement/information bias. Missing data can lead to different results recorded for the same finding.
What is surveillance bias?
type of measurement bias. “the more you look, the more you find” some patients are followed up more or have more diagnostic tests performed on them than others
What is partial verification bias?
measurement bias. In the 1980’s CT scans were obtained when there was a headache present because clinicians assumed that headache increased the likelihood of haemorrhage as a cause of the neurologic deficit. If CTs were more likely to be obtained in the evaluation of acute neurologic deficits when headache was present, then a study of headache as a predictor of haemorrhage would result in higher sensitivity and lower specificity than if headache played no role in obtaining the gold standard CT
What is publication bias?
(Selection bias). When not only the quality, but the results and hypotheses tested influence whether or not a paper is published. Can bias systematic reviews. e.g. research with significant results are more likely to be published.
What is referral or verification bias?
Measurement bias. The results of a diagnostic test affect whether the gold standard procedure is used to verify the test result.
What is reporting bias?
Selection bias. Selective revealing or suppression of information by subjects
What is social desirability bias?
Measurement bias. The tendency of survey respondents to answer questions in a manner that will be viewed favourably by others.
What is spectrum bias?
measurement bias. The performance of a diagnostic test may differ between groups due to the different mix of people in each group.
Cumulative incidence equation?
number of people who develop disease in a specified period/ number of people at risk of getting the disease (at the start of the period)
- measures the proportion of (a group of) people who develop disease during a specified time period
Incidence rate / incidence density equation?
number of people who develop disease/ number of person-years when people are at risk of getting the disease
- measured how quickly people are developing a disease
equation for attributable risk?
incidence in exposed - incidence in unexposed
equation for risk ratio?
cumulative incidence in exposed / cumulative incidence in unexposed
sampling
process of selecting a representative part of the whole population for the purposes of determining parameters or characteristics of the whole population
3 characteristics to think about when determining a population
- members of the population should be susceptible to the outcome
- population relevant to questions begin asked
- sufficiently described so that you can decide to which people the results of the study applies
purpose of statistics
draw inferences from samples about the underlying population or show how representative a sample is of a population - sometimes the degree to which the sample is representative of the population
Sampling error
differences between the sample results and the population characteristic being measured, needs to be kept small
factors contributing to sample error
bias and random sampling error
what is random sampling error
variations within the sample due to the natural variations seen between individuals - occur purely by chance
ways to handle random sampling error
- appropriate sampling techniques (e.g. random selection)
- appropriate sample size (power = statistical terminology): the larger the sample size, the greater variation is measured, the closer the sample value is to the true population variable (the more precise the measurement)
- appropriate statistical calculations measuring error occurring by chance (confidence intervals and p-values) - how much the association is there by chance
- can’t just say the sample isn’t big enough - re study critique - didn’t do a para calculation, what about the techniques to select the sample? ways of measuring cause and effects?
What is a target population?
the collection of individuals about which inferences are desired
What is the study population?
the collection of individuals from which the sample is drawn
What is the sample?
the collection of individuals on which the investigation is conducted
What are the two main sampling methods
non-probability methods and probability methods
What are non-probability methods
- where the probability of choosing someone is unknown
- cheaper, but unable to generalise, potential for biases
e. g. - availability sampling (people are easy to find),
- quota sampling
- convenience sampling: made up of people who are easy to reach e.g. pollster interviews shoppers at a local mall
- deviant case sampling (those who are extremes of that population are chosen)
- typical case sampling (averages are selected)
- snowball sampling (one person tells other people, other people tell other people - e.g. recruiting through Facebook, particularly good in communities hard to break into like drug addicts)
- expert sampling (people who know the most about it are selected - or point to various populations)
- critical case sampling (e.g. using small island nations to analyse the effects of climate change and apply it to larger landmasses)
What are probability methods
the probability of being sampled is known e.g. random sampling methods in RCTs
Importantly, when can statistical inferences be made?
Only when random samples are used.
What is simple random sampling (SRS)
akin to pulling names out of a hat
What is systematic sampling
taking the ith number of a list - less efficient than SRS
What is stratified sampling
taking SRS from mutually exclusive and exhaustive subdivisions of the population: more efficient than SRS
What is cluster sampling
taking SRS from defined clusters; less efficient than SRS
What is a type I (alpha) error
is the probability of finding a difference with the sample compared to the population, when there isn’t really one.
- usually set at 0.05 or 5%
- OR the probability of rejecting the Ho when it is true
=false-positive conclusion
What is a type II (beta) error
the probability of not finding a difference that actually exists between our sample compared to the population, when there really is one. usually set at 0.2 or 20% depending on the study design, again an arbitrary allocation; or 20% chance of missing a true difference is considered reasonably acceptable
- OR probability of accepting Ho when it is false (power; power = 1 - type II error (beta)
=false-negative conclusion
what are the four factors sampling size is dependent on
- level of significance required (p values, CI intervals)
- difference between groups you wish to detect (large or small difference)
- the variability of the estimate
- cost
as the sample size increases… (sample size is dependent on four factors: significance, difference, variability, cost)
- the level of significance becomes smaller
- the difference becomes smaller
- the variability increases
- the cost per unit is reduced
related information generated and sample size
in general, the information generated is a squared function of sample size:
- to get twice the information, you need four times the sample
- to get three times the information, you need nine times the sample
clinical considerations about sampling and samples
- numbers are subordinate to method: a study with rigorous design and a small sample size is superior to a study with poor study design and a larger sample size
- methods and numbers are subordinate to cost: cost is always a factor
- all issues are subordinate to ethical considerations: need to make sure the sample is big enough that an effect can be detected if it is there or prove that it is not there + small enough that we are not asking more participants than is necessary to undergo the inconvenience of being a part of the study
What are the two approaches to chance?
- hypothesis testing (null, alternate hypothesis) - dichotomous outcome, either statistically significant or not e.g. uses the p value
- estimation: estimate a range of values in which the true value is likely to occur e.g. uses confidence intervals
If an RCT is testing whether Drug A is better, what are the four possible conclusions?
- Drug A is better than usual care and this is the conclusion of the study
- *Drug A is the same and has similar effects to usual care and the study concludes that a difference is unlikely
- *Drug A is the same and has similar effects to usual care, but it is concluded that drug A is more effective
- Drug A is more effective than usual care, but the study concludes that there is no difference between drug A and usual care
What is used to estimate the effects of random variation?
statistical testing
When bias is absent, what is responsible for statistical uncertainty?
random error
steps in how to approach a clinical question?
- construct an answerable question
e. g. ‘what is the relationship between coeliac disease and the development of neuropathy’ - set decision threshold (comes from clinical experience)
e. g. a 20-30% increased risk of neuropathy is clinically important (5% level of significance) - gather information
e. g. data collected on small intestine biopsies and matched to neuropathies between 1969-2008 - process information (mechanical act of dealing with large amounts of information - select the right graphical representation of the data, correct selection of statistical tests, cleaned the data)
e. g. risk of neuropathy HR = 2.5 with 95% CI (2.1-3.0), so the results are statistically significant because a HR of 1 = no effect and the CI doesn’t include 1: p value of
What is inference?
drawing conclusions from data
What is statistical inference?
drawing conclusions from quantitative or qualitative information using statistical processes to describe and arrange the data and, in some cases, to test the suitable hypotheses
What are the two major forms of inferences?
estimates (point and interval - ways to describe the data) and hypotheses (use statistics to test the veracity of a hypothesis)
what is a point estimate?
use of sample data to estimate population parameters
- or the best estimate about the population from the data arising within the sample e.g. mean, SD, proportion
= 1 representative figure *the true effect size is unlikely to be exactly that observed in the study (random variation) so for any point estimate, need to have a summary measure to give the statistical precision of that point estimate
What is interval estimate?
- range of reasonable values containing the parameters with a certain degree of confidence
= estimate + and - [confidence x variability]
use of sample data to provide a range of reasonable values intended to contain population parameters with certain degrees of confidence - the more narrow a CI, the more confident one can be about the true size of the effect
= the true value is more likely to be closer to the point estimate than the outer limits of the interval estimate - 95% CI, 5 times out of 100, the true effect size could fall outside the limits or CI; gives statistical precision of the estimate
For data that is individual observations, what measure of spread is used?
Standard deviation
If measurements have been taken from different sample populations, and you want to graph the distribution of the grouped means, what measure of spreads are appropriate and inappropriate?
appropriate: standard error of the mean
inappropriate: SD
When is SD used?
to describe the spread in the frequency distribution
- spread of the individual data; but doesn’t tell what is happening in the population, doesn’t extrapolate data from a sample to a population
What is SEM?
standard error of the mean = standard deviation for the means
- helps to estimate the probable error of the sample mean’s estimate of the true population mean
- standard error = SD / [(square root of N) - 1] where N is sample size; the larger the sample size, the smaller the standard error
Normal distribution values?
1 SD = 68%
1.96 SD = 95%
2 SD = 95.4%
3 SD = 99.7%
Equation for confidence intervals?
CI = mean (-1.96 x SEM), (+1.96 x SEM) = 95% CI, 95% sure the interval will contain the true population mean
When might confidence intervals be used?
to characterise the statistical precision of:
- any rate: incidence or prevalence
- diagnostic test performance
- comparisons of rates: RR, OR, HR
- other summary statistics
to get statistical significance at 0.05: if the point corresponding to no effect (e.g. RR =1 or treatment effect = 0) falls outside the 95% CI for the observed effect, the results are statistically significant
How should a confidence interval be interpreted?
- there is a 95% chance that the true population effect lies within the stated confidence limits
- if the study was repeated 100 times, 95 times out of the 100 the true effect would lie within the expressed confidence limits
- we are 95% sure that what we have measured in the sample is true for the population
- not there is a 95% probability that the parameter lies within the CI - obvious, we are 100% the sample lies within the 95% - not about accuracy of data, but that the data in the sample will be representative of the population
benefits of interval estimates?
more useful in clinical studies than hypothesis testing
- gives how much uncertainty is inherent in the sample statistic because it gives the interval on either side
- formula captures the uncertainty in a sample statistic by defining an interval likely to capture the population parameter with a specific degree of confidence
- the width of the interval estimate depends on:
- level of confidence (higher level, wider interval) e.g. 99% CI?
- variability (sample size: smaller sample, wider interval)
- standard error (larger standard error, wider interval)
Does statistical significance also mean clinical significance?
Statistical significance isn’t always clinically significant; clinically, the effect might be really small, the range might be too wide to have much meaning; depends on the question asked, the patient themselves, clinical experience
What is hypothesis testing?
- drawing of inferences between competing hypotheses as a prediction of the examination of the data is going to show
What are the two types of hypotheses?
- null hypothesis (Ho): the hypothesis to be tested
- alternative hypothesis (H1): the hypothesis contradicting Ho - usually stated in the hypothesis
- all testing is done against the null hypothesis: we reject or fail to reject Ho
what are the steps in hypothesis testing?
- Construct Ho
- construct H1
- determine level of significance
- collect data
- calculate test statistic
- calculate p-value
- compare p-value with level of significance
- reach decision
What should be considered in choosing a test statistic (test statistic is step 5 of hypothesis testing)?
use the table in lecture 21. Predictor = exposure (was the data binary, categorical, ordinal or continuous)
+ outcome (binary, categorical, ordinal or continuous)
- what type of data is it, and is the data predictor or outcome data
What is the p-value? (step 6 of hypothesis testing)
- the p value is a quantitative estimate of the probability that a difference in the treatment effect in a particular study could have happened by random error assuming that there is no difference between the two groups
- the calculated level of significance at which we would be indifferent between accepting and rejecting the null hypothesis given the sample at hand: the smaller the p value, the less likely there is no difference between the two groups
- the probability that the test statistic or relationship would be as extreme or more extreme as the observed value if the null hypothesis were true
- “if there was no difference between treatment groups upon repeated studies, what proportion of the trials would conclude that the difference between the two treatments was at least as large as that found in this study” a large p value supports the null hypothesis that there is no difference between treatment groups
What is statistical significance in terms of p value?
- p value of
Comparing CI and p values, what are their advantages?
CI: emphasise size of the effect, allows the reader to see the range of plausible values and determine the clinically meaningful results, provide information about statistical power
p values: tradition, convenience, sometimes CI aren’t feasible
what is statistical power?
the probability of finding a difference when a difference really exists
- or the probability that a study will find a statistically significant difference when a difference really exists
- where power = 1 - beta
= a determinant of sample size
What does a t-test do?
compares means of a continuous variable in two research samples e.g. treatment and control groups
What are the two types of t-tests?
- A student t-test: used if the samples come from two different groups
- a paired t-test: used if the samples come from the same group (e.g. systolic BP measured in a group of women before and after exercise)
what is a z-test?
works like a t-test but compares the proportion of two groups