Lecture 5 - Study design Flashcards
What is a clinical trial?
A planned experiment in humans designed to measure the effectiveness of an intervention
What is an intervention?
Usually a new drug, but the method can be applied to assessment of a surgical procedure/vaccine/complementary therapy
What are experimental studies?
Different from observational studies as the investigator assigns groups with the risk factors e.g. one group smokes and the other doesn’t
What are the features of a clinical trial?
Experimental study, must contain a control group, prospective (so participants followed through time), patients enrolled/treated/followed up at the same time, participants randomised to control/intervention groups, researches (ideally) are blinded
How are clinical trials designed?
Defined popn> Randomised > a) intervention, b) control > a) and b) cured vs not cured
What is control group?
Study participants who don’t receive intervention under assessment
Why should a control group be included?
Otherwise you can’t be sure of the result, why it happened > could be due to new treatment or may have happened anyway
What are the control groups given?
Placebo or a standard treatment
How are tests randomised?
People eligible for the trial are recruited, consent obtained, and randomly allocated to the intervention and control groups
Why is randomisation done?
To remove treatment allocation bias - can cause skewing of the data
Example of randomisation causing a skew in data?
BCG vaccine for TB in children- children with more ‘cooperative’ parents (more educated, health concious, so lower mortality even without vaccine) were put in the vaccination group and the control group were the others - the death rates were 5x higher in control group than vaccinated children
What does blind/double blind mean?
Blind - patient doesn’t know if new treatment or placebo
Double blind - neither patient nor doctor knows which treatments being given
Triple - statistician doesn’t know different treatments
Why do you blind in a trial?
Prevent bias in reporting/measurement of outcome > measurement bias - either patient or doctor may report signs that aren’t there
How are trials maintained ethical?
Strictly regulated to ensure patients protected - all trials registered, reviewed by independent scientific committee , approved by research ethics committee, adhere to gov and international guidelines
What does the independent data monitoring committee do?
Group of independent researchers who check progress during trial - usually unblind the results to see if major difference in outcome and have power to stop the trial
What are the rights of the patient in the trial?
They are provided with informed consent and are able to withdraw consent at any time without affecting their care
What happens at the end of the trial?
Results are analysed and how their presented depends on the study
How are the outcomes presented?
In terms of efficacy (true biological effect of treatment) or effectiveness (effect of treatment when actually used in normal practice)
What are the trial outputs?
Experimental event rate = incidence intervention arm
Control event rate = incidence in control arm
Relative risk = EER/CER
Relative reduction = (CER-EER)/CER
Absolute risk reduction = CER-EER
Number needed to treat = 1/ARR
How are clinical trials expected to be reported?
According to CONSORT (consolidated standards of reporting trials) guidelines - ensures all papers contain the relevant info to critically appraise the paper
How many clinical trials are there?
4
What happens in Phase I?
Aim to test safety of new treatment - inc. looking at side effects of treatment > involves only a small number of people usually healthy volunteers
Give an example of a trial that was disastrous
Anti-inflammatory drug TGN1412 at Northwick Park hospital in 2006
What happens in Phase II?
Test new treatment in a larger group of people who have the disease to see whether treatment is effective in short term > ~200 - also look at safety
What happens in Phase III?
Test new treatment in an even larger group (~2000) of people - compare new treatment with treatment in use now or placebos > how well treatment works and side effects - smaller +ve effect then larger the cohort
What happens in Phase IV?
Done after drug/treatment has been marketed to gather info on drug’s effect in various popn and any side effects with long term use
What is the drug testing process?
Lab stage (Tissue samples - in vitro tests, Animal testing - mainly rodents for toxicity, MHRA and ethical bodies - drug cleared for human testing), Human testing (Phase I-III), Drug licensed (Phase IV)
Give an example of a successful clinical trial
ACCOMPLISH hypertension treatment trial - RCT comparing benazepril+amlodipine (intervention) with benazepril+hydrochlorthiazide (control) - 1ry endpoint=vascular death > EER=9.6%, CER=11.8%, RRR=18.6% (reduction in events in people taking new treatment cf controls), ARR=2.2% (2.2% lower risk of event when taking new treatment), NNT=45 (need to treat 45 ppl with new treatment for average 3 yrs to avoid one additional vascular event)
What are cohort studies?
Observational analytical epi studies, cohort followed over time -> prospective design
Prospective cohort study ascertains disease during follow up
Exposure measured prior to disease and can measure incidence of disease in exposed/non-exposed and info can be used to calc rate ratios/risk ratios
What are the strengths of cohort studies?
Can look at multiple outcomes, incidence can be calc, good to look at rare exposures, examine temporal relationship between exposure and disease, no ethical concerns, minimise bias in measuring exposure if prospective
What are the weaknesses of cohort studies?
Time-consuming, expensive, loss of follow up may cause bias, healthy worker effect can cause bias in occupational cohorts, inefficient for studying rare diseases
What are case-control studies?
Observational analytical epi studies -> retrospective design, cases defined and exposures compared with controls, controls selected to represent source popn of cases, exposure determined post-diagnosis, odds ratio is only measure of relative risk that can be calc
What are the strengths of case-control studies?
Quick and inexpensive, good for studying diseases with long latency period, good design to evaluate rare diseases, can examine effects of multiple exposures, can collect more detailed clinical info
What are the weaknesses of case-control studies?
Prone to bias, inefficient to examine effects of rare exposures, can’t calculate incidence rates directly, temporal relationship between exposure and disease is difficult to establish
What are some cross-sectional survey examples?
2001 census, health survey for England, NHS inpatient survey on patient experience
What is routine data?
Data routinely collected and recorded in an ongoing systematic way - for admin and statutory purposes without any specific research question in mind at time of collection
Name 3 types of routine data:
Health outcome data, Exposures and health determinant data, disease prevention data, demographic data, geographical data, births, deaths, cancer registrations, notifications of infectious diseases, terminations of pregnancy, road traffic accidents, prescriptions, GP consultation data, community systems, hospital admissions, congenital abnormalities
Why do we need a standardised mortality ratio?
Age is a key confounder that may vary in countries when finding out popn death rates - so age adjustment is needed
What is the standardised mortality ratio (formula)?
Number of observed deaths/number of expected deaths if experienced the same age specific rates as standard popn
What is the standardised mortality ratio (in words)?
Ratio of no. of observed deaths in popn to the expected no. of deaths if the popn had the same mortality as a standard popn corrected for differences in age structure
What is bias?
A systematic error in design, conduct or analysis of a study which produces a mistaken estimate of treatment effect
What is confounding?
When a variable is related to both the study variable and the outcome so the effect of the study variable on the outcome is distorted
What are the different types of randomisation?
Block (assign people to group A/B randomly), Stratification (by centre/ divided by patient characteristics), Minimisation (calculates imbalance and allocates to maintain balance)
How do we minimize measurement bias?
Bliinding, endpoint selection (obj/subj, accurate/precise, consistent/repeatable, 1/2/3ry end points), loss to follow up (missing data, different between groups, intention-to-treat analysis)
How do we evaluate drugs?
Efficacy vs effectiveness
What is efficacy when evaluating drugs?
The biological effect - does it work and its tested in Phase II
What is the effectiveness when evaluating drugs?
True OVERALL effect, does it work in real world and is it worth it? - tested in Phase III
Sample size: Prospective power calculation - What is alpha?
=0.05, the chance of obtaining a difference in the trial is there was no real difference between groups - chance of false positive (type I error)
Sample size: Prospective power calculation - what is beta?
=0.1-0.2, the chance of obtaining no difference in the trial if there WAS a real difference between the groups - chance of false negative (type II error)
How is the power calculated in the prospective power calculation?
Power=1 - beta, usually 80-90%
What is the number needed to treat (NNT)?
No. of patients that need to be treated for one to get benefit
What are the 2 types of epidemiological studies?
Descriptive - to generate hypothesis and Analytical - to test hypothesis
What are examples of descriptive epi studies?
Individuals: case reports/series, cross-sectional studies
Popn: ecological studies
What are examples of analytical epi studies?
Observational (investigator observes w/out manipulating the exposure):case control/cohort
Experimental (investigator determines exposure and who receives intervention): RCT
What do case control studies do?
Investigates relationship between risk factor and 1+ outcomes - ppl already have the investigated disease
What do cohort studies do?
Follow a cohort to see who gets the disease - ppl don’t start with the disease
What is the procedure to carry out a cohort study?
Follow cohort over time and determine no. of ppl with the disease, select ppl who are exposed/unexposed, compare risk of disease in exposed/unexposed cohorts
How to assess exposure?
Self-reported, measured in clinic (BMI), existing records (medical/pharmacy records)
How do you measure the outcome?
Binary (w/wout disease of interest) or continuous (BP, can use arbitrary cut point)
What are retrospective cohort studies?
Determines at present the status of disease - uses data collected in past to ID the popn and exposure status
Why do retrospective cohort studies take less time?
It takes little time to: assemble study popn from past data, determine disease status at present time
What limitations do Retrospective cohorts have?
Exposure measures aren’t collected for the specific purpose
What is the aim of analysis of cohort studies?
How much more likely is an individual to develop a disease if they are exposed to a particular factor?
How do you measure association in a cohort study?
Relative risk/risk ratio - the likelihood of developing a disease if exposed relative to those unexposed
What is the eqn for relative risk?
Incidence in exposed/incidence in the unexposed -> A/row total divided by C/row total
How do you interpret RR
Exposure is associated with decreased risk of outcome
How do you interpret RR=1?
Exposure is NOT associated with outcome
How do you interpret RR>1?
Exposure is associated with an increased risk of the outcome
What is the procedure of a case control study?
Select cases with disease and controls w/out disease, obtain info on past exposures, compare proportion w/exposure vs controls
What is the definition of cases in case control studies?
Newly diagnosed (if possible), objective.strict diagnostic criteria, defined eligibility criteria, is it representative? (un/misdiagnosed, unavailable, dead)
What is the definition of controls in case control studies?
Originate from same study popn as cases, at risk of disease, provide estimate of prevalence of exposure in study popn, defined eligibility criteria, sources: hospitals/clinics, community/popn
What are the advantages and disadvantages of recruiting people from same hospital as cases (case control studies)?
Adv: match with cases in selection factors that bring ppl to certain hospitals/clinics + convenient/low cost
Disadv: medical conditions in control may be associated with investigated exposure
What are the dis/advantages of controls from same geographic area (case-control studies)?
Adv: estimate of prevalence of exposure is popn-based rather than hospital based
Disadv: expensive, less motivation (lower response rates)
What is a limitation of self-reported measurements?
Recall bias
What types of bias cause measurement error in case-control studies?
Recall - patients w/disease respond more carefully
Interviewer - if known who has disease then over-report relevant exposures
Outcome - diagnostic bias, eg: women taking menopausal hormone therapy are screened more often (breast cancer)
How is association measured in case control studies?
Start with disease status and then estimate exposure
What is an Odds ratio?
Likelihood of having exposure if you have the disease relative to likelihood of having exposure if you DON’T have the disease
What is the calc for odds ratio?
Odds that case was exposed/odds that control was exposed -> AD / BC
How do you interpret OR=1?
No association
How do you interpret OR>1?
Positive association between exposure and risk, so exposure asso with ^ risk of disease
How do you interpret OR
Inverse asso between expo and disease, so expo asso with reduced risk of disease
What are descriptive studies?
Aim to describe the distribution of factors of disease in relation to person (age, sex, race), place(w/in or between countries), time (time/season)
What are cross-sectional studies?
Help in decision-making regarding allocation of resources, provide clues leading to hypotheses/analytical studies, describes status of individuals wrt absence/presence of both disease + exposure assessed at same point in time
What can cross sectional studies NOT do?
Easily distinguish whether exposure preceded disease or not
What are the aims of a survey?
Provide annual data about nation’s health, estimate proportion of popn w/health conditions, prevalence of risk factors asso w/conditions, assess freq w/which combinations of risk factors occur, examine differences between popn sub-groups, monitor targets in health strategy, measure height of children at diff ages
What are the core topics of a survey?
General health, psycho-social indicators, smoking, alcohol, demographic/socio-economic indicators, Qs about use of health services and prescribed meds, measurement of height/weight/BP
What are the adv of routine data?
Cheap, already collected and available, standardised collection procedures, relatively comprehensive, wide range of recorded items, available for past years, experience in use and interpretation
What are the disadv of routine data?
May not answer q, incomplete ascertainment, variable quality/validity, disease labelling may vary over time/by area, coding changes could create artefactual ^/decreases in rates, need careful interpretation
What are the top 5 diagnosed cancers in UK (male and female)?
Breast, lung, large bowel, prostate, bladder
Why are single studies unable to conclusively answer a research question?
Poor study design/small numbers, study will often only look at one subset of potential study popn
What is the structure of a systematic review?
Stage I: planning a review, Stage II: conducting the review, Stage III: Reporting and dissemination
What is the Stage I of a systematic review?
Specify the q to be addressed -> framed around PICOS: Popn, Intervention/comparison, Outcomes, Study Design
What is Stage II of a systematic review?
ID of research (clearly defined search criteria, search published medical lit/other sources) and missing studies=BIAS; Selection of Research, Study Quality Assessment, meta-analysis
What is meta-analysis?
Use of statistical techniques in a systematic review to integrate the results of included studies (primary units of analysis)
What is Stage III of a systematic review?
Study details tabulated in a meaningful way, including details of PICOS
What are the issues in systematic reviews/meta-analyses?
Publication bias, inconsistency of results, low study quality, inconsistency of results (diff PICOS)
Why is there publication bias?
Null/non-significant findings are less likely to be reported/published than statistically significant findings - distorting meta-analyses and systematic reviews
How can publication bias be explored?
Using funnel plots - each study is a point and the plot point is determined by OR and study precision
How to interpret funnel plots?
If symmetrical about mean effect, shaped like upside down funnel then no publication bias -> if missing a lower R/LHS corner then pub bias
What is heterogeneity in systematic reviews?
Inconsistency of results - studies answering same q can differ in PICOS, and even when PICOS is same then still can be different due to clinical differences, methodological differences/unknown study characteristics
What to do if too much heterogeneity exists?
May not be appropriate to pool the studies
What are the adv of meta-analyses?
Generate a pooled overall risk estimate, produce a more reliable and precise estimate of effect, explore heterogeneity between published studies, ID if publication bias is occurring
What is a systematic review?
A review of a clearly formulated q that uses systematic and explicit methods to ID, select and critically appraise relevant research and to collect and analyse data from the studies that are included in the review.
What are the adv of systematic approach?
Transparent process (explicit methods ID and rejecting studies), meta-analysis enhances precision of estimates and prescription effects, may ID lack of adequate evidence and areas where research is needed
What do meta-analyses do?
Combine published estimates of effect from each study and generate a pooled risk estimate
What is a forest plot?
Common way of presenting the results from a meta-analysis - graphical representation of results from each study included, with the combined meta-analysis result
Describe a forest plot?
Each study is represented by a box and line - size of box=weight given to study, length of lines= 95% CI
Overall estimate from meta-analysis is shown as diamond at bottom of plot, centre of diamond/dotted line=summary effect estimate , width of diamond=CI around estimate
What is the Bradford-Hill criteria for causation?
Temporal relationship, Plausibility, Consistency with other investigations, Strength of the association, Dose-response relationship, Specificity, Experimental evidence, Coherence, Analogy
