Statistics/Research

Question 1

Study designs

Answer 1

Qualitative and quantitative studies
Within Quantitative studies - have descriptive and analytic
Descriptive studies are all observational - (cross-sectional, case series)
Analytical studies -
Observational = cross-sectional (analytical), cohort study, case-control study, ecological study
Experimental = RCT, non-randomised (quasi-experiment), natural experiments

Question 2

epidemiology

Answer 2

The study of the distribution and determinants of disease or health status in a population.
(Look at the health of populations rather than individuals)

Question 3

Cohort study

Answer 3

Observational analytical study
Can be retrospective or prospective.
Prospective:
Follows exposed and unexposed patients forward over time to determine outcome. i.e. smokers and nonsmokers overtime and rates of lung cancer development.
Issues with prospective cohort studies
1. Loss to follow up
2. Takes a long time to do.
3. Expensive.
4. Confounders

Retrospective:
Looks back for an exposure on a cohort of patinet.s
Statistical analysis based on exposure categories not outcome/disease categories like it is in a case control trial.

Question 4

Case control study

Answer 4

Observational analytical study
Looks at patient with the outcome of interest and looking back to see if they had the exposure in question.
Always retrospective.
Take one group with a disease and one without and look back to see if exposures of interest were different.
Very useful for rare diseases.
Get data fast.
Cheap.
Weak evidence.

Question 5

case series

Answer 5

Observational descriptive study
A report on a series of patients with an outcome of interest, no control group involved.

Question 6

Systematic reviews

Answer 6

A summary of the literature that uses explicit methods to appraise and combine studies.

Question 7

randomised controlled trial

Answer 7

Analytical experimental study
Groups of patients are randomised into either a experiment or control group
Gold standard as can reliably test causality.

Question 8

Hierarchy of evidence

Answer 8

Question 9

Systematic review versus a narrative review

Answer 9

A narrative or traditional literature review is a comprehensive, critical and objective analysis of the current knowledge on a topic. They are an essential part of the research process and help to establish a theoretical framework and focus or context for your research.
Often helpful when the evidence is limited and the research question is broad.
At high risk of bias

Question 10

What is a superiority trial

Answer 10

Trials designed with the intention of showing that one treatment is superior to another or a placebo. This is how most RCTs are designed.
“A is better than B”

Question 11

What is a non-inferiority trial

Answer 11

Trials designed with the intention of showing that one treatment is not inferior to another treatment. (A is not worse than B)

Question 12

What is an equivalence trial

Answer 12

Trials designed with the intention of showing that one treatment is no better or no worse than another. Rare in medical trials as usually you don’t want to prove something is no better, only that it is no worse. (A and B are the same)

Question 13

Why design an non-inferiority or equivalence trial and what differences in trial design are required.

Answer 13

Just because a null hypothesis is rejected, it does not mean that the tested treatment is equal or worse. When a ‘traditional’ test does not demonstrate a difference between treatments, this is often presented (erroneously) as evidence of similarity. It may be that no difference exists, or it may be that the study was not of sufficient power to detect the difference between groups.

Both equivalence and non-inferiority trials assess whether the effects of the new treatment, compared with the standard treatment, stay within or go beyond a predefined clinically acceptable equivalence margin/or the acceptable amount of difference (called the delta value)
This margin is determined prior to the trial and is usually derived from expert opinion and consensus and literature review.

Question 14

REasons to do a non-inferiority trial

Answer 14

Cost reasons: non-inferiority trials often require less patients, therefore costs less for trial to run
Convenience (as above)
When a placebo is considered unethical
When a new treatment

Question 15

Fill in the blanks

Answer 15

Question 16

What is ITT and it’s importance

Answer 16

Intention to treat involves including all participants randomised into the analysis.
Participants are counted in the group they were originally allocated to, even if they discontinued the treatment.
Important as it gives a ‘real world’ answer to an intervention’s effectiveness and limits bias.
The effect of ITT on superiority studies is to bring the two study arms closer together, hence we can be more confident in any difference found. “we found a difference in spite of the participants who switched groups”

Question 17

Explain how intention-to-treat analysis may influence the results of your non-inferiority trial and why per-protocol analysis may have advantages for this trial design.

Answer 17

Per-protocol analysis – when participants are compared on the basis of the treatment they actually received.
In non-inferiority trials we wish to minimise factors that would make the two study arms seem artificially similar, hence per protocol is often the more correct way to proceed. The most convincing results are those in which a non-inferiority is found using both ITT and PP analyses.
“even when the participants received the correct treatment, the treatment was non-inferior”

Question 18

Difference between parametric and nonparametric tests

Answer 18

Tests that are used to calculate a hypothesis.
Parametric tests are used when the data is normally distributed.
Parametric tests are more powerful so you need a smaller sample size to reject the null hypothesis and can have a smaller difference in outcome measure.

Question 19

Fill in the table

Answer 19

Answers

Question 20

Simple T Test

Answer 20

T-Test
* Analysis if there is a statically significant difference between the means of two groups
* Variable for which we want to test a difference must be metric (age, body weight, income)
* Variable must be normally distributed

Simple T-Test
* Used when we want to compare the mean of a sample to a known reference mean
* Null hypothesis: The sample mean is equal to the reference value.

Example: The average birth weight of all babies in NZ is 3200g. The average birth weight of babies born after IVF is 3000g. Is this a statistically significant difference Simple T-test would be used.

Question 21

Independent sample T-Test

Answer 21

• Want to compare the means of two independent groups or samples
• Null hypothesis: The mean values in both groups are the same.
• Assumptions: Normal distribution, variance for the two samples is equal, samples are independent
• If the variance is not equal use Welch’s t-test not the unpaired T-test.

Example: Patients with mild male factor infertility. One group inseminated with standard IVF the other with ICSI. Is there a statistically significant difference in fertilisation rate?

Question 22

Paired sample T-Test

Answer 22

• Used to compare the means of two dependent groups
• The measured values are available in pairs.
• Null hypothesis – the mean of the difference between the pairs is zero.
• Assumptions: Normal distribution and equal variance.

Example: Obese patients with PCOS and IGT are given GLP-1 receptor antagonist Semaglutide (Ozempic) and are weighed before and 6 months after starting on treatment. A paired samples t-test would be used to determine if there was a statistically significant difference in mean weight.

Question 23

Anova (analysis of variance)

Answer 23

• Analysis of variance tests whether there are statistically significant differences between three or more samples.
• Is the extension of the t-test to more than two groups and can be used for independent or dependent samples.
• Assumes normal distribution, observations are independent, variance is equal across the groups.
• Independent samples – single factorial without measurement repetition
• Dependent samples – single factorial with measurement repetition
• Allows us to answer – is there a difference in the population between the different groups of the independent variable (predictors) with respect to the dependent variable (criterion).
• Null hypothesis in one-factor ANOVA
• There are no differences in the population between the means of the individual groups.
• Alternative hypothesis is one-factor ANOVA
• At least two group means differ from each other in the population
• Post-hoc tests to compare the individual groups can be done

Question 24

Pearson correlation

Answer 24

• Analyses the relationship between two variables.
• Provides strength and direction of the linear relationship between two metric variables.
• Assumes normal distribution.
• Note that correlation does not equal causation.
• Allows us to measure the linear relationship between two variables.
• Allows us to determine how strong the correlation is and whether the correlation is positive or negative.
• Both of these are determined with Pearson correlation coefficient r which is between -1 and 1.
• Strength of correlation: (see image)
• Direction of correlation
• Positive correlation (large values of one variable go along with large values of the other or when small values of one variable go along with small values of the other variable)
• Negative correlation (large values of one variable go along with small values of the other and vice versa)
• Null hypothesis in Pearson correlation: The correlation coefficient does not differ significantly from zero (there is no linear relationship).

Question 25

Parametric regression anaylsis.
What
Why (goals)
Assumptions

Answer 25

A regression analysis makes it possible to infer or predict a variable based on one or more other variables.
I.e. what variables impact of salary (i.e. age, education, years working, gender etc.)
Dependent variable = criterion (salary)
Independent variable = predictors (age/education etc.)

Used to achieve two goals:
1. Measurement of the influence of one or more variables on another variable.
2. Prediction of a variable by one or more other variables.

Assumptions:
1. Constant variance for different values of dependent variable
2. Dependent variable is a linear combination of the independent variables
3. In theory low/no measurement error
4. No high collinearity (where two or more independent/predictor variables are closely related to one another)

Question 26

Simple linear regression

Multiple linear regression

Answer 26

Simple linear regression
- dependent variable is a continuous variable
-one independent/predictor variable

Multiple linear regression
- dependent variable is a continuous varibale
- multiple independent variables

Question 27

Simple logistic regression

Multiple logistic regression

Answer 27

Simple logistic regression
* Dependent variable is a categorical variable
* One independent/predictor variable

Multiple logistic regression
* Dependent variable is a categorical variable
* Multiple independent variables

Question 28

Internal validity

Answer 28

Internal validity - when an observed difference between groups are related to the intervention tested in the trial and not to flaws in study design.
The internal validity of a clinical trial is directly related to appropriate design, conduction, and reporting of the study.
- ensure confounders have been addressed
- Bias minimised
The two main threats to internal validity are bias and random error. Bias hereby refers to a systematic error that leads to a systematic deviation of the results from the truth due to flaws in the design, conduction, or reporting of the trial.
Typical sources of bias are flaws in collection, statistical analysis, or interpretation of study data.
Consequently, the true difference between study groups may be either under- or overestimated.

Question 29

External validity

Answer 29

the extent to which the results of an RCT can be generalized into clinical practice and the general population.
internal validity of a study is the prerequisite of its external validity since incorrect data due to missing internal validity can, per se, not be applied to the general population. Even if internal validity is assumed, insufficient external validity may reduce the clinical relevance of an RCT.

Question 30

Types of bias

Answer 30

Flaws in collection, statistical analysis, or interpretation of study data. Causes an under or overestimation of the true difference between study groups.

Selection bias
Recall bias
Funding bias
Reporting bias
Detection bias
Exclusion bias
Attrition bias
Observer bias
Analytical bias

Question 31

Selection bias

Answer 31

When the study group is not fully representative of the population intending to be studied.
(Higher education, English speaking patient’s more likely to join a clinical trial).
Can also occur when recruiters selectively enrol paitents based on what the next treatment allocation is likely to be.

(overcome using allocation concealment and radnomly vary block size and use large block sizes)

Question 32

Recall bias

Answer 32

When people remember things incorrectly.
Often an issue with case-control trials

Question 33

Funding bias

Answer 33

If the studied is sponsored more likely (even subconsciously) to please sponsor.

(overcome by doing studies without sponsorship, or registering clearly the outcome measures and SAP prior to commecning study thourgh trial databse and published protocol)

Question 34

Reporting bias

Answer 34

More likely to publish a clinical trial that shows a positive finding.

(over come by registering clinical trials in a trials database prospectively, and journals publishing both positive and negative research)

Question 35

Detection bias

Answer 35

When the act of looking for an effect erroneously gives the impression of a large effect.
e.g. doctors are more likely to look for diabetes in obese patients than thinner patients.
Can give you a skewed perception.

(overcome with double blinding)

Question 36

Exclusion bias

Answer 36

Excluding patients results in a non generalisable cohort (excluding non-english speakers)

(can be overcome by not having overly restricting exclusion criteria)

Question 37

Attrition bias

Answer 37

Patients drop out - participants may withdraw consent, may be uncontactable, participants violate the study protocol or decline to continue treatment.
(Can be overcome with ITT rather than PP analysis.)

Question 38

Observer bias

Answer 38

Researcher believes they know the outcome.
(overcome with blinding)

Question 39

Analytical bias

Answer 39

Alter analysis to ensure you get the answer you expect.
(overcome with blinidng of statistician, publishing protocol and registrering clinical trial prior to commencing).

Question 40

confounder

Answer 40

A variable, other than the independent variable, that may affect the dependent variable.

Distortion in the measure of association due to a relationship between the study factor (exposure) and another variable which also influences the outcome.

Example in a study looking at embryo transfer catheters and live birth rate. Female age will be a confounder.

Question 41

How to you control for confounding

Answer 41

Design phase:
- randomisation (best way to control for both known and unknown confounders)
- exclusion criteria (e.g. if smoking is a confounder then restrict the study to nonsmokers, limits generalisability of study)
- matching (each pair of persons enrolled in a study are similar for one or a few characteristics)
- stratification (association between exposure and outcome is examined within different levels (strata) of the confounding variable.

Analysis phase:
- stratified analysis
- multivariable analysis

Question 42

Survival time analysis

Answer 42

Survival time analysis is a group of statistical methods in which the variable studied is the time until an event occurs. Doesn’t need to have anything to do with actual survival.
Examples:
1. Kaplan-Meier curve – tells you how likely it is that a something will last longer than a certain point in time. I.e. could be used for age of first child (can’t compare independent samples)
2. Log rank test – compares the distribution of the time until an occurs of two or more independent samples. (so can be used for hypothesis testing)
3. Cox regression (Cox proportional hazards model) – used to determine the influence of different variables on survival time.

Could be used for outcome Time to pregnancy.

Note – censoring of data very important for survival time analysis (i.e. if the event doesn’t occur in the study timeframe this data is censored)

Question 43

ROC curves - receiver operating characteristic curve

Answer 43

Diagnostic tests can produce a binary result (yes/no) or a number on a continuous scale. EG DVT diagnosis ultrasound doppler - yes/no, D-Dimer is a continuous scale.
Diagnostic tests that produced continuous results are often dichotomised as the outcome of interest is generally binary. Done by setting a threshold where the test is ‘positive’ or ‘negative’ (D-Dimer 500)
Setting the correct threshold can be assisted through the use of ROC curves.
ROC curve:
First the sensitivities and specificities for different values of a continuous test measure are tabulated. This results in a list of various test values and the corresponding sensitivity and specificity of the test at that value.
The graphical ROC curve is then produced by plotting sensitivity (TPR) on the y-axis and 1-specificity (FPR) on the x axis.
A ROC curve that follows the diagonal line y=x produces true positives at the same rate as false positive (coin toss).
Area under the curve (AUC) of a ROC is a global measure of the ability of a test to discriminate whether a specific condition is present or not. AUC 0.5 = no discriminatory ability (coin toss - true positive the same as false positive rate), AUC 1.0 is perfect discrimination.
When selecting the optimal threshold you need to consider aims of the test. If you give equal weight to sensitivity and specificity then you would use the point at the top left most corner of the ROC curve (Youden Index).

Question 44

Criterion for a variable to be a confounder (three)

Answer 44

• an independent risk factor for the outcome of interest.
• associated with the study factor.
  -not caused by the study factor

For a variable to be a confounder -
- has to have an association with the dependent variable (female age has an association with live birth rate)
- has to have an unequal distribution between the exposed and unexposed cohorts (the group with one embryo transfer catheter has a high mean age than the other)
- can’t be a causal link in the chain between exposre and outcome

Question 45

components required to perform a sample size calculation

Answer 45

Decide on type of study (superiority or non-inferiority).
Decide on the desired effect size (the smaller the effect size the larger the required sample).
Decide on the power measurement (1-Beta error) - usually at least 80%.
Choose significance level (alpha value) usually 0.05 (95% CI).

Question 46

Sensitivity

Answer 46

Likelihood of testing positive in those with the disease
Sn = TP/(TP+FN)
Good screening test has high sensitivity.
SNOUT = sensitivity rules out disease. Useful for excluding disease as the false neg rate is low in a highly sensitive test. I.e. FFN for PTB - if negative highly likely to be correct as false neg rate very low. So can be reassured won’t labour.
Independent of prevalence.

Question 47

Specificity

Answer 47

Likelihood of testing negative in those without disease.
Sp = TN/(TN+FP)
High specificity tests can be used for definitive diagnosis.
SPIN = specificity rules in a disease. Useful for diagnosis as the false positive rate is low in a highly specific test (i.e. pregnancy test is a highly specific test - if positive the patient is almost certainly pregnant)
Independent to prevalence.

Question 48

Positive predictive value

Answer 48

Clinical relevance of a test

PPV = the ability of a test to detect the presence of disease.
How many people with a positive test have the disease.

PPV = TP / (TP+FP)

Directly related to prevalence. As the prevalence increases the PPV gets higher. (not good for rare diseases).

Question 49

Negative predictive value

Answer 49

NPV: The ability of a test to detect the absence of disease.
The number with a negative test who don’t have the disease.

NPV = TN/ (TN+FN)

Inversely related to prevalence. As the prevalence of a disease increases the NPV falls.

Question 50

Incidence

Answer 50

Incidence is a measure of disease risk
Number of new cases of a disease over a specific time period divided by no. persons at risk (often then multipled by 1000).

Written as “400 cases per 1000 population per year”

Every person in the demoninator has the ability to be in the numerator. (i.e. people with uterus removed can’t be in the denominator for the incidence of uterine cancer)

Question 51

Prevalence

Answer 51

Measure of disease burden
Number of affected persons in the population/the number of all persons in that population.
Prevalence = incidence x duration (determined by death rate and cure rate)
Generally the shorter the duration of an illness the lower the prevalence.
Good tool for distributing health resource

Question 52

Risk, Odds, rate

Answer 52

Risk - chance of a person getting a condition in certain period of time
- need defined population, defined period of time and number of new cases (during that period of time). (basically the same as incidence).
Rate - reported as cases per person years at risk.
Odds - number of events divided by the number of non events.

Question 53

Relative risk

Answer 53

Main measures of association in cohort studies. Also used in RCTs.
Expresses how many times more (or less) likely an exposed person develops an outcome relative to an unexposed person.

Relative risk = the risk in the exposed group divided by the risk in the unexposed group.
RR = incidence of outcome with exposure / incidence of outcome without exposure.

Interpretation:
RR >1 = increased risk of outcome
RR = 1 = no risk of outcome
RR <1 = reduced risk of outcome

I.e SGA Normal weight total
Smoked 19 139 158
Did not smoke 53 798 851

Relative risk = risk of SGA in those who smoked / risk of SGA in those who didn’t smoke
RR = 19/158 / 53/851 = 0.12/.062 = 1.9

Smoking during pregnancy nearly doubles your risk of SGA

Question 54

Odds ratio

Answer 54

Main measure of association in case-control studies.

OR = odds that a case was exposed / odds that a control was exposed.

OR = A/C / B/D or A x D / B x C

Odds = Probability/ (1-probability)
Equation different that RR as you don’t use ‘totals’ so not using total exposed or unexposed anywhere in the equation

How many times more likely the odds of finding an exposure in someone with disease is compared to finding the exposure in someone without the disease.

Interpretation:
OR >1 = increased frequency of exposure among cases
OR = 1 = no change in frequency of exposure
OR <1 = decreased frequency of exposure

Lung cancer	No Smoker	      87	147 No 	            201	508

OR = 87/201 / 147/508
.43 / .29 = 1.49

Question 55

Phases of drug trials

Answer 55

Application to proceed with phase 1 trial must have:
- Animal study data and toxicity (side effects that cause great harm) data
-Manufacturing information
-Clinical protocols (study plans) for studies to be conducted
-Data from any prior human research

Phase 1 (cohort)
- 20-50 participants.
- sometimes randomsied.
- goal is to identify 1.safety
2. correct route and dose of medication
3. Preliminary data on response to treatment.

Phase 2
-<100
- most are RCTs
- goal to identify effectiveness compared to current treatment
- safety profile and side effects still closely monitored.

Phase 3 (Pivotol study)
- RCTs involving hundreds to thousands of patients
- control group either placebo or standard therapy.
- intervention - new treatment.
- used to assess safety and efficacy.

FDA then approve use
Phase IV trial = continues to study side effects and risks of a new treatment.

Question 56

Cross over trial (define) + pros and cons

Answer 56

Cross-over design.
RCTs where study groups “cross-over” part way through.

Pros:
Permits within person comparison
Potential saving in sample size (usually half the sample size)
Allows assessment of patient preference
Cons:
Condition has to be suitable (chronic and stable)
Treatment effect has to be reversible and temporary
Issues with period effect and carryover effect
In infertility trials they are problematic because outcome is usually live birth and so that participant would exit study before crossover and this can bias treatment effect

Question 57

Define systematic review.

Answer 57

Research summary that addresses a focused clinical question in a structured, reproducible manner.
- Often (but not always) accompanied by a meta-analysis.

Question 58

Define meta-analysis

Answer 58

statistical pooling or aggregation of results from different studies providing a single estimate of effect.

Question 59

Types of meta-analysis (3)

Answer 59

Conventional – uses study level data – aggregate data from each study are combined
Network – technique for comparing three or more interventions simultaneously in a single analysis by combining both direct and indirect evidence across a network of studies
Individual participant data (IPD) – when data from every individual enrolled in each study are combined.

Question 60

Two aspects of assessing quality of meta-analysis

Answer 60

1. Credibility of the methods of the systematic review (quality of the conduct of the SR)
2. Degree of confidence in the estimates that the evidence warrants (quality

Question 61

Credibility of methods:

Answer 61

• Sensible clinical question
• Exhaustive and comprehensive literature search
• Demonstrates the reproducibility of the selection and assessment of studies
• Presents results in a useful manner
• Did the review address confidence in the estimates of effect

Question 62

Degree of confidence in results (quality of evidence)

Answer 62

• Study design
• Risk of bias
• Precision and consistency of results
• Whether results directly apply to the patient of interest
• Likelihood of reporting bias

Question 63

Process of conducting a systematic review and meta-analysis

Answer 63

1. Formulate the question
2. Define the eligibility criteria for included studies
   a. PICO
   b. Study design (on RCTs or other studies)
3. Develop a priori hypotheses to explain heterogeneity
4. Conduct the search
5. Screen titles and abstracts
6. Review full text of possibly eligible studies
7. Assess risk of bias
   7b. assess for trustworthiness
8. Abstract data
9. Meta-analysis (if being performed)
   a. Generate summary estimates and CIs
   b. Look for explanations of heterogeneity
   c. Rate confidence in estimates of effect

Question 64

Advantages + disadvantages of a SR

Answer 64

• Summary of all the available evidence
• Include a greater range of patients
• Can enhance confidence in results
• If performed with a meta-analysis
  o Can improve precision of estimates
  o If inconsistency among studies – can explore why

Dis
- Produce estimates that are as reliable as the studies summarised
- If the studies included are heterogeneous and at high risk of bias the results may be misleading.
- Not all SR and MA will follow rigorous design and may produce biased results

Question 65

What is a forrest Plot

Answer 65

• Graphical way of depicting results of a meta-analysis
• Point estimate (mean) of each study represented by a square
  o Size of square proportional to weight of study
• Confidence interval presented as a horizontal line.
• Combined summary effect/pooled estimate, typically represented by a diamond
  o Lateral points of diamond representing the confidence or credible interval
• Vertical line representing no effect is also plotted.
  o If CI overlap this line, it demonstrates that at the given level of confidence their effect sizes do not differ from no effect for the individual study.
  o If the points of the diamond overlap the line of no effect the overall meta-analysed result cannot be said to differ from no effect at the given level of confidence.

Question 66

What is a funnel plot

Answer 66

• A scatter plot that compares the precision and results of individual studies
  o How close the estimated intervention effect size is to the true effect size
• Commonly used in meta-analysis to visually detect publication bias
• Precision of the estimated intervention effect increases with the size of the study
  o Small study effect estimates will typically scatter more widely at the bottom of the graph, with the spread narrowing among larger studies as they are more precise and closer to the true effect.
• Should only be used when at least 10 studies are included in a meta-analysis, because the power of the tests is lower when there are fewer studies.
• Asymmetrical funnell plot usually indicates bias, indicates whether the summary estimate is likely over or underestimating the intervention effect.
• Possible reasons for asymmetrical funnel plot
  o Non reporting bias
  o Poor methodological quality leading to exaggerated effects
  o True heterogeneity
  o Artefactual
  o Chance (usually when small number of studies

Question 67

GRADE system for evaluating evidence

Answer 67

Grading of recommendations, assessment, development and evaluation.

Includes;
Risk of bias
Imprecision
Heterogeneity
Publication bias
Indirectness

Separeted into
High
Moderate
Low
Very low

Question 68

Factors that can decrease or increase Grade assessment

Answer 68

RCTs always start with high confidence.
Observational studies always start with low confidence.

Factors that can modify these ratings:
Decreased confidence with – high risk of bias, inconsistency, imprecision, indirectness, concerns about publication bias
Increase in confidence is uncommon but occurs primarily in observational studies when the effect size is large.

Question 69

Risk of bias

Answer 69

Risk of bias
Many different ways to assess this, should look at trial design, conduct and reporting
Cochrane risk of bias tool can be used:
Random sequence generation (selection bias)
Allocation concealment (selection bias)
Blinding of participants and personnel (detection bias)
Incomplete outcome data (attrition bias)
Selective reporting (reporting bias)
Other bias

Question 70

Consistency

Answer 70

Heterogeneity in study outcomes.
Large differences in point estimates or Cis that do not overlap suggest that random error is an unlikely explanation of the different results.
Can perform statistical evaluation of heterogeneity
1. Cochrane Q test
a. Null hypothesis = underlying effect is the same in each of the studies (i.e. the RR in study 1 is the same as in study 2/3/4)
i. A low P values means that random error is an unlikely explanation for the differences in results, thus decreasing confidence in a single summary estimate. (i.e. you want a high P value)
2. I2 statistic
a. focuses on the magnitude of variability rather than statistical significance.
b. I2 of 0% suggests chance explains variability in point estimates and a single summary estimate can be used.
c. As I2 increases we become progressively less comfortable with unexplained variability in results.
Heterogeneity is study outcomes can be attempted to be explained using subgroup analysis and test of interaction. Or through meta-regression. Often remains unexplained, and high inconsistency/heterogeneity would decrease confidence in evidence.

Question 71

Two ways a study can mislead with outcome results

Answer 71

1. Systematic error (bias)
2. Random error (relates to precision)

Question 72

Precision

Answer 72

• When sample size and number of events are small = imprecise
• When sample size and number of events are large = precise
  Assessed using confidence intervals (the range of values within which the true effect plausibly lies)

Question 73

Indirectness

Answer 73

Optimal evidence for decision making;
1. Research that directly compared the interventions in which we are interested
2. Evaluated the population in which we are interested
3. Measured outcomes important to patients
If population, intervention or outcome in studies differs from those of interest, the evidence can be viewed as indirect.

Question 74

Publication/reporting bias

Answer 74

1. Positive study 3x more likely to be reported than one showing no effect
2. When authors or study sponsors selectively report specific outcomes or analyses the term selective outcome reporting bias is used.
3. Can be identified in met-analysis by the use of the funnell plot with a gap in the plot indicating that studies may have been done and not published.
4. Can be overcome – by obtaining unpublished studies – looking at prospective trial registration

Question 75

PRISMA

Answer 75

Preferred reporting items for systematic reviews and meta-analyses

Question 76

What is heterogeneity

Answer 76

Any kind of variability among studies in a systematic review may be termed heterogeneity.
- Clinical heterogeneity/diversity = Variability in the participants, interventions and outcomes studied (PICOs for studies are different)
- Methodological diversity/heterogeneity = variability in study design and risk of bias
- Statistical heterogeneity = variability in the intervention effects being evaluated in the different studies, this is a consequence of clinical or methodological diversity, or both, among the studies.

Statistical heterogeneity manifests itself in the observed intervention effects being more different from each other than one would expect due to random error (chance) alone.

Clinical variation will lead to statisical heterogeneity if the intervention effect is affected by the factors that vary across studies; most obviously, the specific interventions or patient characteristics.

Differences between studies in terms of methodological factors, such as use of blinding and concealment of allocation, or if there are differences between studies in the way the outcomes are defined and measured, may be expected to lead to differences in the observed intervention effects. Statistical heterogeneity associated solely with methodological diversity would indicate the studies suffer from different degrees of bias.

Meta-analysis should only be considered when a group of studies is sufficiently homogeneous in terms of PICO to provide a meaningful summary. It is often appropriate to take a broader perspective in a meta-analysis than in a single clinical trial.

Question 77

Ways to adjust for heterogeneity

Answer 77

Random-effect model
- accounts for variability both within and between studies, suitable when there is significant heterogeneity.
Meta-regression
- extends a meta-analysis by examining the relationship between study-level covariates and the effect size. Useful for exploring sources of heterogeneity.
Sensitivity analysis
- assess the robustness of the meta-analysis results by repeating the analysis using different assumptions or excluding certain studies. Can help determine if the results are sensitive to particular studies or assumptions.
Subgroup analysis
- analyses subgroups of studies that share similar characteristics to see if the effect size differs across these subgroups. Useful for exploring sources of heterogeneity.

Question 78

How to you assess heterogeneity

Answer 78

Cochran’s Q test
- a statistical test to detecet presence of heterogeneity - provides a p-value indicating whether observed heterogeneity is greater than expected by chance.
I2 statistic
- describes the percentage of variability in effect estimates that is due to heterogeneity rather than sampling error. 25%, 50% 75% indicate low, moderate and high heterogeneity.

Question 79

AI, machine learning and deep learning

Answer 79

Artificial intelligence – Umbrella term - broad field of computer science focused on creating systems capable of performing tasks that would typically require human intelligence (includes deep learning and machine learning)
Machine learning – developing algorithms that allow computers to learn from and make predictions or decisions based on data, instead of being programmed for every task, ML systems improve their performance as they are exposed to more data.
Process of learning from experience without explicit programming
Logistic regression is an example of machine learning.
Deep learning – subset of machine learning that works on principles of human neural networks with many layers. Uses artificial neural networks to extract, process and predict information by learning from examples.