critical appraisal Flashcards
What does evidence-based practice involve?
- asking focussed questions
- finding the evidence
- critical appraisal
- making a decision
- evaluating performance
PICO - what must be included in a focussed research question?
Population
Intervention
Comparator
Outcome
What types of evidence are at the top of the hierarchy of evidence?
Hierarchy 1a (level A) Systematic reviews or meta analysis of randomised controlled trials *gold standard* Hierarchy 1b (level A) At least 1 randomised controlled trial Hierarchy 2a At least 1 controlled trial without randomisation Hierarchy 2b At least 1 other type of quasi-experimental study
What is the purpose of critical appraisal?
To assess and consider validity, reliability and applicability.
What are some quality markers of a good study?
Should:
- have random allocation of participants to interventions (reduces bias/confounding variables)
- have outcome measures for at least 80% participants
- show causation rather than association
what is validity in critical appraisal?
Validity is how close to the truth something is. I.e. is a study testing what it says it’s testing or are there
confounding variables which are in fact the reason for the results.
What is reliability in critical appraisal?
Reliability is how consistent results are. If the experiment was repeated again, would the same/similar results be seen?
What is applicability in critical appraisal?
Applicability is how relevant a study is to clinical medicine.
What are observational vs experimental/intervention studies?
Observational studies: can be descriptive, descriptive and analytical or analytical studies.
Intervention studies are generally randomised control trials.
Examples of descriptive observational studies
(2)
Uses/flaws?
- case reports/case series study individuals
- ecological studies use routinely collected data to show trends in data - useful for generating hypotheses, but shows prevalence/association not causation (e.g. John Snow water pump)
Example of an observational study design that is both descriptive and analytical.
Uses/flaws?
Cross sectional study/survey - divides population into those without the disease and those with the disease - data is collected on them once to find associations at a specific point in time.
Used to generate hypothesis but are prone to bias and have no time reference/follow up.
2 examples of observational studies that are analytical
- case-control studies
- cohort studies
Describe case-control studies:
- benefits
- downsides
Retrospective studies that take people with a disease and match them to people without for age/sex/habitat/class and study previous exposure to the agent in question.
Positives: quick, inexpensive
Negatives: retrospective nature means only shows an association, and data may not be reliable due to problems with patients memories
Describe cohort studies:
- benefits
- downsides
Start with a population without the disease in question and study them over time to see if they are exposed to the agent in question (e.g. stress, smoking, etc) and if they develop the disease in question or not.
advantages: possible to distinguish preceding causes from concurrent associated factors, lower chance of bias, absolute relative and attributable risks can be determined. Prospective meaning causation can be shown where retrospective studies cannot.
downsides: requires controls to establish causation
What is an:
Independent variable:
Dependent variable:
An independent variable is a variable that can be altered in a study.
A dependent variable is a variable that is dependent on the independent variables or one that cannot be altered.
What are some points of the Bradford-Hill criteria, used to evaluate the likelihood that an association is causal?
- consistency: results are replicated in studies in different settings
- strength of association
- dose-response relationship: does increased exposure increase risk?
- temporal: does the exposure always precede the outcome?
- biological plausibility
- coherence with existing theories
- specificity: weakest criteria, established only when a single cause produces a specific effect (usually not the cause - e.g. effects of smoking)
- condition can be altered by experimentation
when should reverse causality be considered?
where it is unclear which variable is dependent and which is independent
what is a systematic review?
A review of a clearly formulated question that uses symptomatic and explicit methods to identify, select and critically appraise relevant research and to collect and analyse data from the studies that are included in the review.
what is meta analysis?
- statistical methods are used to analyse and summarise the results of the included studies
- can be graphically represented as a forest plot
- review and analysis of a variety of existing randomised control trials
why is routine health data collected, collated and disseminated on a regular basis in the UK?
- to monitor the health of the population
- to generate hypotheses on causes of ill health
- to inform planning of services
- to evaluate and assess performance of policies and services
What types of health information are routinely studies in the UK?
- mortality
- morbidity
- use and quality of healthcare
- health status/quality of life
- individual lifestyle
- socio-economic, cultural and environmental conditions
What are some types of qualitative research?
- ethnography (immersing oneself in a particular lifestyle or group)
- interviews
- documentary analysis
what are the problems that can arise when using qualitative research as a source of information?
- Meaning imposition: not truly understanding what someone else is thinking
- Crisis of representation: research cannot capture lived experiences
- Reflexivity: personal interpretations of data
- Generalisation
Graphs used to represent continuous data:
- stem and leaf diagram
- histogram
- box and whisker
Graphs used to represent discrete, categorical data:
- bar charts
- pie charts
Define median.
advantages?
The central value when data is placed in order.
It is not affected by outliers, reflects what most people experience and is useful when data isn’t skewed.
Define mode
most frequent value
rarely used
Define mean
The sum of the values divided by the number of values.
Uses all the data, is the expected value.
Commonly used.
What is the range?
The difference between the highest and lowest values.
What is the inter-quartile range?
Data is divided into quarters (quartiles) and the inter-quartile range is the difference between the middle two quartiles.
Is the middle 50% of values.
What is standard deviation?
The average distance of the observations from the mean value/ Used to find abnormal results/outliers.
What information is plotted on a box and whisker diagram?
Box representing the interquartile range is drawn.
The median is shown by a line across the box.
Lines extend outside of the box to the highest and lowest results (the range) excluding outliers.
Outliers are shown as dots.
How is normal distribution represented graphically?
Bell-shaped curve where 2/3rds of the data lies within 1 standard deviation of the mean and 95% lies within 2 standard deviations of the mean. The median and mean will be the same in a normal distribution.
Skew:
- if the data is symmetrical, what calculations should be used to summarise data?
Symmetrical: mean and standard deviation should be used
Skewed: median and interquartile ranges should be used
What is positive skew?
Negative skew?
e.g for each
Positive: the mode is less than the median which is less than the mean. e.g household income
Negative: the mode is greater than the median which is greater than the mean. e.g. age of death
What information does a reference range give in data sampling?
How is this range usually determined?
Gives limits within which we would expect the majority of data to fall.
For normally distributed data 2 standard deviations above/below the mean are used (95% of the data).
Define population and sampling in data collection
Population - all the individuals we are interested in in a study
Sample - a group within a population which we will study to provide estimates of the true values within the group. Cam be random/non-random.
Types of non-random sampling?
- convenience sampling: e.g. all available patients at a particular point in time
- purpose/quota sampling
types of random sampling?
- simple random: all individuals within a population are equally likely to be picked
- stratified random sampling: population is divided into groups and then randomly sampled within these groups
- cluster sampling: rather than sampling individuals, groups/clusters of individuals are sampled
Downside of random sampling?
Advantage?
Sample not necessarily representative of the whole population.
Advantage: avoids selection bias
What is standard error?
The standard deviation of all the sample means.
The standard deviation divided by the square root of the number in the sample.
Standard error is an estimate of precision. It provides a measure of how far from the true value the sample estimate is likely to be.
Significance of higher standard deviation/higher standard error.
How to lower standard error?
= more variable data, so more difficult to get representative data from a sample.
Larger sample size = lower standard error
What is the confidence interval and how is it calculated?
The 95% confidence interval is found between two standard errors (1.96 standard errors) above and
below the mean. It is smaller with larger sample sizes. If the size of the sample is squared, the confidence intervals are half the size.
It is used to assess the sample mean to see how good it is as an estimate of the true population mean.
List of steps in estimation and hypothesis testing to make a decision?
- Set null hypothesis H 0 and study hypothesis H 1
- Carry out significance test
- Obtain test statistic
- Compare test statistic to hypothesised critical value
- Obtain P-value
- Make a decision
What is the null hypothesis?
The statement we are looking to disprove. We assume no relation until proven otherwise. Usually the opposite of the study hypothesis.
How is the test statistic calculated? what is it?
reduces the data to a single value.
= (observed value-hypothesised value)/standard error of the hypothesised value
How are significance tests used?
The calculated test statistic is compared to a hypothesised critical value (using a distribution we expect if the null hypothesis is true) to obtain a P value.
What is the p value?
The probability of obtaining the test statistic from the data, assuming the null hypothesis is true.
If the P value is less than 0.05 the null hypothesis can be rejected.
What is a clinically significant difference?
One that is big enough to be worthwhile. It is important that the size of the sample is adequate to detect the clinically significant result, at the 5% significance level with at least 80% power.
What is absolute risk?
The incidence divided by the population.
The probability that an event will occur.
What is relative risk?
The risk of an event in an exposed group divided by the risk in the unexposed group.
null value is that there is no difference in risk between groups.
If relative risk = 1, there is no difference between groups
What is absolute risk difference (/risk reduction/risk excess)
The absolute additional risk of an event following an exposure.
ARD = risk in exposed group - risk in unexposed group
e.g there is a 5% increase in risk of developing cancer if you take the pill
null value is that there is no difference
What is meant by the number needed to treat to benefit?
calculation?
The additional number number of people you would need to treat in order to cure one extra person compared to the old treatment
= 1/absolute risk reduction
What is mean by the number needed to treat to harm?
calculation?
For a harmful exposure - the additional number of individuals who need exposure to the risk in order to have one extra person develop the disease compared to an unexposed group.
= 1/absolute risk difference
What is meant by the odds of an event?
Calculation?
The ratio of the probability of an occurence compared to the probability of a non-occurrence.
Odds = probability/(1-probability)
What is the odds ratio?
The ratio of odds for exposed group to the odds for the not exposed groups.
{Pexposed/ (1-Pexposed)} /
{Punexposed/ (1-Punexposed)}
At what point to individuals feel stress?
When demands on an individual > their ability to cope
What are the 2 types of stress, positive and negative?
Distress = negative stress Eustress = motivating
What are some responses of the body to stress?
- biochemical
- physiological
- behavioural
- cognitive
- emotional
- biochemical: alterations in endorphin and cortisol (stress hormone)
- physiological: increased stomach acid (stomach ulcers), raised blood pressure, tension headaches
- behavioural: over-eating/anorexia, smoking/alcohol
- cognitive: anything affecting comprehension/brain: negative thoughts, poor concentration
- emotional: moodswings, irritability, aggressive, tearful, boredom
What are some factors contributing to chronic stress?
- prolonged exposure to stress
- lack of control over the stress (helplessness)
- personal characteristics of an individual
- adverse life events
What are some complications linked to chronic stress?
- hypertension
- anxiety
- peptic ulcers (H pylori bacteria)
- IBS
What are some techniques for stress management?
Exercise
Meditation, physiotherapy
Cognitive Behavioural Therapy
Medications
What is meant by the opportunity cost in medicine?
The loss of other alternatives when one alternative is chosen - e.g. more heart transplants are done with the opportunity cost of doing more hip replacements
What is economic efficiency?
Using your limited resources to maximise benefit
How is economic evaluation calculated?
3 methods
- cost effectiveness analysis: outcomes measured in natural units (e.g. cost per uyear of life gained)
- cost utility analysis: outcomes measured in quality adjusted life years (incremental cost per quality of year of quality life gained)
- cost benefit analysis: outcomes measured in monetary units (net monetary benefit)
what are the principles of Gillick competency?
Whether or not a child is capable of giving necessary consent is dependent not only on age but on the child’s maturity and understanding and the nature of the consent required.
The child must be capable of making a reasonable assessment of the advantages and disadvantages of the treatment proposed, so the consent, if given, can be properly and fairly described as true consent.
involved for example in giving contraceptive advice to an under 16 year old
What is ecological study design?
Details:
Observational study of population measuring at least one variable.
The occurrence of disease is compared between groups that
have different levels of exposure, thus offering a comparison group for this study design.
Ecological studies: advantages
Quick and cheap - routinely collected data used.
Units of analysis are whole populations.
Can examine patterns by age, sex, ethnicity, and/time.
Few ethical issues.
Useful for generating hypotheses.
Ecological studies: disadvantages
No link between individual exposure and effect. Bias-Variation in diagnostic criteria. Absence of records of individuals. Unsuitable format. Inconsistency in data presentation.
Describe cross-sectional study design
Observational study. Individual-level variables that measures exposure and disease at a specific point in time.
A snapshot of the study population.
Advantages of cross-sectional studies
Results generate hypothesis.
Rapid feedback of current events in community.
Quick and cheap.
Few ethical issues.
Disadvantages of cross-sectional studies
Could just be reporting a medial oddity.
Prone to sampling, subject and observer bias.
No time reference.