Research Methods

Question 1

What is Confounding?

Answer 1

when we miss the real relationship between one variable and another when looking for a causal relationship, this is usually because a confounder is operating in the background

• A confounder can be another risk factor for the disease
• A confounder can also be a preventive factor for the disease
• A confounder can also be a surrogate or a marker for some other cause of disease

Question 2

What is the confounder in the idea that the MMR vaccine causes Autism?

Answer 2

• the age of symptoms/ diagnosis of autism aligns with the age the MMR vaccine is given

Question 3

What is a randomized controlled trial?

Answer 3

It is an Analytic-experimental trial

A study which participants are allocated randomly between an intervention and a control group

The most common being a two arm parallel design

it protects against confounding

Question 4

Why are trials conducted?

Answer 4

Safety

• Ascertain the safe dose of a new drug.
• Demonstrate safety and tolerability of a new compound
• Monitor adverse events profile of a new drug (against an existing drug or placebo)

Efficacy/ Effectiveness

• Demonstrate efficacy of new drug – does it work?
• Show that treatment T is superior or equivalent to treatment X
• Demonstrate effectiveness, and cost-effectiveness, of A vs. B

Question 5

How doe randomized controlled trials protect against confounding?

Answer 5

Taking baseline data, providing an intervention, and then measuring change since baseline = temporal precedence of exposure to outcome

• If we randomly assign people to have intervention/comparator…
• Allocation decided by chance
• Confounders should be equally distributed between groups
• Any effect of confounders on outcome likely equal within each the group
• Any observed between-group difference in observed outcome likely due to intervention

Question 6

What is Clinical equipoise?

Answer 6

• genuine uncertainty in the expert medical community over whether one treatment will be more beneficial
• ethical basis for medical research that involves assigning patients to different treatments
• It is not ethically acceptable to randomize patients to a treatment/condition known to be inferior
• a lack of research evidence suggesting a difference in efficacy does not stop participants from having a preference
  
  • strong preferences can make it harder to recruit

Question 7

What makes a good randomized controlled trial?

Answer 7

• Internal validity - is the exposure causing the desired outcome in this study
• External validity - to what extent can these findings be generalized to other people, situations and times

Question 8

What is Bias?

Answer 8

Generally - ‘a partiality that prevents objective consideration of an issue or situation’

In statistics - ‘a tendency of an estimate to deviate in one direction from a true value’

Bias is any departure of results from the truth

Question 9

How do biases occur in randomized controlled trials?

Answer 9

• Bias in RCTs occurs when systematic error is introduced into sampling or testing by selecting or encouraging one outcome or answer over others
• Bias is independent of both sample size and statistical significance
• Unlike random error - this results from sampling variability and decreases as sample size increases

Question 10

What types of bias are there?

Answer 10

• Selection bias
• Performance bias
• Attrition bias
• Observer/detection/information bias

Question 11

What is Selection bias?

Answer 11

• Representativeness of sample to wider population: not adequately capturing the relevant population
• Systematic differences between baseline characteristics of groups that are compared
  
  • imbalance in demographic/ clinical characteristics

Question 12

What is Performance bias

Answer 12

Systematic differences between groups in care provided, or in exposure to factors other than the interventions of interest

e.g different incentives, follow-up appointments in different locations

Question 13

What is Attrition bias?

Answer 13

Systematic differences between groups in withdrawals from a study

e. g more control group participants withdrawal from the study because they become unwell
e. g greater interventional participants drop-out due to side effects

Question 14

What is Observer/Detection/Information bias?

Answer 14

Outcome measure does not adequately capture outcome of interest

Systematic differences between groups in how outcomes are determined - or how information ic collected for the groups

e.g blood tests vs questionnaires

Question 15

What are solutions to bias in RCTs?

Answer 15

• Selection bias - inclusion/exclusion criteria + sampling strategy/size; randomization +allocation procedures
• performance, attrition, /observer/detection/information bias - blinding/masking

Question 16

What are you options if possed this question?

Some of the trial participants who get randomised to the treatment arm only have (e.g.) one dose of the vaccine instead of two. What should I do with their data?

Answer 16

• Intention-to-treat analyses
  
  • analyse outcomes for everyone randomized, irrespective of whether they have/adhere to intervention/s allocated
  • more conservative test, better reflection of real-life in which not everyone adheres to treatment
• Per-protocol analyses
  
  • analyse outcomes for only those who received ‘does’ of intervention as specified by the protocol
  • better test of intervention’s actual effectiveness when received as optimally specified

Question 17

What Errors can we make in a study?

Answer 17

• type 1 error means that we observed a difference when there wasn’t really one e.g. our intervention was significantly better in our study, but this effect does not actually exist
• type II means that we didn’t observe a difference when there actually was one e.g. our interventions looked equivalent but actually the new intervention is better
• as we reduce the chance of making a type 1 error we increase our chance of making a type 2 error

Question 18

What is the Significance level?

Answer 18

The rate at which we are comfortable in making a type 1 error - type 1 error rate

the convention is 5% or 0.05

meaning we are comfortable to make a type 1 error 5% of the time

Question 19

What is the ‘Power’ of a study?

Answer 19

this the probability that a test will not amiss an effect when an effect truly exists

power is set at 1 minus 0.2 = 0.8/ 80%

Question 20

What is the P value?

Answer 20

When you compare groups using a statistical test

the p value is the probability that the difference observed could have occurred by chance if the groups compared were really alike

E.g. If p=0.12, the probability of observing this result (or one more extreme), given that the null hypothesis is true, is 12%

• this can be effect by our sample size

Question 21

How can the p-value be effected by the sample size/

Answer 21

• Small sample that is highly variable → high p-value
  
  • there is more likely to look like the two groups differ even if the null hypothesis is true
• Many sample/observations with low variability → low p-value with a lower probability of making a type I error

Question 22

What do we need to calculate sample size?

Answer 22

1) What is the clinically important effect / minimal clinically important difference?

• Smallest difference in outcome, on average, between treatment groups that would demonstrate clear advantage of one treatment over another (on chosen primary outcome measure)
• Primary outcome of trial should be clearly defined before the trial
• Size of effect determined from previous literature and/or experience

2) How much does this outcome score vary in general in the population?

• Understand the likely spread of our scores (standard deviation)

3) How much error we are willing to allow?

• Set significance level and power

Question 23

What is causation?

Answer 23

• David Hume (1737-40): If X causes Y…
• (i) Xs are “constantly conjoined” with Ys,
• (ii) Ys follow Xs and not vice versa, and
• (iii) there is a “necessary connection” between Xs and Ys such that whenever an X occurs, a Y must follow
• Bertrand Russell (1912-13): ‘the word “cause” is … inextricably bound up with misleading associations’

Question 24

Can we measure/ observe causation?

Answer 24

• Causation can be inferred (deduced, concluded) only –
• no research design can allow us to directly test causal relationships
• i.e. we can just observe associations in time and space and generate evidence to support the notion that causation occurred
• Causation explicitly applies to cases where action A causes outcome B

Question 25

What are Koch’s Postulates?

Answer 25

1. The specific microorganism should be shown to be present in all cases of organisms with specific disease but should not be found in organisms without the disease.
2. The specific microorganism should be isolated from the diseased organism and grown in pure culture on artificial laboratory media.
3. This freshly isolated microorganism, when inoculated into a healthy laboratory organism, should cause the same disease seen in the original organism.
4. The microorganism should be re-isolated in pure culture from the experimental infection

Question 26

What is the Bradford-Hill criteria?

Answer 26

• Strength of association → Stronger association between risk and outcome, less likely caused by other factors
• Consistency → Replication of results by different researchers, different settings
• Specificity → Exposure associated with single specific disease
• Temporality → Exposure precedes disease
• Biological gradient → Increased exposures correspond increased disease
• Plausibility → Rational scientific basis for exposure-disease association
• Coherence → Association consistent with other knowledge on topic
• Experimental evidence → Experimental evidence reinforces causal inference
• Analogy → Association analogues to known causal relationship

Question 27

What are criticisms of the evidence hierarchy pyramid?

Answer 27

• Over-value of quantitative data/ analysis, statistical significance
• Over-value of the idea of a single ‘evidence’
  
  • multiplicity of evidence is best to support causation - all levels of investigations needed for full understanding of disease/phenomenon
  • • not problem-driven - some types of evidence not possible or ethical to generate in some scenarios
• Over-value internal validity….(RCT, causal study design, bias appraisal)
• …under-value external validity (local and individual context)
• Lack of consideration of importance of different outcomes and their primacy to patients, and harder to measure outcomes
• Less evidence for ‘real-life’ patients e.g. with comorbidities, long-term conditions, conditions/interventions less suited to quantitative study/RCTs
• Under-value of mechanism evidence
• Under-value of patient perspectives, doctor expertise and judgement, tailoring and individualised medicine

Question 28

What is qualitative research useful for in health and medicine?

Answer 28

• Sets patient-centred agenda
• challenges received wisdom
• can generate new theory and new hypotheses
• Exploratory looking at the ‘what’, ‘how’ & why’ questions
• rather than ‘ how many’, ‘how much’ or how frequently’
• Many/most important issues people face—their feelings, hopes, and beliefs, for example—cannot be meaningfully reduced to numbers or understood without reference to socio-cultural context
• Especially relevant to complex and sensitive conditions, e.g. STDs, mental health problems = as much socially as medically experienced

Question 29

What is the role of qualitative research before a RCT?

Answer 29

For the intervention

• understanding the problem
• intervention development and refinement

For the RCT

• optimising RCT design
• optimising recruitment strategies

Question 30

What is the role of qualitative research during a RCT?

Answer 30

For the intervention

• engagement with intervention
• experiences of interventionists
• understanding how intervention works (or why it did not), for whom, and under what circumstances

For the RCT

• understanding treatment as usual
• understanding delivery of trial processes
• understanding recruitment problems

Question 31

What is the role of qualitative research after a RCT?

Answer 31

For the intervention

• participants’ reflections and perceptions of change
• understanding delivery and fidelity
• understanding issues in implementation

For the RCT

• understand unexpected RCT results or outcomes
• understand and share learning from the RCT
• inform next stage developments

Question 32

What are examples of data colellection tools?

Answer 32

• Questionnaires
• Assessor/expert-rated instruments
• Patient Notes/Records review
• Survey of staff (paper/online)
• Survey of patients (paper/online)
• Telephone interview
• Face to face interviews
• Focus groups

Question 33

When and how are questionnaires used for data collection?

Answer 33

• Often used for primary research and broader health improvement projects such as service evaluation e.g.
• Survey of staff (paper/online)
• Survey of patients (paper/online)
• Long surveys with multiple questionnaires should be randomly ordered if possible
• Missing data should be avoided where possible
• Participant-facing questionnaires may also be used in audit/service evaluation/quality improvement

Question 34

When and how are interviews and focus used for data collection?

Answer 34

• Often used for primary research and health improvement projects e.g.
• Focus group with staff (paper/online)
• Interview with patients (paper/online)
• May be unstructured, semi-structured or fully structured
• Example interview/focus group topic guides available and information about how to develop new ones (e.g. see My Studies)
• Depending on research question, may involve other materials e.g. photo elicitation, social network mapping etc.
• Interviews/focus groups may also be used in audit/service evaluation/quality improvement

Question 35

What should you consider when identifying which data collection tool to use?

Answer 35

• Look at similar studies conducted previously
• E.g. tools used to measure same concept/outcome, same/similar setting, same/similar patient/staff group/data
• And especially consult any relevant measurement reviews
• What is appropriate and high quality
• If participant-facing, age/capacity and vulnerability, language used in tool
• Psychometric properties – how well and how reliably does it measure the construct of interest?
• Again consult any relevant measurement reviews
• What’s freely available – being careful of restricted access/copyright
• Journal articles/theses sometimes provide full tool
• Author websites/Request from authors
• Measure compendium books

Question 36

When and how are Patient notes/record review and focus used for data collection?

Answer 36

• Often used for audit/service evaluation/quality improvement projects
• Limited number of types of information available
• Tailor your planned analysis to
• Your project objective AND
• The data available AND
• The skills within your team for analysis
• Identify/create a data collection tool to make sure you collect data completely and consistently
• Create a spreadsheet for your data
• NO PATIENT IDENTIFIERS can leave the clinical environment
• Use anonymous patient notes/data if possible
• If non-anonymous, anonymise as soon as is possible and thereafter use anonymous participant numbers
• Collect all data and immediately anonymise in one go if possible
• You cannot and should not attempt to download/copy/take actual notes/records (even if electronic)

Question 37

What is a Likert scale and how does it work, when is it used?

Answer 37

a scale used when looking at response from one extreme attitude to another

• Unipolar (neutral/nothing to one extreme) or bipolar scaling method (negative extreme to positive extreme)
• Use for Knowledge, Beliefs, Attitudes, Symptoms, Traits
• Assumes distances between each response are equal, that distance between each value is the same
• 7-point scale is a good choice – can be treated as continuous but not too overwhelming for respondent
• Usually multiple statements, all related to each other are posed
• Scores are summed across statements to form a scale – all items usually have equal weight

Question 38

What is the use of Statitstical tests?

Answer 38

• Test data from our sample population to make inferences about a wider population
• Test whether we can reject the null hypothesis of
• No difference
• No association/correlation
• Provide us with estimates of probability and precision
• Probability = p values
• Precision = confidence intervals
• Can provide us with (or we can calculate) estimates of the importance of effects we find
• Effect sizes e.g. correlation r, Cohen’s d, relative risk, odds ratio

Question 39

What is the T-test or ANOVA used for?

Answer 39

• Tests of difference (between groups) with continuous data
• Two groups = t-test
• More than two groups = ANOVA
• These tests compare the means between groups
• Example: weight loss (kg) for intervention versus control group in a lifestyle intervention Randomised Controlled Trial
• Provides a mean difference and a p value i.e. how likely mean difference would be observed if the two groups were actually alike (/null hypothesis was valid)

Question 40

What is the Chi-squared test used for?

Answer 40

• Tests of association (within or between group) with categorical/binary data
• Chi-square test
• This test compares the observed frequency of a variable at each level of another variable/between groups with the expected frequency (i.e. by chance)
• Example: association between diabetes (present/absent) and weight (e.g. over/normal)
• Provides a p value i.e. how likely frequencies would be observed if there was no association between the two variables (/null hypothesis was valid)

Question 41

What is Sustainability as a domain of Quality?

Answer 41

Sustainable = Outcomes for patient and populations / (Environmental + Social + Financial impacts)

• healthcare should aim to minimise the environmental, social and financial costs and maximise the health outcomes for patients and populations

Question 42

What are the WHO healthcare quality dimensions (6)

Answer 42

Effective - healthcare that is adherent to an evidence base and results in improved health outcomes for individuals and communities, based on need

Efficient - healthcare that maximises resource use and avoids waste

Accessible - healthcare that is timely, geographically reasonable, and provided in a setting where skills and resources are appropriate to medical need

Acceptable - healthcare that takes into account the preferences and aspirations of individual service users and the cultures of their communities

Equitable - healthcare that does not vary in quality because of personal characteristics such as gender, race, ethnicity, geographical location, or socioeconomic status

Safe - healthcare that minimises risks and harm to service users

Question 43

What is an Audit?

Answer 43

A way to find out if healthcare is being provided in line with standards and let care providers and patients know where their service is doing well and where there could be improvements

Question 44

What does the Donabedian Model show about the components of healthcare

Answer 44

• Structure → the facility is it accessible, have the BP monitors been serviced
• Process → what is actually done to the patient, are they seeing the right person? how long does it take them to see them
• Outcome → patient knowledge, patient behaviour

Question 45

What is the function of quality improvement?

4 steps

Answer 45

Primary intent— “To bring about measurable improvement to a specific aspect of healthcare delivery, often with evidence or theory of what might work but requiring local iterative testing to find the best solution”

Plan → Do → Study → Act

• Using the understanding of our complex healthcare environment
• Applying a systematic approach
• Designing, testing, and implementing changes using real-time measurement for improvement
• To make a difference to patients by improving safety, effectiveness and experience of care

Question 46

What may result in completing a Quality Improvement?

Answer 46

• cases - reflective case reviews
• data - large scale national audit, formal audit, review of personal outcome data, small scale data searches, information collection and analysis,
• events - learning event analysis (LEA) and significant event review
• feedback – improvement activities undertaken as a result of the outcomes of reflection on your formal patient and colleague feedback survey results, other solicited and unsolicited feedback, compliments and complaints.

Question 47

What’s the difference between research versus audit and quality improvement

Answer 47

• Research = Creating new knowledge that can be GENERALISED beyond the participant sample or setting
• Clinical Audit = Evaluating service against a BENCHMARK, e.g. national waiting times standard
• Service evaluation / Quality Improvement = Finds out something about THAT SERVICE which can (SE) or is (QI) used to IMPROVE THAT SERVICE
• Audit, service evaluation and quality improvement – knowledge created DOES NOT generalise beyond the service to other settings/populations
• Does not require research ethics

Question 48

What is a systematic review?

Answer 48

A systematic review is a summary of the medical literature that uses explicit and reproducible methods to systematically search, critically appraise, and synthesize on a specific issue. It synthesizes the results of multiple primary studies related to each other by using strategies that reduce biases and random errors

Question 49

When should the protocol of a systematic review be registered/ submitted for publication

Answer 49

Before the systematic review is conducted or its results are published

• a detailed protocol should be developed before registration
  
  • it should describe the rational hypothesis and planned methods

Question 50

What are the steps for developing a systematic review? (7)

Answer 50

1. Review question: clear detailed and specific question - not already answered or needs an update (PICO, SPIDER)
2. Search terms and strategy: check MedSH Medical Subject Headings, and pilot terms and see what is returned, consider relevant databases
3. Inclusion/exclusion criteria: usually selected at two stages, title/ abstract and full-text, pilot inclusion/ exclusion criteria, record all decision-making (RAYYAN)
4. Risk bias: assess the quality or risk of bias of all studies you include
5. Data extraction: create and cross-reference data extraction form
6. Synthesis methods: how you aggregate the findings from your initial studies to produce a bottom line (meta-synthesis, meta-analysis, narrative systematic synthesis)
7. Writing the report: use the protocol, what you did and didn’t do and why, PRISMA checklist, results, discussion

Question 51

What is meta-analysis? and how does it work?

Answer 51

• Statistical combination of at least 2 primary studies to produce a pooled single estimate of the effect under consideration
• Key steps- data extraction, statistical meta-analysis, consider heterogeneity, consider publication bias, interpret results
• Meta-analysis results are commonly displayed graphically as ‘forest plots’
• Binary variables expressed as ratios (risk ratio or odds ratio) between the two groups
• Continuous outcomes measures are usually expressed as ‘weighted mean difference (WMD)’ between the two groups
• Estimates of heterogeneity and precision - can explore predictors of variability in effect sizes
• Publication bias expressed with ‘funnel plot’

Question 52

What are the strengths of systematic reviews?

Answer 52

• Provides more robust and reliable conclusions than individual primary studies
• Core part of evidence-based medicine
  
  • Effectiveness of interventions
  • What works for whom, when and where

Question 53

What are the weaknesses of a systematic review?

Answer 53

• Limited by the availability and quality of evidence to synthesise
• Takes a lot of time (even rapid reviews are not that rapid!
• Meta-analysis/synthesis – require a lot of expertise
• Meta-analysis – necessary data may not be included in primary publications (and many studies not published at all)

Question 54

What is IMRaD as an acronym for writing a scientific report?

Answer 54

• _I_ntroduction – What is this about? What was the scientific question/ the known unknown? Why does it matter?
• _M_ethods – What was done?
• _R_esults – What was discovered? (and/or not discovered)
• _a_nd
• _D_iscussion – What new and compelling knowledge was found (or not)? Why does it matter? Did it answer the known unknown?