Ch. 11: Reasoning About the Design and Execution of Research Flashcards
defn: the scientific method
a set of steps that define the appropriate order of events to structure and carry out an experiment
the established protocol for transitioning from a question to a new body of knowledge
what are the 8 steps of the scientific method?
- generate a testable question
- gather data and resources
- form a hypothesis
- collect new data
- analyze the data
- interpret the data and existing hypothesis
- publish
- verify results
explain why step 1 of scientific method (generate a testable question) happens?
usually occurs after observing something anomalous in another scientific inquiry or in daily life
defn: hypothesis
the proposed explanation or proposed answer to our testable question
often in the form of an if-then statement
defn: experimentation vs. observation
experimentation: involves manipulating and controlling variables of interest
observation: often involves no changes in the subject’s environment
explain: step 6 of the scientific method (interpret the data and existing hypothesis)
consider whether the data analysis is consistent with the original hypothesis
if the data is inconsistent, consider alternative hypotheses
explain: step 8 of the scientific method (verify results)
most experiments are repeated to verify the results under new conditions
most questions that begin with what word are too broad to be testable through a single experiment?
WHY
what does the if-then format of a hypothesis ensure?
that it is testable
func + defn: FINER method
for evaluating a research question
a method to determine whether the answer to one’s question will add to the body of scientific knowledge ini a practical way and within a reasonable time period
what are the 5 questions of the FINER method?
- Is the necessary research study going to be FEASIBLE?
- Do other scientists find this question INTERESTING?
- Is this particular question NOVEL?
- Would the study obey ETHICAL principles?
- Is the question RELEVANT outside the scientific community?
what are 4 feasibility concerns that we consider in the FINER method?
- obtaining necessary supplies
- financial constraints
- time constraints
- the inability to gather enough subjects
what is the reason for controls in basic science research?
we use controls because in order to make generalizations about our experiments, we must make sure that the outcome of interest would not have occurred without our intervention
aka: control
standard
defn: positive controls + example
those that ensure a change in the dependent variable when it is expected
example: in the development of a new assay for detection of HIV, administering the test to a group of blood samples known to contain HIV could constitute a positive control
defn: negative controls + example
ensure no change in the dependent variable when no change is expected
example: the same assay as above, administering the test to a group of blood samples known NOT to contain HIV could constitute a negative control
what is a negative control group often used for in drug trials?
to assess for the placebo effect
defn: placebo effect
an observed or reported change when an individual is given a sugar pill or sham intervention
do we manipulate or measure/observe the independent or dependent variable?
MANIPULATE: independent variable
MEASURE/OBSERVE: dependent variable
what is another big advantage to being able to manipulate all of the relevant experimental conditions?
basic scientific researchers can often establish causality
what relationship must there be between independent and dependent variables for causality to be investigated?
when there is a theoretical or known mechanism that links the independent and dependent variables
what relationship must there be between independent and dependent variables for causality to claimed?
if the change in the independent variable always precedes the change in the dependent variable, and the change in the dependent variable does not occur in the absence of the experimental intervention
there is minimal experimental bias in basic science research, but what are three ways that bias can appear?
- generation of a faulty hypothesis from incomplete early data and resource collection
- eliminating trials without appropriate background
- failing to publish works that contradict the experimenter’s own hypothesis
defn: accuracy (+aka) vs. precision (+aka)
ACCURACY = validity = the ability of an instrument to measure a true value
PRECISION = reliability = the ability of the instrument to read consistently or within a narrow range
explain the difference between accuracy and precision by describing a person weighing themselves on a scale
170 lb person
ACCURATE but IMPRECISE scale = readings between 150-190 lbs
INACCURATE but PRECISE scale = readings between 129-131 lbs
will an inaccurate or imprecise tool introduce bias or error? why?
bias is a SYSTEMATIC ERROR
so only an INACCURATE tool will introduce bias, but an IMPRECISE tool will still introduce error
defn: random error
error introduced by random chance
how do we avoid random error?
usually overcome by a large sample size
defn: randomization
the method used to control for differences between subject groups in biomedical research
uses an algorithm to determine the placement of each subject into either a control group that receives no treatment or a sham treatment or one or more treatment groups
what will a proper randomization algorithm be equal to?
a coin toss or die roll
defn + func: blinded
the subjects and/or investigators do not have information about which group the subject is in
to remove bias
defn: single-blind experiments
only the patient or the assessor is blinded
defn: assessor
the person who makes measurements on the patient or performs subjective evaluations
defn: double-blind experiments
the investigator, subject, and assessor all do not know the subject’s group
how does the placebo effect differ between the control and treatment group WITHOUT blinding?
WITHOUT blinding, the placebo effect would be greatly reduced in the control group, but still be present in the treatment group
examples: binary vs. continuous vs. categorical variables
BINARY (yes vs no, better vs worse)
CONTINUOUS (amount of weight lost, percent improvement in cardiac output)
CATEGORICAL (state of residence, socioeconomic status)
what are the three types of observational studies?
- cohort
- cross-sectional
- case-control
what do observational studies often look at?
the connections between exposures and outcomes
can observational studies demonstrate causality?
no, although the tendency toward causality can be demonstrated by Hill’s criteria
defn: cohort studies
those in which subjects are sorted into groups based on different risk factors (exposures) and then assessed at various intervals to determine how many subjects in each group had a certain outcome
defn: cross-sectional studies
attempt to categorize patients into different groups at a single point in time
defn: case-control studies
start by identifying the number of subjects with or without a particular outcome, and then look backwards to assess how many subjects in each group had exposure to a particular risk factor
defn: Hill’s criteria
describe the components of an observed relationship that increase the likelihood of causality in the relationship
are all of the Hill’s criteria necessary for a relationship to be causal?
no, only the first is necessary, but it is not sufficient
the more criteria that are satisfied by a relationship, the likelier it is that the relationship is causal
why should relationships be described as correlations not causation for an observational study?
Hill’s criteria do not provide any absolute guideline on whether a relationship is causal
what are the 9 Hill’s criteria?
- temporality
- strength
- dose-response relationship
- consistency
- plausibility
- consideration of alternative explanations
- experiment
- specificty
- coherence
defn: temporality (Hill’s criteria)
the exposure (independent variable) MUST occur before the outcome (dependent variable)
defn: strength (Hill’s criteria)
as more variability in the outcome variable is explained by the variability in the study variable, the relationship is more likely to be causal
defn: dose-response relationship (Hill’s criteria)
as the study or independent variable increases, there is a proportional increase in the response. the more consistent this relationship, the more likely it is to be causal
defn: consistency (Hill’s criteria)
the relationship is found to be similar in multiple settings
defn: plausibility (Hill’s criteria)
there is a reasonable mechanism for the independent variable to impact the dependent variable supported by existing literature
defn: consideration of alternative explanations (Hill’s criteria)
if all other plausible explanations have been eliminated, the remaining explanation is more likely.
defn: experiment (Hill’s criteria)
if an experiment can be performed, a causal relationship can be determined conclusively
defn: specificity (Hill’s criteria)
the change in the outcome variable is only produced by an associated change in the independent variable
defn: coherence (Hill’s criteria)
the new data and hypothesis are consistent with the current state of scientific knowledge
defn: bias vs. confounding
bias: a result of flaws in the data collection phase of an experimental or observational study
confounding: an error during analysis
defn: selection bias
subjects used for the study are not representative of the target population
defn: detection bias + example
results from educated professionals using their knowledge in an inconsistent way (i.e. because prior studies have indicated that there is a correlation between two variables, finding one of them increases the likelihood that the researcher will search for the second)
example: doctors may screen patients who are obese for hypertension and diabetes at a higher rate than other patients, inflating the true value of the secondary measurement
defn + aka: Hawthorne effect + why is this an example of bias?
aka: observation bias
defn: posits that the behavior of study participants is altered simply because they recognize that they are being studied (often these lifestyle alternations improve the health of the sample population)
this is an example of bias because the change in data is systematic and occurs before data analysis
defn: confounding
a data analysis error - the data may or may not be flawed, but an incorrect relationship is characterized
what is confounding inaccurately described as?
confounding bias or omitted variable bias
what are the four core ethical tenets of medicine?
- beneficence
- nonmaleficence
- respect for patient autonomy
- justice
defn + aka: confounding variables
aka: confounders
defn: third-party variables that are the actual “cause” of a seemingly causal relationship between two variables
defn: beneficence
the obligation to act in the patient’s best interest
defn: nonmaleficence
the obligation to avoid treatments or interventions in which the potential for harm outweighs the potential for benefit
defn: respect for patient autonomy
the responsibility to respect patients’ decisions and choices about their own healthcare
defn: justice
the responsibility to treat similar patients with similar care and to distribute healthcare resources fairly
what are the three necessary pillars of research ethics and what document was this determined by?
- respect for persons
- justice
- a slightly more inclusive version of beneficence
the Belmont Report
defn: respect for persons
the need for honesty between the subject and the researcher and generally, but not always, prohibits deception
also includes the process of informed consent, no coercion, respect the patient’s wishes to continue or cease participation, and confidentiality
defn: informed consent
a patient must be adequately counseled on the procedures, risks and benefits, and goals of a study to make a knowledgeable decision about whether or not to participate in the study
func: institutional review boards
put into place systematic protections against unethical studies
defn: vulnerable persons
children, pregnant individuals, and prisoners
they require special protections above and beyond those taken with the general population
defn: justice in research
applies to both the SELECTION of a research topic and the EXECUTION of the research
defn: morally relevant differences + examples + non-examples
those differences between individuals that are considered an appropriate reason to treat them differently
examples: age, population size, likelihood of benefit
non-examples: race, ethnicity, sexual orientation, gender identity, disability status, and financial status
what is an example that may or may not be considered a morally relevant difference?
religion (to keep patient autonomy)
what is the difference in the groups of people involved in studies when there is no perceived difference in the likelihood of benefit and when there IS a perceived difference?
NO difference: all individuals should assume equal risk
YES difference: the population that is most likely to benefit should assume a higher proportion of risk
in drug trials, can risk be put on a group that does not have the illness?
yes, as long as it has been address through informed consent and respect for persons has been maintained
defn: beneficence (research)
it must be our intent to cause a net positive change for both the study population and general population and we must do our best to minimize any potential harms
research should be conducted in the least invasive, painful, or traumatic way possible
defn: equipoise
one cannot approach the research with the knowledge that one treatment is superior to the other
if it becomes evident that one treatment option is clearly superior before a study is scheduled to finish, what should happen?
the trial must be stopped because providing an inferior treatment is a net harm
defn: population
the complete group of every individual that satisfies the attributes of interest
defn: parameter
information that is calculated using every person in a population
defn: sample
any group taken from a population that does not include all individuals form the population
ideally representative of a population
defn: statistic
information about a sample
can be used to estimate population parameters (with large or repeated samples)
defn: internal validity vs. external validity
INTERNAL validity = support for causality
EXTERNAL validity = generalizability
defn: high vs. low generalizability studies
LOW: very narrow conditions for sample selection that do not reflect the target population
HIGH: have samples that are representative of the target population
what is an implication of the fact that we are interested in applying research to our patients?
we need to consider whether the data is sufficient for the recommendation or exclusion of any therapy or treatment plan
defn: statistically significant
not the result of random chance
