the anatomy and physiology of clinical research part 4 Flashcards
Learning objectives
Focus on the variables and statistical issues
variables - predictor and outcome, confounding, hypothesis
statistical errors - hypothesis, sample size and analytical approach
Understand the different types of variables in a clinical study / trial.
Understand the concept of hypothesis testing and sample size effects and how they are applied in a clinical study / trial.
Distinguish between accuracy and precision, and ways to improve accuracy and precision respectively.
Understand how statistical issues are encountered in a clinical study / trial.
Distinguish statistical significance from biological significance.
Appreciate and understand the applications of the abovementioned points to real-world scenarios.
2 types of variables
Continuous variables and Categorical variables
What are continuous variables
quantitative variables
2 Examples of quantitative variables
interval and ratio
What are discontinuous variable
qualitative variables
3 Examples of qualitative variables
nominal, dichotomous, ordinal
Interval (continuous variable)
Quantified on an infinite scale and which have a numerical value e.g. temperature, Fahrenheit
The scale can take on both positive and negative values
It is assumed that the intervals keep the same importance throughout the scale
This allows us not only to rank order the items that are measured but also to quantify and compare the magnitudes of differences between them
Ratio (continuous variable)
These are interval variables, but with the added condition that zero of the measurement indicates that there is none of the variable.
Hence temperature is not a ratio variable as 0 does not indicate that there is no temperature.
The name ratio reflects that you can use the ratio of measurements e.g. distance of 10 m is twice the distance of 5 m
Examples of ratio variables include height, weight, distance, number of cigarettes
Nominal (categorical variable)
These are variables that have 2 or more categories, but which do not have an intrinsic order
E.g. Blood type for human, there are 4 distinct groups, A, B, AB and O. So blood type is a nominal variable with 4 categories
They tend to have a qualitative and absolute character that makes them very straight forward to measure e.g. 35 people have blood type O
Dichotomous (categorical variable)
These are nominal variables which have only TWO categories or levels
E.g. gender, we would most likely categorize someone as “male” or “female”
E.g. “Dead” or “Alive”
E.g. do you own a mobile phone? Or do you have blood type O? the answer would be either a “Yes” or “No”
Ordinal (categorical variable)
Variables have 2 or more categories like nominal variables only that the categories can be ordered or ranked
E.g. Ranking the frequency of consumption of mala from not frequent to very frequent
What kind of variable is temperature
continuous variable (interval)
What kind of variable is weight
continuous variable (ratio)
What kind of variable is gender
category variable (dichotomous)
What kind of variable is stage of cancer
continuous variable (ordinal)
What kind of variable is eye color
continuous variable (nominal)
what kind of variable is blood type
continuous variable (nominal)
what kind of variable is education level
continuous variable (ordinal(
what kind of variable is satisfaction level
continuous variable (ordinal)
what kind of variable is birth weight
Continuous Variable (Dichotomous)
what kind of variable is bubble tea consumption
Continuous Variable (Ordinal
Continuous variables give more data than categorical variables
Continuous variables provide additional information, which helps to improve statistical efficiency compared to categorial variables.
Since continuous variables provide more information, a smaller sample size can be established and provides more meaningful results.
Even when categorical data is more meaningful, it is still a better option to collect data as continuous variables as it will leave the analytical option open for discussion.
BP in mmHG
continuous
normotensive vs hypertensive
categorical - dichotomous
low, normotensive, moderately hypertensive, severely hypertensive
ordinal
what is precision
The precision of a variable is the degree to which it is reproducible, with nearly the same value each time it is measured.
Precision has a very important influence on the power of a study (the ability of a study to detect a difference that is real), the more precise the measurement, the greater the statistical power at a given sample size to estimate mean values and to test hypothesis (proposed explanation for a phenomenon)
This is a function of random error, the greater the error, the less precise the measurement
//
Precision represents how close different sample measurements you take are to one another, and accuracy represents how close those sample measurements are to the true measurement
//
Precision has a very important influence on the power of a study (the ability of a study to detect a difference that is real)
The more precise the measurement, the greater the statistical power at a given sample size to estimate mean values and to test hypothesis (proposed explanation for a phenomenon)
This is a function of random error, the greater the error, the less precise the measurement
what is accuracy
The accuracy of a variable is the degree to which it actually represents what it is intended to represent.
This has an important influence on the validity of the study (the degree to which the observed findings lead to the correct inferences about phenomena taking place in the study sample and in the universe)
Accuracy is a function of systematic error (bias), the greater the error, the less accurate the variable. The three main classes of measurement error noted for precision each have their counterparts here.
//
Accuracy is a function of systematic error (bias), the greater the error, the less accurate the variable. The three main classes of measurement error noted for precision each have their counterparts here.
Observer bias – is a distortion, conscious or unconscious, in the perception or reporting of the measurement by the observer. It may represent systematic errors in the way an instrument is operated e.g. the tendency to round down measurements, or in the way an interview is carried out e.g. the use of leading questions etc.
Instrument bias – this can result from a faulty function of a mechanical instrument. A scale that is not calibrated recently may have drifted downward, producing consistently low BW readings
Subject bias – is a distortion of the measurement by the study subject, e.g. reporting an event. Patients with lung cancer who believe that the number of cigarettes smoked per day is the cause of the disease would exaggerate the number of cigarettes they smoke per day
The accuracy of a measurement is best assessed by comparing it, when possible to a “gold standard”. This is a reference technique that is considered to be accurate.
Illustration of accuracy
Accuracy can be illustrated by throwing the darts onto the centre of the target (the true value).
illustration of precision
Precision can be illustrated by throwing the darts closer to each other.
the 3 sources of random errors
Observer variability – variability in the measurement that is due to the observer and includes aspects such as skill in using a mechanical instrument, methods of asking questions and conducting interviews etc.
Instrument variability – variability in the measurement due to changing environmental factors e.g. temperature, aging mechanical components, different reagent, different brand etc.
Subject variability – intrinsic biologic variability in the study subjects due things such as mood, time of last medication, food intake etc.
what are random errors
Random errors are errors that affect the precision of a measurement. Random errors are two-sided errors, because, in the absence of other types of errors, repeated measurements yield results that fluctuate above and below the true or accepted value.
Measurements subject to random errors differ from each other due to random, unpredictable variations in the measurement process.
5 ways to improve precision
- standardizing measurement methods - written directions and guidelines, creating SOP,
- training and certifying observers - improve the consistency of measurements technique
- refining instruments - mechanical and electronical instruments
- automating instrument - the influence of random error is reduced by repeating the measurements and using the mean of 2 or more readings
Standardizing the measurement methods
All the study protocols should include operational definitions (specific instructions for making the measurements)
This includes written directions on how to prepare the environment and the subject, how to carry out and record the interview etc.
The operational manual is essential for large and complex studies and recommended for smaller ones.
Specific written guidelines for making each measurement will help the observer’s performance and ensure uniformity over the duration of the study.
This will also serve as the basis for describing the methods when the results are published
Training and Certifying the Observers
Training will improve the consistency of measurement techniques especially when several observers are involved.
It is often desirable to design a formal test of the mastery of techniques specified in the operations manual and to certify that observers have achieved the prescribed level of performance
Refining the instruments
Mechanical and electronic instruments can be engineered to reduce variability
Questionnaires and interviews can also be written in a certain style/manner to avoid potential ambiguities and to increase clarity
Automating the instruments
The use of automated mechanical devices and self-response questionnaires can reduce observer variability by eliminating variations in the way human observers make measurements
Repetition
The influence of random error from any source is reduced by repeating the measurement and using the mean of the two or more readings.
Precision will be substantially increased by this strategy
The limitation being the added cost and practical difficulties of repeating the measurements
Making Unobtrusive Measurements
It is sometimes possible to design measurements that the subjects are not aware of, thereby eliminating the possibility that they will consciously bias the variable.
Example: A study on healthy eating patterns for school children, could measure the number of candy bar wrappers in the trash rather than questioning each child on the number of candy he or she consumes
Calibrating the instruments
The accuracy of many instruments, especially those that are mechanical or electrical can be increased by periodic calibration using a gold standard
Blinding
This is intended to ensure that subjective assessments and decisions are not affected by knowledge of treatment assigned.
Different groups should be treated and observed in the same way during a trial
Clinical trials are often “double blinded” to ensure that subject and investigators (as well as investigator staff) involved in the treatment or clinical evaluation of subjects are unaware of the subject’s assigned treatment.
Blinding is intended to minimize the potential biases resulting from differences in management, treatment or assessment of patients or interpretation of results that could arise as a result of subject or investigator knowledge of assigned treatment
Examples include
Subjects on active drug might report more favorable outcomes, because they expect a benefit or might be more likely to stay in a study if they knew they were on active drug
Observers might be less likely to identify and report treatment responses in a non-treatment group or might be more sensitive to a favorable outcome or adverse event in patients receiving active drug
Hypothesis testing
Research hypothesis is a specific version of the research question that summarizes the main elements of the study
Hypothesis testing is needed for studies that will use test of statistical significance to compare findings among groups
Most observational studies
ALL experimental research questions involved in making comparisons
Stating the hypothesis in advance would help the researcher and research efforts focused on the primary objective
Null hypothesis
States that there is NO association between the predictor and outcome variables in the population
Alternative hypothesis
The proposition that there is an association between predictor and outcome variable
Statistical significance
Statistical significance means that there is a good chance that we are right in finding that a relationship exists between two variables.
But statistical significance is not the same as practical significance. We can have a statistically significant finding, but the implications of that finding may have no practical application. The researcher must always examine both the statistical and the practical significance of any research finding.
A one-sided hypothesis
Specifies the direction of the association between the predictor and outcome variable.
A one sided hypothesis may be appropriate when there is a very strong evidence from prior studies that an association is unlikely to occur in one of the two directions
A two-sided hypothesis
States only that an association exists and does not specify the direction.
This is usually the case as both sides of the alternate hypothesis (i.e. greater or lesser) are interesting and investigators would want to publish results no matter which direction is observed
if they ask study design
they ask for a case control study
type 1 error
Reject Null Hypothesis when the Null Hypothesis is TRUE
False Positive
Probability of committing a Type 1 Error is assigned α
α is often set at 5% or 0.05 with a conventional range of 0.01 – 0.10.
Type 1 Errors are generally more serious than Type 2 Errors.
type 2 error
Fail to reject the null hypothesis when the null hypothesis is false i.e. there is actually an association between predictor and outcome variable
False negative
Probability of committing a Type 2 Error is assigned β
β is often set at 0.2
Power
defined as the ability to prove that there is a difference where there is one.
power is defined as the probability of correctly rejecting the null hypothesis when it is actually false
Power formula
Power can be determined by subtracting the Type II error (β) from the total probability of 1.
The minimum value for power
80%
If power is not sufficiently large
then you will not be able to detect a difference even though it is significant
example of power
If the investigator sets β to 0.10, it means that the investigator is willing to accept a 10% chance of missing an association of given effect size if it exists.
This represents a power of 0.90 i.e. 90% chance of finding an association of that effect size or greater.
If the investigator sets a higher β value (e.g. 0.50), then there is only a 50% chance of finding an association.
sample size affects power
Even if your effect size is large but your sample size small, the power of the study may be poor and you might not be able to detect a treatment effect when there is actually one
definition of effect size
The size of association that is expected to be present in the sample = Effect size
The likelihood that a study will be able to detect an association between a predictor and an outcome variable in a sample depends on the magnitude of that association in the population.
A larger effect size will make it easier to detect an association between the predictor and outcome variable, and vice versa.
choosing an effect size
Choosing an Effect Size
Often investigators do not know the exact size of the association and would have to make an estimate
Make an informed guess about a reasonable effect size
In addition to determining the effect size, pilot studies may also investigate how to design recruitment approaches, measurements and interventions
Larger the effect size increase ease in detecting an association between predictor and outcome variable
Smaller the effect size increase difficulty in detecting an association between predictor and outcome variable
Effect size would affect the sample size i.e. smaller the effect size, the larger the sample size required to obtain statistically significant results
effect size and sample size
Effect size has an inverse relationship with the sample size
A smaller effect size will require a larger sample size to obtain statistically significant results, and vice versa.
Statistical vs Biological Significance
SMALL treatment differences which are biologically UNIMPORTANT can be statistically significant if sample size is large.
LARGE treatment differences which are biologically IMPORTANT may not be statistically significant if sample size is small.
