Stats - clinical trials and validity Flashcards
What is the purpose of the Phase 1 trial?
How many people are involved?
Phase I clinical trials involve only a small number of healthy people (possibly as few as 15-20). The focus is to evaluate the drugs safety, determine a safe dosage range, and identify side effects.
What is the purpose of the Phase 2 trial?
How many people are involved?
In Phase II clinical trials the drug or treatment is given to a larger group of people (100-300) to see if it is effective and to further evaluate its safety
What is the purpose of the Phase 3 trial?
How many people are involved?
In phase III trials the drug or treatment is given to large groups of people (1,000-3,000) to confirm its effectiveness, monitor side effects, compare it to commonly used treatments (or placebos), and collect information that will allow the experimental drug or treatment to be used safely.
What is the purpose of the Phase 4 trial?
Phase IV trials (aka Post Marketing trials) are done after the drug has been granted a license. They gather further information on the drug in areas not addressed in the previous trials (e.g. Safety in pregnancy) and also to find other potential uses for the drug.
Validity refers to the extent to which something measures what it claims to measure. The first major distinction is between internal and external validity.
What is internal validity?
Internal validity is the confidence that we can place in the cause and effect relationship in a study. It is the confidence that we have that the change in the independent variable caused the observed change in the dependent variable (rather than due to poor control of extraneous variables)
Validity refers to the extent to which something measures what it claims to measure. The first major distinction is between internal and external validity.
What is external validity?
External validity is the degree to which the conclusions in a study would hold for other persons in other places and at other times, i.e. its ability to generalise
3 key features are:
1) Generalization: Extends findings to wider populations.
2) Relevance: Ensures practical applicability in real-world settings.
3) Transferability: Facilitates replication of results across varied contexts.
What is the difference between validity and reliability?
Validity refers to the extent to which something measures what it claims to measure.
Reliability is the extent to which an experiment, test, or any measuring procedure yields the same result on repeated trials.
Internal validity can be threatened by various factors that introduce bias or confounding effects.
What are 8 threats to internal validity?
Selection bias
History Effect
Maturation Effect
Testing Effect
Instrumentation
Regression to the mean
Attrition (mortality)
Diffusion of Treatment
What are 6 threats to external validity?
Sample Bias
Hawthorne effect
Setting effects
Novelty effect
Temporal factors
Interaction effects
What are 6 ways to improve internal validity?
Step 1: Design an Appropriate Control Group
Including a control group helps distinguish the effects of the independent variable from other factors. Control groups offer a baseline for comparison and allow researchers to control for natural changes over time, such as maturation or history effects.
Step 2: Randomly Assign Participants to Groups
Random assignment reduces the likelihood of selection bias by ensuring that each participant has an equal chance of being in any group. This creates groups that are comparable and reduces the influence of confounding variables.
Step 3: Standardize Procedures
Consistency in measurement tools, procedures, and data collection methods is crucial. This helps prevent instrumentation threats and ensures that all participants experience the same conditions, aside from the manipulation of the independent variable.
Step 4: Blinding and Placebos
In studies involving human subjects, blinding participants and experimenters to group assignments can prevent biases in treatment administration and data collection. Using placebos in control groups helps isolate the effect of the independent variable from psychological or expectation-related effects.
Step 5: Pre-Test and Post-Test Design with Care
When using pre-test and post-test designs, be aware of potential testing effects. Alternative testing forms or post-test-only designs can minimize the influence of repeated testing on participants’ responses.
Step 6: Monitor and Address Attrition
Track participant dropouts and analyze whether they differ significantly from those who remain in the study. Use statistical methods to account for missing data, ensuring that attrition does not bias results.
What are 6 ways to improve external validity?
- Use Representative Samples
Selecting a diverse and representative sample of the target population improves the likelihood of generalization.
Tip: Use random sampling to avoid selection bias.
- Conduct Field Studies
Studies conducted in natural settings, rather than artificial laboratory environments, enhance ecological validity.
Example: Observing customer behaviour in a supermarket instead of a simulated store.
- Replicate Studies
Replication in different settings, populations, and times ensures that findings are robust and generalizable.
Example: Testing a psychological theory across cultures to validate its universality.
- Increase Sample Diversity
Include participants from varied demographic, socioeconomic, and geographic backgrounds to ensure broad applicability.
Tip: Avoid over-reliance on WEIRD (Western, Educated, Industrialized, Rich, Democratic) samples.
- Test Across Settings and Times
Evaluate the intervention or phenomenon in multiple contexts and at different times to account for variability.
Example: Assessing a public health campaign’s effectiveness in rural and urban settings over several years.
- Acknowledge Limitations
Clearly state the contexts in which the findings apply and discuss potential barriers to generalization.
Tip: Address limitations in the discussion section of your study.
What type of validity is this?
This refers to the general impression of a test. A test has this validity if it appears to test what it is meant to
Face validity
What type of validity is this?
This refers to the extent to which a test or measure assesses the full content of a subject or area. For example if a test is designed to help diagnose depression, it would have poor [X-type] validity if it only asked about psychological symptoms and neglected biological ones
Content validity
What type of validity is this?
This concerns the comparison of tests. You may wish to compare a new test to see if it works as well as an old, accepted method. The correlation coefficient is used to test such comparisons
Criterion validity
What type of validity is this? (Sub category)
The predictor and criterion data are collected at or about the same time. An example could be testing a new, shorter test of intellectual functioning against a standard measure
Criterion (concurrent)
