Week 9: RCT Flashcards
What are randomized controlled trials (RCT’s)?
Why?
• Assign each subject randomly to one of two groups
o Subjects are unaware which they are in (i.e. “blind”)
• One group receives the treatment of interest, one receives a placebo (e.g. a sugar pill)
o Both treatments are indistinguishable
• Experiments are often “double blind”
o Practitioner also unaware of groupings
Why use RCTs?
• Subjects are assigned randomly
o No regularity, smooths out variations in subjects
• Allows comparison of treatment results, isolating for the individual
o With enough subjects, fluctuations even out
• Reduces selection bias
o i.e. Doctors can’t select patients expected to show favourable results
• Reduces confounding factors
What was Fechner’s Psychophysical survey?
- Fechner studied human responses to external stimuli
- Performed experiments in 1850’s involving human’s sense of weights
Identified a Threshold “T”
• Model the weight sensed by a human as a Gaussian random variable, h(W)
o T = ½ (WA+WB)
• From this theory, you could calculate the probability of a measuring error
o P(error) = P(h(WA) > T) + P((WB) < T)
What was the Peirce-Jastrow Experiment? What were the key features of its implementation? What work was it based on?
Into the 1860’s and 1870’s, this idea of a “threshold” gained popularity. But Fechner’s theory remained unproven, so researchers tried to test it. One such researcher was Charles Sanders Peirce
In 1884 he tested Fechner’s theory with student Joseph Jastrow: The “Peirce-Jastrow Experiment”
Peirce & Jastrow introduced the two major ideas:
- Put a screen between the subject and the experimenter
- Use a mechanism for the experimenter to adjust the weight applied to the subject
They tested using the following two test cases:
- Subject initially has 1kg weight applied. Experimenter then adds an extra small weight, and finally removes it
- Subject initially has 1kg + small weight applied. Experimenter then removes the small weight, and finally adds it back
One of the two cases was selected by drawing a card, and then the subject must answer the following:
- Which case it was
a. i.e. Was the weight reduced and then increased, or increased and then reduced? - How confident is the user?
a. On scale from 0 to 3
• Conclusion: There is no threshold for sensory differentiation
The Peirce-Jastrow experiment was an important first step, in that it INTRODUCED RANDOMIZATION
• Prevented user from guessing the case, leading to more reliable results
However, this was still very early work. For example, the sample size was just two (Fechner and Jastrow)
Weaknesses in Fechner’s theory?
- Fechner didn’t specify at all how an experiment should be carried out – it was just a theory
- Problem: subject might know the actual weights of the objects being compared
Was actually unproven
What experiment introduced randomization?
Peirce-Jastrow, via the card to select the experiment
What was telepathy like in 19th C?
In the 19th century, a major area of research was telepathy
• Many intellectuals believed in spiritualism by late 19th century
o The “inner spiritual power” of humans
• Would study its effects, such as the weight of souls, or the sensing of distant objects
• There were many societies all over Europe and the U.S.
What was Richet’s experiment?
Charles Richet (Paris, 1884)
• Performed random trials involving telepathy
Richet performed the following experiment:
• Consider two people – One is the “Agent” and one is the “Reagent”
o They are physically separated (i.e. in different rooms)
• Agent randomly draws a card from a deck, and focuses on it
• Reagent must guess the Agent’s card
After 3000 trials, results were inconclusive
• Reagent’s success rates were only slightly better than random guessing
• While inconclusive, it paved the way for more involved study.
What was Coover’s experiment? How did it relate to the modern RCT?
Psychologist John Edgar Coover
• By the early 20th century, psychophysics was somewhat out of fashion
• However, its founder Leland Stanford had stipulated that the University conduct psychic research
In the 1920’s, Coover performed a very thorough telepathic experiment
• Like Richet, identify an Agent who draws cards and a Reagent who guesses
• The Twist: Added further randomization through a coin toss
o Toss first decides if the Agent draws a card at all
This is the first implementation of a control group
Provides a reference effectiveness of the Agent
o If a card is chosen, toss also dictates how the Agent should focus on the card
Visually, kinaesthetically, or both?
• Reagent is completely unaware if the Agent drew a card, or if they did how they were focusing on it
o Just told to determine the card
• Performed >10 000 trials with hundreds of subjects: no trace of telepathy found
Coover’s experiment introduced key elements of the modern RCT:
1. Control (Agent’s die toss)
2. Blind (reagent unaware of test’s nature)
3. Randomized (Agent draws random card)
Who and what laid the groundwork for the RCT?
Fisher’s “The Design of Experiments” (1935)
• A leading English statistician, Fisher was a pioneer of the modern synthesis
o Recall: combination of Darwinian Evolution & Medelian Inheritance
• Fisher worked at the British Rothamsted Experimental Station (1914-1935)
o Research revolving around agriculture – e.g. methods of improving crop yields
• In his book, The Design of Experiments, he lays out the groundwork and advantages of RCTs
What is the Null hypothesis?
• We call this the “Null Hypothesis”
o Case where the signal we seek does not exist
o i.e. there is no relationship between the lady’s selection and the method of preparation
• We can never eliminate the Null hypothesis – in every test it’s possible that there’s no relationship
• So, we must ensure the Null hypothesis is very unlikely to provide a reliable test
o In this case, the probability of the Null hypothesis (or its “Level of Significance”) is 1/70
o Fisher reasoned that 1/20 was good enough (i.e. 2σ)
• Consider: What if we only filled two cups? (one of each preparation)
o The Lady has a 50% chance to get it right, even if she does not have the ability
o Thus, the experiment is not good enough to disprove the Null hypothesis
What was the first medical RCT?
¬The first RCT came in 1948, from the British Medical Research Council
• Wanted to prove the efficacy of an expensive Tuberculosis drug before purchasing it (Streptomycin)
Their qualifications were notable because they included both a trial and a control group
• Initially, they used alternate assignment to bin the patients
o i.e. line everyone up, every other person goes in a group
• Many saw alternate assignment as unfair
• Austin Bradford Hill, an English pioneer of the RCT, suggested randomization
• Successfully proved the drug’s efficacy
What paved the way for RCT as the gold standard?
• The Thalidomide crisis (1957) boosted the accepted reliability of RCTs
o Morning sickness drug which caused an epidemic of stillbirths
Issues with RCT?
• RCT is good for when you’re ignorant of experimental phenomena
o E.g. telepathy – no idea of the underlying mechanisms
o If you understand the phenomena at hand though, RCTs may not be the best method of experimentation
• Some experiments can’t have a placebo/control group, so blindness can’t be guaranteed
o Faking surgery is not as easy as giving a sugar pill
• Goes against “personalized medicine”
o Does not account for patient’s variances
o Are the placebo groups being robbed; given false treatments despite knowing they have real medical issues?
• Huge costs create monopolies on clinical research
o If RCT is the only accepted gold standard, we lock out competition
• Transnational inequality
o RCT typically performed in developing countries
o But treatments/benefits go to the rich countries
