Evidence Base + DiD & RD

Question 1

What are the three components to classification of data science?

Answer 1

Description: Using data to provide a quantitative summary of certain features of the world

Prediction: using data to map some features of the world to other features of the world

Counterfactual prediction: using data to predict features of the world as if the world had been different

Question 2

What is often required for description, prediction, and counterfactual prediction?

Answer 2

Statistical inference

Question 3

What is the regression equation?

Answer 3

Y = alpha + beta(x) + episilon

Question 4

What does epsilon represent in the regression equation?

Answer 4

Random Error term (tries to correct for the “other stuff”)

Question 5

What does beta represent in the regression?

Answer 5

The change in y due to x (duh)

Question 6

What are common regression models?

Answer 6

• linear regressions (super common)
• logistic regression (nonlinear relationships)
• two-part model

Question 7

Does a regression prove causation?

Answer 7

No…duh

Question 8

What are some of the shortcomings of regressions?

Answer 8

• can be subject to reverse causation, simultaneity, or third factor causation
• regression is inconsistent if dist of estimates does not converge to true value as sample grows
• bias and inconsistency lead to false + and/or -

Question 9

What is a CDAG (Casual Directed Acyclic Graph)?

Answer 9

Way to guide regression analysis as it shows the director of hypothesized causal effects

Question 10

What can DAGs include?

Answer 10

Exposure (E): independent variable
Outcomes (O): dependent variable
Mediator (M): mechanism that transmits effect from E to O
Confounder (C): causer of both E and O
Collider (S): something caused by both E and O

EOMCS - everyone owes me cash sir

Question 11

What is done with mediators and colliders?

Answer 11

Mediators - usually controlled for or analyzed w/ careful mediation analysis
Colliders - hard to deal with but often used to help think of limitations to the study

Question 12

What are ways to deal with data missing at random

Answer 12

• complete case analysis
• last observation carried forward
• mean value imputation
• random imputation

Question 13

What is internal validity?

Answer 13

Stat concept that requires any inferences about casual effects to be valid in the population

English: Degree to which findings can be attributed to an independent variable

Question 14

What is external validity?

Answer 14

That findings can be generalized from the population and setting to other populations and settings

Question 15

How randomized is health care policy?

Answer 15

Like never lol (20% of studies are)

Question 16

Do health policies usually make an evaluation plan before rolling out?

Answer 16

Nope

Question 17

What are two key changes that need to be made to improve data quality in healthcare?

Answer 17

• reducing measurement burden by developing systems that autocollect date
• favor electronic health records

Question 18

What do randomized control trials get rid of that make it so great?

Answer 18

selection bias

Question 19

What was the Rand Health Insurance Experiment?

Answer 19

An experiment from 1974-1982 in which individuals were randomized to various insurance plans w/ diff levels of cost sharing

Free care lowered blood pressure, improved hyperptension, and reduced serious symptoms among poor patients

Question 20

What was the Oregon Medicaid Lottery?

Answer 20

Granted adults the opportunity to apply for Medicaid in 2008, Finkelstein & Baicker found that health care utilization increased, and self reported health increased on average. However, ED visits increased

Question 21

Why aren’t RCTs more common in health studies?

Answer 21

They can be infeasible, invalid, or not generalizable

Question 22

What kind of experiments are great when randomization isn’t possible or ethical?

Answer 22

Natural or Quasi Experiments!

Question 23

What are the 3 Quasi-Experimental Approaches?

Answer 23

• DiD
• Regression Discontinuity
• Instrumental variables (Uses an exogenous element that nudges some subject towards receiving a certain treatment like a lottery)

Question 24

What are the 3 Quasi-Experimental Approaches?

Answer 24

• DiD
• Regression Discontinuity
• Instrumental variables (Uses an exogenous element that nudges some subject towards receiving a certain treatment like a lottery)

Question 25

What are some examples of things that were breakthroughs that didn’t come out of RCTs?

Answer 25

• hand washing reduces infant mortality
• smoking increases the risk of lung cancer
• repeated head injury may CTE

Question 26

RCTs are the best approach in terms of ___ validity, but ___ validity is another matter

Answer 26

Internal, external

Question 27

Describe the “gold standard” RCT

Answer 27

• trial: experimenter controls elements of study design
• control: one group that receives treatment and the other doesnt
• randomized: no selection bias, observed difference in outcomes = treatment effect + selection bias

Question 28

Describe a quasi-experimental design (non-randomized control studies)

Answer 28

• trial turns into study: experimenter no longer has direct control of the setting
• control: still a treatment and control
• non-randomized: selection bias is a potential issue

Question 29

What settings are common for the use of Difference in Differences?

Answer 29

Estimating the impact of a policy change on some outcome of interest, where the policy change occurs at some specific point in time

Question 30

Why can’t you just compare health status before and after a health policy?

Answer 30

Other things could have happened that affected health besides just the policy

Question 31

What is the actual difference in differences?

Answer 31

The change in the treatment group minus the data from an alternate reality where the policy didnt happen

Question 32

What kind of data is required for DiD?

Answer 32

Panel data (units over time)

Question 33

What is the key idea that allows DiD to find the effect of policies on healthcare?

Answer 33

You can isolate the treatment effect by taking the difference of the change in the outcome between your treatment and control group, removing the effect of other stuff as it effects both groups outcomes

Question 34

What is the assumption of DiD?

Answer 34

The “other stuff” is assumed to be identical between treatment and control groups (called the parallel trends assumption)

Question 35

When is RD applied?

Answer 35

Estimating the impact of some treatment on some outcome of interest when in regards to a threshold

Question 36

What do the treatment and control groups look like for an RD experiment?

Answer 36

Treatment: units that receive treatment because they are barely just above threshold
Control: units that do not receive the treatment because they are just barely below the threshold of treatment

Question 37

What are the assumptions of RD?

Answer 37

Assumption 1: the difference is entirely due to the treatment of interest
Assumption 2: the difference in the “other stuff” is likely to be minimal