aim 3 data analysis Flashcards
Describe synth models.
comparative interrupted time-series design
comparing the treated unit to weighted combination of control units
Control unit weights are used to create a control that most closely matches the pre-intervention trends of the outcome and covariates
Once the weighting of the SC has been determined for the preintervention period, it is used to construct a counterfactual trend for the outcome in the postimplementation period
Estimating the difference between the outcome variable in the treated unit and the outcome variable in the synthetic control (SC) unit postintervention.
Why are SCM better than traditional time series?
they account for unmeasured time-varying covariates that co-occur during the pre-intervention period. (DID can only account for measured time varying covariates)
Relaxes parallel trends assumption- in the absence of treatment, average outcomes of the treated and control units would have followed parallel paths
Describe how different from aug synth models.
SCM however are only appropriate when the synthetic unit’s pre-treatment outcomes closely match the treated unit pre-treatment outcomes. When it is not feasible to construct a close match, ASCM are an extension of SCM that estimates and adjusts for bias in pre-treatment fit.
ASCM begins with the original SCM estimate, uses an outcome model to estimate the bias due to imperfect pretreatment fit, and then uses this to de-bias the SCM estimate. If pre-treatment fit is good, the estimated bias will be small, and the SCM and ASCM estimates will be similar. Otherwise, the estimates will diverge, and ASCM will rely more heavily on extrapolation.
Analogous to bias correction for inexact matching
Ridge ASCM admits negative weights, using extrapolation to improve pre-treatment fit- some uncomfortable with extrapolation
Limitations/assumptions of ASCM
Cant account for spillover effect
Cant account for “shocks” that differentially effect outcome in treated or control units
Describe all of the covariates mentioned in aim 3. Why include? What biases in the data? How measured and how modeled? Significance in regression models?
Covariates were selected based on prior evaluations of Safe Streets and other violence prevention interventions.
included in all models to improve model fit and attempt to isolate the impacts of Safe Streets specifically.
Incident level data for drug possession arrests, drug trafficking arrests, and weapon possession were obtained from BPD and OB.
violence prevention initiatives led by the BPD that overlapped with the study period: Violent Crime Impact Section zones (VCIS), Violence Reduction Initiatives (VRI), and Ceasefire. These are all hot spot policing initiatives directed at gun violence.
The East Baltimore redevelopment efforts were included as two covariates, one indicating the area where the redevelopment occurred and one indicating potential extended effects of the redevelopment on posts adjacent to those in the redevelopment catchment area.
outcomes were modeled as three-month moving averages to smooth out the volatility of homicides and non-fatal shootings, and arrests were modeled as yearly averages
How is donor pool defined? Pros/cons of that decision
142 police posts.
Used the sum of all HNFS in all posts across the full intervention time period (2003-2022) was calculated, and only the posts in the top 30th percentile (N=44) were eligible for the control pool. Posts that received the BPD VRI intervention, defined as any post where more than 2/3 of the area of the post was in a VRI catchment area, were further removed from the control pool, leaving 38 posts eligible for the control pool.
Assumed better to have a smaller control pool of eligible sites that are more similar to the treated unit. If v different in outcome than treated unit, will not be weighted nearly at all.
4 year censored estimates- if site came on more than 4 years after implementation, eligible for control
What specific hypotheses are being tested? What are the test statistics youre using?
Average monthly count of homicides, nfs, or HNFS different in treated site than synthetic control site
jacknife standard error: iterate over the model taking out 1 post at a time n-1 times to create a range of estimates.
How are models parameterized? Describe fixed effects, ridge, pre and post amount of data
fixed effects: adjust for unobserved unit-specific (post) Specifying fixed effects de-means the lagged outcomes in the pre-treatment period prior to the creation of the weighted synthetic control.
ridge-regularized linear model: when the treated unit lies outside the convex hull (range of time/homicide units) of the control units, Ridge ASCM improves the pre-treatment fit relative to SCM alone by allowing for negative weights and extrapolating away from the convex hull. only resorts to negative weights if the treated unit is outside of the convex hull.
4 years pre and post: Forecasting counterfactuals for long periods increases risks of confounding factors biasing estimates of program impacts. Therefore, each model is restricted to 4 years post l. To ensure that the most relevant and recent trends are used to construct the policy counterfactuals, pre-treatment training years were restricted to the 4 years prior. Also uncensored models for city
Why suggested HNFS is the outcome? What else could you measure?
Hard to measure what doesn’t happen but could measure violent conflicts avoided
Evaluation that looks at referrals, proportion of engaged youth connected to GED, job, anti-recidivism, months w/out violence per individual
Define spatial lag
Outcome or exposure based on neighbor
Can define a neighbor different ways
Queen: common edge or vertex. Rook = just common edge. Map defines as queen contiguity
Describe spatial hypotheses. whats another one?
patterns of HNFS in neighboring (e.g. lag) areas may impact SSB ability to reduce HNFS in treated areas (e.g. focal).
OR how staff they believe the work in their area is impacting HNFS in neighboring areas. If SSB in one area is also reducing HNFS in a neighboring area, it would be biasing current model results to the null
Another: look at relationship between arrest variables and homicides/NFS to assess some measure of over/under policing
Why use count not rate
No good way to measure population size in police post
What is spatial correlation. How account?
looking at how well objects correlate with other nearby objects across a spatial area. Positive autocorrelation occurs when many similar values are located near each other
it helps to define how important spatial characteristic is in affecting a given object in space and if there is a clear relationship of objects with spatial properties.
calculate spatial correlation and include error term in regression model to account
What are other spatial approaches you could use?
Clustering (small scale) to define areas of change in violence interruption or sensitivity analyses to boundaries ( l function- compare pattern to random pattern, accounting for complete spatial randomness)
Cluster detection to help answer if violence is being reduced or displaced (sat scan)
A lot of hnfs occurs at the edge of the police posts, perhaps because their boundaries are often drawn on main roads. It would be interesting to see how sensitive results are to for example, expanding the boundary of posts by a small buffer
Shani findings w census data. How applied census data to police post
2005-2009 ACS 5 year average, “averaged by police post” estimated up from census block group
data on: percent of households below poverty, percent total male pop, percent black, percent white, median household income, percent vacant housing
