4 - Policy Evaluation Tools Flashcards
Sunstein 1
(CBA for Welfare Policies)
- inquiries into subjective well-being are too coarse to provide a great deal of help to administrators, & CBA = best proxy they have for (much of ) what matters.
- But it cannot possibly tell us everything we need to know. In fullness of time, it will be supplemented or perhaps even superseded by a more direct focus on welfare.
Sunstein 2
(CBA for Welfare Policies)
- For welfare, CBA, may not adequately handle:
1) Unusually large or unusually small numbers of life-years saved;
2) Adverse unemployment effects;
3) Questions about welfare effects of small economic losses faced by large populations;
4) Intense emotions associated with certain outcomes, such as parental anguish (or fear); &
5) Hedonic benefits associated with increased ease & convenience - Does not capture distributional impacts
Radaelli 1
(Regulatory Impact Assessment - Europe)
3 lessons learned
1) About the relationship between conceptual analysis and measurement. Having established that the former comes before the latter what is the best way to proceed in terms of concept formation? We do not yet know how to classify per genus et differentian in this field of research. Future research will have to ‘go up’ and identify the family of the genus RIA, arguably this would be the better regulation family.
Radaelli 2
(Regulatory Impact Assessment - Europe)
3 lessons learned
2) About the asymmetries between demand and supply of research, or the relationship between the community of researchers and the policy-making world. The clients want more research on RIA, thus we badly want the RIAs to be there.
One way to balance the relationship between demand and supply is to engage in problem redefinition.
The relationship with the client can also be rebalanced by providing them with more.
Radaelli 3
(Regulatory Impact Assessment - Europe)
3 lessons learned
3) About ‘who defines our object of study?’ There is no doubt that better regulation is an excellent term for political action. It mobilises politicians and bureaucrats. It resonates with ‘improvement’ and notions like the modernisation of the public sector.
Pearce and Raman
(RCTs - UK Local Gov)
- For evidence to be credible it must take account of its specific context.
- Trials hold the promise of increasing the plurality of evidence, but that the rhetoric surrounding the new movement provides a potential pitfall in promoting RCTs at the expense of other research methods.
- There is a danger that the current UK government’s interest in RCTs is driven not by their methodological suitability, but because they lend themselves to a model of governance that values context-free quantification and benchmarking.
Serin et al.
(RCTS)
Since the application of the policy is fully randomised, subjects in the control group should, in theory, be similar to those in the treatment group, so changes in their behaviour and outcomes when the policy is implemented can be associated with the policy itself rather than individual characteristics or wider trends or influences.
Hariton and Locascio
(RCTs - Pros :) )
“gold standard for policy evaluation”
- Although no study is likely on its own to prove causality, randomisation reduces bias and provides a rigorous tool to examine cause-effect relationships between an intervention & outcome.
- This is because the act of randomisation balances participant characteristics (both observed and unobserved) between the groups allowing attribution of any differences in outcome to the study intervention. This is not possible with any other study design.
Hariton and Locascio
(RCTs - Cons :( )
RCTs can have their drawbacks, including their
- High cost in terms of time and money,
- Problems with generalisabilty (participants that volunteer to participate might not be representative of the population being studied)
- And loss to follow up
