Week 10- Policy Success Framework

Question 1

What is the key problem?

Answer 1

The claims/assessments about policy outcomes do not establish any systematic criteria for assessing success or failure

Question 2

What are the three criteria for programmatic success?

Answer 2

• Effectiveness
• Efficiency
• Resilience

Question 3

How is political success characterised?

Answer 3

• ‘The political dimension of assessment refers to the way in which policies and policy makers become represented and evaluated in the political arena’
• Indicators of political success or failure are political upheaval (press coverage, parliamentary investigations, political fatalities, litigation) or lack of it, and changes in generic patterns of political legitimacy (public satisfaction with policy or confidence in authorities and public institutions)

Question 4

Why might criteria for policy success be contradictory?

Answer 4

Policies may be unsuccessful in programmatic terms, but ‘spun’ to appear a political success, or successful in programmatic terms, but represented as a failure and thus, politically negative for the government

Question 5

What are indicators of process policy success, and what is evidence for this?

Answer 5

• Legitimacy in the formation of choices: that is, produced through due processes of constitutional and quasi-constitutional procedures and values of democracy, deliberation
• Legislative records, executive minutes, absence of legal challenges, absence of significant criticism from stakeholders
• Passage of legislation: was the legislation passed with no, or few, amendments
• Analysis of legislative process, using legislative records, including identification of amendments and analysis of legislative voting patterns
• Political sustainability: did the policy have the support of a sufficient coalition?
• Analysis of support from ministers, stakeholders, especially interest groups, media, public opinion
• Innovation and influence: Was the policy based on new ideas or policy instruments, or did it involve the adoption of policy from elsewhere (policy transfer/diffusion)?
  -Government statements and reports (e.g. white/green papers) academic and practitioner conferences, interest group reports, think tank reports, media news and commentary, identification of form and content of cross-jurisdictional meetings/visits by politicians and/or public servants

Question 6

What are indicators of programmatic policy success, and what is evidence for this?

Answer 6

• Operational: was it implemented as per objectives?
  -Internal programme/policy evaluation, external evaluation (e.g. legislative committee reports, audit reports), review by stakeholders, absence of critical reports in media (including professional journals)
• Outcome: did it achieve the intended outcomes?
  -Internal programme/policy evaluation, external evaluation (e.g. legislative committee reports, audit reports), review by stakeholders, absence of critical reports in media (including professional journals)
• Resource: was it an efficient use of resources?
• Internal efficiency evaluations, external audit reports/assessments, absence of critical media reports
• Actor/interest: did the policy/implementation benefit a particular class, interest group, alliance, political party, gender, race, religion, territorial community, institution, ideology etc?
  -Party political speeches and press releases, legislative debates, legislative committee reports, ministerial briefings, interest group and other stakeholders’ speeches/press releases/reports, think tank reports, media commentary

Question 7

What are indicators of political policy success, and what is evidence for this?

Answer 7

• Government popularity: is the policy politically popular? Did it help government’s re-election chances? Did it help secure or boost its credibility?
  -Opinion polls, both in relation to particular policy and government popularity, election results, media commentary

Question 8

Broadly define ‘process’

Answer 8

Refers to the stages of policy-making in which issues emerge and are framed, options are explored, interests are consulted and decisions made

Question 9

It is assumed ‘programmatic success’ is more likely if what?

Answer 9

• If the policy process involves and reflects the interests of a sufficiently powerful coalition of interests
• Innovation and influence can also be measures of process success, irrespective of the particular policy outputs and outcomes. A policy process may produce new and innovative ways of tackling a problem

Question 10

How can policy transfer be considered innovation?

Answer 10

• Innovation can also be based on policy transfer from another political jurisdiction
• If policy instruments or even broad frameworks are imported from elsewhere, the assumption is that there is some value in doing so
• In effect, a policy transferred from elsewhere can bring with it not only particular policy instruments, but also the idea, sometimes erroneous, that it was successful in the original

Question 11

Why is policy implementation generally a much more complex affair than it was?

Answer 11

• The growth of:
  -Multi-level governance
  -Public sector fragmentation through arm’s length agencies
  -Quangos
  -Non-departmental public bodies
  -Privatisation
  -Outsourcing

Question 12

In what ways might a policy be considered a programmatic success?

Answer 12

• the nature of programmatic success can also encapsulate the subsequent impact on society
• Efficient use of resources can also be considered a success, and the growth of public audit bodies, value for money studies, national competition councils and productivity commissions are indications of the salience of resource efficiency as a measure of contemporary public policies
• May be successful if it benefits a particular actor, target group or interest, based on issues such as territory, race, religion and gender

Question 13

What is the biggest indicator of political success?

Answer 13

• From the perspective of the government and the governing party, a policy may be successful if it assists their electoral prospects, reputation or overall governance project

Question 14

What is a ‘token policy’?

Answer 14

• The creation of a new programme without any additional funding, which does little more than keep a “wicked issue” off the political agenda

Question 15

What are other indicators of political success?

Answer 15

• Successful policy may also rescue the party or government from low popularity, or help consolidate a lead in opinion polls
• May also apply to a policy which helps pave the way for a national election campaign
• Policy can be framed as successful because of an assumption that it has positive political impact

Question 16

Outline potential issues with determining whose success that of a policy is

Answer 16

• The nature of politics means that ‘success’ will always be contested to some degree
• Raises issue: do outcomes serve particular interests?
• Interested parties can attempt to influence evaluations

Question 17

How might the issues raised in determining whose success is achieved in a policy be managed?

Answer 17

• We must keep at the forefront of any analysis a recognition that policy outcomes reflect power relations and are likely to be perceived by different individuals/groups who see themselves as benefitting from/being adversely affected by those policies
• Simultaneously, we must also recognise that some would claim that policy success is nothing more than a social construct reflecting existing power relations

Question 18

What is argued about time, space and culture in assessing policies

Answer 18

Assessments of success or failure in government are therefore dependent on temporal, spatial, cultural and political factors

Question 19

Why might success of policies be viewed differently in different timeframes?

Answer 19

• A policy which appears successful at one time, with a short time frame, may seem less successful after a long period of time
• There are likely to be clear patterns in terms of the timeframes which different actors choose

Question 20

Why will governments transfer policy?

Answer 20

Governments can transfer policy from another jurisdiction when they have limited information about the effectiveness of the policy because there has been insufficient time or lack of political will to assess whether a policy has been successful

Question 21

What time frames will media, consultants and academics use to assess policy (respectively)?

Answer 21

• Media and consultants tend to operate with a shorter time frame
• Contrastingly, academics are more likely to use a longer timeframe as their concerns are broader and not so driven by the immediacies of politics

Question 22

What is the impact of space on determining policy success?

Answer 22

• Spatial dimension is also involved in assessing comparative policy success across different countries
• Define space not in terms of geography, but in terms of different polities, each with its own culture, values, political system and socio-economic conditions

Question 23

What is the impact of culture on determining policy success?

Answer 23

• What is regarded as success in one political system or political culture will not necessarily be regarded as a success in another

Question 24

What does policy success depend on in terms of methodological issues?

Answer 24

• Variety of methodological problems in assessing policy success
• Must acknowledge success is not all or nothing
• Determining political success depends on how we weight the differing imperatives of dispute resolution and non-negotiation with those prepared to break the law in order to influence the policy process

Question 25

In what ways may policy be somewhat considered a success?

Answer 25

• Some policy objectives may be partially achieved, creating space for protagonists and commentators to focus on those dimensions which accord with their beliefs and/or purposes
• ‘Non-failure’ may also occur, in that policies are successful in some aspects, but not all

Question 26

Outline contradictions in policy success

Answer 26

• Possibility that success may be greater than planned, and an ‘excess’ of success can lead to failure
• Overall, there may be contractions between what is achieved in relation to the process, programmatic and political dimensions of success
• Difficulty also occurs when a policy is a success in programmatic terms, but not in political terms because of the way a policy is portrayed in the media or by its opponents
• Policies also have unintended consequences and it is important to assess their impact and importance and how far (if at all) they undermine the original policy objectives

Question 27

List ways unintended consequences may produce success

Answer 27

1. May spur positive change
2. Success may simply be an unexpected spin-off from a ‘successful’ programme (such as government funded scientific research in one area producing breakthroughs for another)
3. Variation on point 2, a government may hope for and even anticipate, that an unintended consequence will be positive

Question 28

Outline the (7) critical choices to be made in assessing policy success

Answer 28

1. Form of policy success
   * Which form or forms of success is/are being assessed? Process? Programmatic? Political?
2. Timeframe
   * What time period(s) is/are being assessed? Short-term? Medium-term? Long-term?
3. Interests
   * In relation to whose interests is success being assessed, for example, target group? Stakeholders? Institution? Interest group? Individual? Collective?
4. Reference points
   * What is the standard by which success is to be judged? Compared to intentions? Compared to policy domain criteria, for example, efficiency and effectiveness? Compared to the past? Compared to ethical or moral principles? Compared to another jurisdiction?
5. Information
   * Is there sufficient and credible information to assess the extent of success?
6. Policy isolation
   * With what degree of certainty and credibility it is possible to isolate and assess the impact of a policy from other factors such as other policies or media influences?
7. Conflict and ambiguity
   * What significance should be given to conflicts and ambiguities, and how should they be weighted in the overall judgement of success? For example:
   -Process vs programmatic vs political success
   -Short-term vs long-term
   -Interests benefitting vs interests losing
   -One reference point vs another, for example, moral principles vs stated intentions
   -Availability of information vs lack of information
   -Certainty in isolating the ‘policy effect’ vs uncertainty in being able to do so
   -One formal objective vs another informal objective
   -Unintended consequences vs actual or intended consequences
   -Foreseeable shocks vs unforeseeable shocks

Question 29

Why is identifying and assessing the importance of objectives in policy success complicated?

Answer 29

• Ascertaining success depends on the availability of relevant information
• Complicated further when accepted that not only are many of the policy objectives unstated, but also, from the perspectives of elected policy makers, a policy fulfils a ‘political’ function which they cannot publicly articulate for fear of being accused of not acting in the public interest

Question 30

Ideally, what would need to happen to ascertain policy success?

Answer 30

• It is a highly subjective task to match the unstated policy goals against the political spin-offs of any particular policy
• Problems of ascertaining policy success are further compounded, as ideally, we would need to isolate and ascertain the effect of the policy on the outcome, controlling for other potential casual factors such as media coverage, the broader economic climate, external shocks, interest group activity, the role of the private sector pressure, the actions of other jurisdictions, whether national or international, and even other linked policy sectors