exam 2 Flashcards
Superintendent Brooks’ reform plan (4)
- eliminating (or reclassifying) 101 central office positions
- reorganizing the district’s central office bureaucracy to create a clear line of authority from the superintendent to school principals
- the closure of 4 elementary schools in the predominantly low-income and African American southern part of LR
- using a mixture of private and district funds, Brooks supported a pilot program to pay bonuses to Meadowcliff Elementary School teachers and staff
purpose of the LRSD implementation research (2)
- to explore the reorganization policy of the LRSD within the context of policy implementation
- analyze data collected from the school district to better understand the barriers to successful implementation of this policy
describe the LRSD reorganization policy (2)
create a more efficient bureaucracy by
1. establishing a clear line of authority from the superintendent to principals
2. streamlining the organization by eliminating central office positions and redistributing resources to schools
Freeman’s 4 points of a needs assessment
- constructing a precise definition of the problem
- assessing its extent
- defining and identifying the target of intervention
- accurately describing the nature of the service needs of that population
emison’s key points for control (6)
- have a real work plan
- meet the deadline
- getting people with the right skills and temperament
- expect something to go wrong
- use more than one set of eyes
- know your core values
emison’s key points for communication (4)
- lead with communicating ideas, not details
- write and speak in short words and sentences
- preview, provide, and review messages
- visuals that support, not supplant, the briefing
fink’s presentation points (visuals) (6)
- do the talking and explaining, let the audience listen
- focus on the key points of the presentation
- each slide has a title
- keep tables/figures simple
- do not simply read a slide
- rehearse the presentation
solomon and shortell communication points (5)
- understand the cognitive styles of decision makers
- timely and available results
- respect the stakeholders’ commitments
- utilization/dissemination should be part of the design
- overcome organizational resistance and politics
emison’s characteristics of a work plan/timeline (4)
- it must be complete; no important activities should be overlooked
- it must be specific; products to be delivered and the timing of that delivery must be clear
- it must connect actual humans on the evaluation team with the acitivities and products; names should be tied to obligations
- it must be usuable; both understandable and updated as conditions change
main difference between qualitative and quantitative research
qualitative research relies on words, a narrative, and/or a story; quantitative research relies on counts and statistical analysis
advantages of qualitative research (5)
- greater awareness of the perspective of program participants
- awareness of time and history
- sensitivity to context
- ability to learn what is happening without prepared instruments
- greater flexibility of perspective
range/types of qualitative research (6)
- ethnography
- participant-observation
- observation (non-participant observation)
- informal interviewing
- focus groups
- documents
ethical issues with qualitative research (3)
- deception
- confidentiality
- bias
combining qualitative and quantitative studies (5)
- study different parts (in different forms)
- study different stages of a program
- to help evaluation design and measures
- multiple methods
- interpretation of results
3 areas one wants a process evaluation to provide data/information
- description of program operations
- comparison between sites
- conformity of the program to its design
program organization
comparing the plan with what the program should be doing and what is actually being provided
service utilization
understanding or examining the extent to which the target population receives the intended service
coverage
the extent to which participation by the target population achieves the levels specified in the program design
bias
the degree to which some subgroups participate in greater proportions than others
3 perspectives of process monitoring
- evaluator - implementation issues
- sponsors - program accountability
- program managers - corrective actions
how is the problem described when assessing the extent (needs assessment)
in terms of the size of the problem; how widespread it is
ways to describe the target population (3)
- risk/need
- incidence/prevalence
- rates
risk/need
a population at risk would consist of persons or units that have a significant probability of developing this problem or condition that the program is trying to address
incidence
the number of new cases that are identified in a specific time period
prevalence
the total number of existing cases
rates
a standardization; as you describe incidence or prevalence, you might use it as a rate
importance of understanding the nature of service needs of the target population
if you don’t understand the services needed, the target population cannot get the proper help
Management Information System (MIS)
what data the organization is already collecting
ways to develop relevant outcomes (3)
- stakeholders
- program theory/impact model
- prior research
how process evaluation findings are used per Fox, Grimm, and Calderia
to shape the intervention/recommendations
key point of distinguishing monitoring from normal process/outcome evaluations
repeated measurement over time
approaches to setting levels of program performance (3)
- often based off of impact model/program theory
- identify which aspects of program performance are more important
- look at administrative standards to judge what good performance is
2 characteristics of outcome evaluation per Rossi
- outcomes = observed characteristics of the target population or social conditions
- outcomes don’t mean the program targets have changed/programs have caused them to change
implementation type for each group and the strategy for successful implementation (LRSD) (4)
- superintendent & executive staff = administrative implementation; key to success = providing sufficient resources (training)
- principals = political implementation; key to success = power of district leadership to force compliance
- teachers = experimental implementation; key to success = providing sufficient resources (understanding that additional resources were a result of the reorganization)
- central office & school-based staff = administrative implementation; key to success = providing sufficient resources (training)
why academic research time and evaluation time differ
academic time = more perfect
evaluation time = time constraints/sacrifice data
parts of an evaluation presentation (5)
- address the evaluation purposes
- describe the program/participants
- describe the main methods (data collection)
- provide main results
- provide a conclusion (include limitations) and recommendations
2 issues evaluators might face in communicating results
- agency/organizational structure
- agency/organizational politics
pros noted in the LRSD case study article (4)
- reorganization –> greater accountability (Larry Berkely)
- efficiency & effectiveness
- reduce top-heavy administration (bloated central office)
- cost saving
cons noted in LRSD case study article (5)
- moved too quickly
- disproportionate effect on black personnel ($913k v. $28k loss for white personnel)
- transition team determined who to fire in secret with no justification
- Brooks failed to connect with black community & grass roots organizations
- poor relationship with teachers union (brought in to “destroy” it)
stakeholders involved in relation to their motivations of an evaluation (2)
- pro-Brooks school board members (Berkley, Fox, Kurrus): justify support and continued support of Brooks reforms
- anti-Brooks school board members (Mitchell, Armstrong, Curry, Daugherty): justify removal and reversal of Brooks’ decisions
importance of defining target population
helps set boundaries of the program; different stakeholders may have different ideas of who the target population is
concepts of specification of services and accessibility (delivery of services) (2)
- specification of the services: the actual service provided by the program in operational terms (measurable terms) (an explicit definition of a service and how you go about measure that)
- accessibility: are the program’s actions consistent with providing access?
on-going problem with evaluation studies in terms of timeline
data collection; collection of data can take longer than one estimate, which then pushes back how much analysis there is
