Evaluation, Termination and Follow-Up in Generalist Practice Flashcards
Definition of Evaluation:
“A process of determining whether a given change effort was worthwhile”
evaluation is step _ in the Generalist Intervention Model
5
evaluation is designed to determine the extent work was successful with a specific client; at the mezzo level want to know if groups met their goals and for treatment groups, if individual members met their goals; at the macro level may focus on whether entire programs accomplished their purpose.
At the MICRO level
wowowowow
Evaluation spurred by:
Demands for accountability
Interest in improving practice
“best practices”: ensures that we use knowledge that has been tested through some scientific rigour
External Factors in Evaluation
Governments requiring proof their money (and our tax dollars) was being put to good use
Assurance that the money was accomplishing intended purpose
Users of the service have right to know about program effectiveness
Obstacles to Evaluation
Worker’s themselves: don’t want to know if their work is effective – heightens vulnerability
Workers often too busy for formal evaluation
Many agencies do not emphasize its importance – involves time and resources
Social workers lack training in evaluative measures
The Evaluation Process
Typically evaluations have one of two major thrusts: 1) monitor the ongoing delivery of a program or service (process of giving service), and 2) assess the outcomes of an intervention or program (effectiveness of service)
Formative Evaluation means
ongoing with focus on the process of providing service; can assess if anticipated progress is being achieved and whether the program or service is proceeding as planned
Summative Evaluation means
occurs at end of helping process with focus on whether or not anticipated outcomes have been achieved
Baseline means
measure of the frequency, intensity, or duration of a behavior
Validity means
“the extent to which you are measuring what you think you are measuring”
Reliability means
the extent to which an instrument measures the same phenomenon in the same way each time the measure is used”, produces consistent results over time
Independent Variable means
factor believed responsible for causing certain behaviors, reactions, events such as your intervention
Dependent Variable means
results or outcome of the independent variable, such as outcome of an intervention
Generalizability means
the ability of a set of results in one situation to fit another circumstance or instance
Commonly used methods include surveys, scores on instruments, interviews, collected data, and observations
Desirable to use multiple measures—triangulation are ____
Data gathering methods
Name assesment tools
Valid and reliable instruments already developed for many issues
Example of valid and reliable instruments
For example:
Primary Communication Inventory (PCI)
Single-Subject Designs
research methods aimed at determining whether or not an intervention was successful, most simple is the AB design, with A representing the pre-intervention state and B the intervention
what is this
single subject design
_________ is used when achievement of goal is used as the primary outcome criterion and where goal doesn’t lend itself to the kind of charting associated with single-subject design such as counting behaviors, goal attainment is rated based on degree to which goal was attained; could use 4 or 5 categories of goal attainment from least favorable to most favorable outcome
Attainment Scaling
Attainment Scaling is
used when achievement of goal is used as the primary outcome criterion and where goal doesn’t lend itself to the kind of charting associated with single-subject design such as counting behaviors, goal attainment is rated based on degree to which goal was attained; could use 4 or 5 categories of goal attainment from least favorable to most favorable outcome
Task Achievement Scaling
used to evaluate the degree to which tasks have been accomplished, typically use a 5 point scale ranging from 4 ‘completely achieved’ to 0 ‘no opportunity to work on task’
Client Satisfaction Questionnaires
assess client’s reactions to interventions, can be used at worker level and at program/agency level, can help identify problems with how services are being delivered
Target-Problem Scaling
allows you to measure if the identified problem has changed in severity or seriousness over time, problems are rated by degree of severity using a scale, e.g., 1=worse to 5=a lot better or NP=no problem to ES=extremely severe
Needs Assessment
form of front-end analysis to identify unmet needs, gaps in service, and extent to which a program is needed)
Evaluability Assessments
used to assess if a program can be evaluated and agency readiness for evaluation…may point to need for agency changes to allow future assessments
Process Analysis
or formative evaluations, used to evaluate the way interventions in the agency are carried out
Program Outcome Analysis
used to assess if a program is working based on planned program outcomes, may also assess cost effectiveness
Continuous Quality Assurance Evaluations
focus on measuring attainment of previously identified program goals in terms of outcomes of interventions and factors related to delivery, e.g., timely completion of paper work
Program Monitoring
ongoing activity designed to provide information to the agency on all aspects of its operation, many types of data may be collected, e.g., client demographics, presenting problems, services provided
Problems in Evaluation
Problems in Generalizability (need a representative sample)
Wrong Choices in Evaluation Tools
Failure to Involve Clients in the Evaluation Process
Staff Distrust of Evaluation
Evaluation Process Interferes with Service Giving
Alternative Explanations of Program Outcomes
Unanticipated Consequences
Alternate Explanations for Outcomes
History (any event that occurs prior to the end of the intervention that could impact upon outcome)
Maturation (as client ages behaviors change)
Mortality (participants drop out of program, should be factored in with clients who failed to attain goals)
Creaming (taking only the very best candidates for the program, so may get higher success rate than if similar candidates in the population were included)
Regressions toward the Mean (tendency for high scores to move toward the mean on subsequent testing)
Reactance (reaction or change in behavior that occurs simply because of a new situation or environment, such as an observer)
Termination Defined
The end of the professional social worker-client relationship
Termination is step
6
name the steps
what are some tasks of termination
Deciding when to terminate the worker-client relationship
Evaluating the achievement of goals/objectives
Evaluating the services provided by the agency and/or worker
Maintaining and continuing progress (generalizing or stabilizing change effort)
Resolving emotional reaction of the worker and client
Making appropriate referrals
Stabilization of Change
1) Help clients select relevant and appropriate situations to work on
2) Help clients build confidence in their own abilities
3) Use multiple situations and settings when helping members learn new behaviors
4) Using naturally occurring consequences rather than creating artificial ones
5) Extending treatment through use of follow-up sessions
6) Reduce setbacks in other environments
7) Helping members confront future problems by teaching them a problem solving process
Client follow up
Follow-up Defined: “The act of acquiring information about a client after termination. It is usually focused on how the client is functioning in those areas that originally brought the client in for assistance.”
client follow up is step
7
why client might not maintain changes
Easier to slip into old patterns than maintain new ones
Environments may not be supportive of new changes
New behavior not established long enough to become the norm
reasons for doing a follow up
Finding out if the client is functioning without intervention
Determine if the intervention worked
Offering assistance if the client is having difficulty
Also can get feedback on helpful and perhaps unhelpful aspects of social work service
