COURSE PACKET 4 Flashcards
refers to the evaluation of training that takes place during program design and development. That is, formative evaluation helps ensure that (1) the training program is well organized and runs smoothly, and (2) trainees learn and are satisfied with the program. It provides information about how to make the program better; usually involves collecting qualitative data about the program.
Formative evaluation
refers to an evaluation conducted to determine the extent to which trainees have changed as a result of participating in the training program. That is, have trainees acquired knowledge, skills, attitudes, behavior, or other outcomes identified in the training objectives? Summative evaluation may also include measuring the monetary benefits (also known as return on investment or ROI) that the company receives from the program. It usually involves collecting quantitative (numerical) data through tests, ratings of behavior, or objective measures of performance such as volume of sales, accidents, or patents.
Summative evaluation
Kinds of Standard Training Evaluation
a. Formative evaluation
b. Process evaluation
c. Outcome evaluation
d. Impact
provides ongoing feedback to the curriculum designers and developers to ensure that what is being created really meets the needs of the intended audience.
Formative evaluation
provides information about what occurs during training. This includes giving and receiving verbal feedback.
Process evaluation
determines whether or not the desired results (e.g., what participants are doing) of applying new skills were Achieved in the short-term.
Outcome evaluation
determines how the results of the training affect the strategic goal
Impact
The Evaluation Process
Conduct a Needs Analysis
Develop Measurable Learning Objectives and Analyze Transfer of Training
Develop Outcome Measures
Choose an Evaluation Strategy
Plans and Execute the Evaluation
Criteria for Evaluation of Training
Reaction outcomes
Learning or cognitive outcomes
Behavior and skill-based outcomes
Affective outcomes
Results
Return on Investment (ROI)
refer to the trainees’ perceptions of the program, including the facilities, trainers, and content. These are often referred to as a measure of “creature comfort” and called class or instructor evaluations.
Reaction outcomes
are used to determine to which trainees are familiar with the principles, facts, techniques, procedures, and processes emphasized in the training program.
Learning or cognitive outcomes
are used to assess the level of technical or motor skills and behaviors. These include acquisition or learning of skills (skill learning) and use of skills on the job (skill transfer).
Behavior and skill-based outcomes
include attitudes and motivation. These that might be collected in an evaluation include tolerance for diversity, employee engagement, motivation to learn, safety attitudes, and customer service orientation. These can be measured using surveys.
Affective outcomes
are used to determine the training program’s payoff for the company.
results
refers to comparing the training’s monetary benefits with the cost of the training. This can be measured and communicated based on a percentage or a ratio.
Return on Investment (ROI)
former Professor Emeritus at the University of Wisconsin,
first published his model in 1959. He updated it in 1975, and again in 1993, when he
published his best-known work, “Evaluating Training Programs.”
Donald Kirkpatrick
“Four Levels of Training
Evaluation.”
Level 1: Reaction – How well they liked the training
Level 2: Learning – How much they learned
Level 3: Behavior – How well they applied the learning to work
Level 4: Results – What return the training investment yielded
refer to factors that will lead an evaluator to question either (1) the believability of the study results or (2) the extent to which the evaluation results are generalizable to other groups of trainees and situations.
Threats to validity
generalizability of the study results to other groups and situations
External validity
Methods to Control for Threats to Validity
Pretests and post-tests – establishing a baseline or pretraining and post-training
measures.
Use of comparison groups – a group of employees who participate in the evaluation study but do not attend the training program
Hawthorne effect – employees in an evaluation study performing at a high level simply because of the attention they are receiving.
Random assignment – assigning employees to the training or comparison group on the basis of chance alone.
