Four Levels of Evaluation Flashcards
What is effective training?
Training that provides relevant knowledge and skills to participants and the confidence to apply them on the job.
What is training effectiveness?
Training and follow-up leading to improved job performance that positively contributes to key organizational results.
How should training requests be considered?
As invitations to the conversation about increasing performance and improving results.
At which stages should we involve ourselves in the training program?
- Before
- During
- After
What are the four levels of the New World Kirkpatrick Model?
- Reaction
- Learning
- Behavior
- Results
What does the ‘Reaction’ level in the New World Kirkpatrick Model measure?
The degree to which participants find the training favorable, engaging, and relevant to their jobs.
What does the ‘Learning’ level in the New World Kirkpatrick Model measure?
The degree to which participants acquire the intended knowledge, skills, attitude, confidence, and commitment based on their participation in the training.
What does the ‘Behavior’ level in the New World Kirkpatrick Model measure?
The degree to which participants apply what they have learned during the training when they are back on the job.
What does the ‘Results’ level in the New World Kirkpatrick Model measure?
The degree to which targeted outcomes occur as a result of the training and the support and accountability package.
What is the ultimate level 4 result for any organization?
The mission.
What question should you ask to check if you have the right mission statement?
Is this what the organization exists to do, deliver, or contribute to its customers and society at a high level?
What are leading indicators?
Short-term observations and measurements that suggest critical behaviors are on track to create a positive impact on desired results.
What are some examples of leading indicators?
- Monthly sales volume
- Defective rate
- Retention
- Satisfaction scores
What are critical behaviors?
Specific actions that, if performed consistently on the job, will have the biggest impact on desired results.
What are required drivers?
Processes and systems that reinforce, monitor, encourage, and reward the performance of critical behaviors on the job.
What percentage of learning happens on the job?
70%
List the five learning components.
- Knowledge: “I know it.”
- Skill: “I can do it right now.”
- Attitude: “I believe this will be worthwhile to do on the job.”
- Confidence: “I think I can do it on the job.”
- Commitment: “I will do it on the job.”
What is often inaccurately diagnosed as a lack of knowledge and skill?
Poor performance.
What should customer satisfaction be used for in training?
To identify and eliminate barriers to learning in the program.
What does engagement measure in the ‘Reaction’ level?
The degree to which the participants are actively involved in and contributing to the learning experience.
What does relevance measure in the ‘Reaction’ level?
The degree to which the participants will have the opportunity to use what they learned in training on the job.
Levels 1 and 2 of the New World Kirkpatrick Model are about _______.
effective training
Levels 3 and 4 of the New World Kirkpatrick Model are about _______.
training effectiveness
What are the two main considerations when designing a training program?
Usefulness and credibility
What does usefulness in a training program evaluation refer to?
Information used to make decisions related to the program
What does credibility in a training program evaluation refer to?
Information demonstrating program value in business terms defined by stakeholders
When should programs be evaluated on all four levels?
- Large in scale
- Stakes are high
- Expensive
- Align with corporate goals
What are the three phases of evaluation activities for programs?
- Planning
- Execution
- Demonstration of value
What is the first step of the planning phase?
Identify the level 4 result of the organization
During the planning phase, what should you discuss with stakeholders after identifying the level 4 result?
How the program contributes to that result or what leading indicators will positively influence it
What is Return on Expectations?
What a successful training initiative delivers to key business stakeholders, demonstrating the degree to which their expectations have been satisfied
What should be determined with stakeholders regarding success?
What success looks like and how it will be measured
What are the success factors in a training program?
- Required drivers
- Training program
- Personal responsibility
- Necessities for success
What is the third step of the planning phase?
Define critical behaviors and required drivers
What should you ask employees about during the planning phase?
- Their work
- Their goals and targets for the year
- Business challenges they are experiencing
- Causes of performance reduction
What should be done once critical behaviors are known?
Discuss the interventions that might solve the challenges
What is the fourth step of the planning phase?
Set specific expectations for post-training and define roles and responsibilities. Create a schedule.
What are the necessities of success?
Prerequisite items, events, conditions, or communications that will help leverage success or avoid problems
What is the fifth step of the planning phase?
Design the intervention and determine if training is needed, and which critical behaviors it will focus on
What is the sixth step of the planning phase?
Design the evaluation tools for the program
What is blended evaluation?
When data is collected from multiple courses using multiple methods for all four levels
What is the first step of the execution phase?
Execution of the training program and associated activities
During the execution phase, what materials should be given to managers and supervisors?
Materials to introduce training to direct reports, outlining expectations for before, during, and after
How should training be tied to performance during the program?
By discussing ideas and any concerns, and emphasizing the relevance
How should support and accountability be ensured after training?
Managers should check in with direct reports on how the training went, what they learned, and how they will apply it
What should be compiled in the demonstration of value phase?
Reports of value using storytelling, visual dashboards, or graphics
