Four Levels of Evaluation

Question 1

What is effective training?

Answer 1

Training that provides relevant knowledge and skills to participants and the confidence to apply them on the job.

Question 2

What is training effectiveness?

Answer 2

Training and follow-up leading to improved job performance that positively contributes to key organizational results.

Question 3

How should training requests be considered?

Answer 3

As invitations to the conversation about increasing performance and improving results.

Question 4

At which stages should we involve ourselves in the training program?

Answer 4

• Before
• During
• After

Question 5

What are the four levels of the New World Kirkpatrick Model?

Answer 5

• Reaction
• Learning
• Behavior
• Results

Question 6

What does the ‘Reaction’ level in the New World Kirkpatrick Model measure?

Answer 6

The degree to which participants find the training favorable, engaging, and relevant to their jobs.

Question 7

What does the ‘Learning’ level in the New World Kirkpatrick Model measure?

Answer 7

The degree to which participants acquire the intended knowledge, skills, attitude, confidence, and commitment based on their participation in the training.

Question 8

What does the ‘Behavior’ level in the New World Kirkpatrick Model measure?

Answer 8

The degree to which participants apply what they have learned during the training when they are back on the job.

Question 9

What does the ‘Results’ level in the New World Kirkpatrick Model measure?

Answer 9

The degree to which targeted outcomes occur as a result of the training and the support and accountability package.

Question 10

What is the ultimate level 4 result for any organization?

Answer 10

The mission.

Question 11

What question should you ask to check if you have the right mission statement?

Answer 11

Is this what the organization exists to do, deliver, or contribute to its customers and society at a high level?

Question 12

What are leading indicators?

Answer 12

Short-term observations and measurements that suggest critical behaviors are on track to create a positive impact on desired results.

Question 13

What are some examples of leading indicators?

Answer 13

• Monthly sales volume
• Defective rate
• Retention
• Satisfaction scores

Question 14

What are critical behaviors?

Answer 14

Specific actions that, if performed consistently on the job, will have the biggest impact on desired results.

Question 15

What are required drivers?

Answer 15

Processes and systems that reinforce, monitor, encourage, and reward the performance of critical behaviors on the job.

Question 16

What percentage of learning happens on the job?

Answer 16

70%

Question 17

List the five learning components.

Answer 17

• Knowledge: “I know it.”
• Skill: “I can do it right now.”
• Attitude: “I believe this will be worthwhile to do on the job.”
• Confidence: “I think I can do it on the job.”
• Commitment: “I will do it on the job.”

Question 18

What is often inaccurately diagnosed as a lack of knowledge and skill?

Answer 18

Poor performance.

Question 19

What should customer satisfaction be used for in training?

Answer 19

To identify and eliminate barriers to learning in the program.

Question 20

What does engagement measure in the ‘Reaction’ level?

Answer 20

The degree to which the participants are actively involved in and contributing to the learning experience.

Question 21

What does relevance measure in the ‘Reaction’ level?

Answer 21

The degree to which the participants will have the opportunity to use what they learned in training on the job.

Question 22

Levels 1 and 2 of the New World Kirkpatrick Model are about _______.

Answer 22

effective training

Question 23

Levels 3 and 4 of the New World Kirkpatrick Model are about _______.

Answer 23

training effectiveness

Question 24

What are the two main considerations when designing a training program?

Answer 24

Usefulness and credibility

Question 25

What does usefulness in a training program evaluation refer to?

Answer 25

Information used to make decisions related to the program

Question 26

What does credibility in a training program evaluation refer to?

Answer 26

Information demonstrating program value in business terms defined by stakeholders

Question 27

When should programs be evaluated on all four levels?

Answer 27

• Large in scale
• Stakes are high
• Expensive
• Align with corporate goals

Question 28

What are the three phases of evaluation activities for programs?

Answer 28

• Planning
• Execution
• Demonstration of value

Question 29

What is the first step of the planning phase?

Answer 29

Identify the level 4 result of the organization

Question 30

During the planning phase, what should you discuss with stakeholders after identifying the level 4 result?

Answer 30

How the program contributes to that result or what leading indicators will positively influence it

Question 31

What is Return on Expectations?

Answer 31

What a successful training initiative delivers to key business stakeholders, demonstrating the degree to which their expectations have been satisfied

Question 32

What should be determined with stakeholders regarding success?

Answer 32

What success looks like and how it will be measured

Question 33

What are the success factors in a training program?

Answer 33

• Required drivers
• Training program
• Personal responsibility
• Necessities for success

Question 34

What is the third step of the planning phase?

Answer 34

Define critical behaviors and required drivers

Question 35

What should you ask employees about during the planning phase?

Answer 35

• Their work
• Their goals and targets for the year
• Business challenges they are experiencing
• Causes of performance reduction

Question 36

What should be done once critical behaviors are known?

Answer 36

Discuss the interventions that might solve the challenges

Question 37

What is the fourth step of the planning phase?

Answer 37

Set specific expectations for post-training and define roles and responsibilities. Create a schedule.

Question 38

What are the necessities of success?

Answer 38

Prerequisite items, events, conditions, or communications that will help leverage success or avoid problems

Question 39

What is the fifth step of the planning phase?

Answer 39

Design the intervention and determine if training is needed, and which critical behaviors it will focus on

Question 40

What is the sixth step of the planning phase?

Answer 40

Design the evaluation tools for the program

Question 41

What is blended evaluation?

Answer 41

When data is collected from multiple courses using multiple methods for all four levels

Question 42

What is the first step of the execution phase?

Answer 42

Execution of the training program and associated activities

Question 43

During the execution phase, what materials should be given to managers and supervisors?

Answer 43

Materials to introduce training to direct reports, outlining expectations for before, during, and after

Question 44

How should training be tied to performance during the program?

Answer 44

By discussing ideas and any concerns, and emphasizing the relevance

Question 45

How should support and accountability be ensured after training?

Answer 45

Managers should check in with direct reports on how the training went, what they learned, and how they will apply it

Question 46

What should be compiled in the demonstration of value phase?

Answer 46

Reports of value using storytelling, visual dashboards, or graphics