Week 11: Training Evaluation

Question 1

What is this the Definition of?:

A process to assess the value or worthiness of training
programs to employees and to organizations.

A continuum of techniques, methods, and measures.

Answer 1

Training Evaluation

Question 2

List 4 Reasons Training Evaluation is of value to organizations:

Answer 2

1- Improve training
2- Determine the programs most useful to employees and determine who should be trained
3- Determine cost benefits and cost-effectiveness of a program
4- Determine if a program has achieved expected results or solved the problem it was meant to solve

Question 3

Just because you can measure everything, should you?

Answer 3

No, not always.

Question 4

Why is it important to have metrics?

Answer 4

Without metrics, you’re “just another person with an opinion.”

Question 5

What are the 2 types of BARRIERS to Training Evaluation?

Answer 5

Pragmatic and Political

Question 6

Give 3 Examples of “Pragmatic” barriers to Training Evaluation:

Answer 6

• Evaluation requires knowledge of evaluation models,
research design, measurement, questionnaire
construction, and data analysis (may be intimidating)
• Time and effort
• Financial costs

Question 7

Give 3 Examples of “Political” barriers to Training Evaluation:

Answer 7

• Absence of accountability
• Evaluation can be threatening
• Ethical concerns; a belief that evaluation should be externally conducted (think about Makers and Checkers example)

Question 8

Two types of Training Evaluation are:

Answer 8

1- Formative (Throughout, i.e. reading the crowd)

2- Summative (After)

Question 9

This type of training evaluation happens:

• during development to modify and improve the program
• during training to improve activities
• during training to monitor learning
• during training to improve facilitation

Answer 9

Formative Evaluation

Question 10

Tugsel realizing everyone in the class is very tired and deciding not to do the case study is an example of

Answer 10

Formative Evaluation - he is reading the crowd to monitor learning and improving facilitation / modifying

Question 11

This type of training evaluation is generally done 6-12 months after the training to measure Business Results

Answer 11

Summative Evalution

Question 12

Summative Evaluation determines which 2 things:

Answer 12

1 - Was the learning effective?

2- Were the Business Objectives met?

Question 13

4 Training Evaluation Models:

1- KIRKPATRICK MODEL:

Answer 13

4 STEPS: 
1 - did they LIKE it? (reaction)
2- did they LEARN it? (learning)
3- do they USE it? (behaviour)
4- did it improve RESULTS? (results)

Question 14

4 Training Evaluation Models:

2- PHILIPS MODEL

Answer 14

Added Level 5 (ROI) to Kirckpatrick’s Model

Question 15

4 Training Evaluation Models:

3- SUCCESS CASE METHOD

Answer 15

Return on Expectations - using a scorecard, collaborating with stakeholders to reach a business goal

Question 16

4 Training Evaluation Models:

4- COMA MODEL don’t need to know this

Answer 16

A training evaluation model that has 4 categories of
measurement:
 Cognitive (the trainees mastered the skills?)
 Organizational (the environment supports?)
 Motivation (motivation there?)
 Attitudes (right attidues to apply?)

Question 17

Salespeople received training on selling a new phone plan that wasn’t selling as well previously - the sales peoples sales numbers were compared before and after the training - this is an example of which level of Kirckpatrick’s Model?

Answer 17

• Level 4 - Results - Did it improve results?

Question 18

Salespeople received training on selling a new phone plan that wasn’t selling as well previously - the sales peoples sales numbers were compared before and after the training
A) this is an example of which level of Kirckpatrick’s Model?
B) If they were to evaluate the Gain of the Investment compared to the cost of the investment to figure out how much profit the training got the company(in dollars) they would be doing what?

Answer 18

• A) Level 4 - Results - Did it improve results?

- B) Measuring ROI - AKA level 5 - Philips Model

Question 19

Salespeople received training on selling a new phone plan that wasn’t selling as well previously - the sales peoples sales numbers were compared before and after the training
A) this is an example of which level of Kirckpatrick’s Model?
B) If they were to evaluate the Gain of the Investment compared to the cost of the investment to figure out how much profit the training got the company(in dollars) they would be doing what?

Answer 19

• A) Level 4 - Results - Did it improve results?

- B) Measuring ROI - AKA level 5 - Philips Model

Question 20

What is the Calculation of ROI?

Answer 20

[(Gain - Cost) / Cost] x 100

Question 21

If you wanted to measure Organizational Perceptions (how trainees perceive the organization’s culture and climate for learning and transfer) you would use which method of measurement?

Answer 21

Standardized Questionnaires

Question 22

Examples of how you would measure behaviour ?

Answer 22

• Self Reports
• Supervisory Reports
• Observations (Direct and Indirect)
• Production Records

Question 23

How would you measure Self Efficacy (the trainee confidence in learning and/or behaviour display on the job)

Answer 23

Questionnaires

Question 24

How would you measure Organizational Results post training?

Answer 24

Organizational Records

Question 25

The two methods to test / measure learning are:

Answer 25

Declarative: AKA M/C or T/F Tests

and

Procedural: Situational / Mastery tests

Question 26

3 Learning Variables Categories

Answer 26

1- Knowledge Retention Immediately Post-Training
2- Knowledge Retention A While after Training
3- Behaviour / Skills demonstration OJT once trainee has returned to work

Question 27

What are the 3 Basic Approaches to Measuring Behaviour?

Answer 27

Three basic approaches:
a) Self-reports—Trainee indicates if and/or how often newly trained behaviours are used on the job
b) Observations—others observe and record whether
and/or how often trainee used the newly trained
behaviours on the job
c) Production indicators—objective output assessed
through productivity records

Question 28

If you are measuring a trainees CONFIDENCE, you are measuring their…

Answer 28

SELF - EFFICACY

Question 29

What are are often viewed as the ultimate criteria and may include measures such as: productivity, quality, profitability, customer satisfaction, and accidents

Answer 29

RESULTS CRITERIA

Question 30

2 types of Results criteria data

Answer 30

 Hard data: results that are assessed objectively

 Soft data: results that are assessed through perceptions and judgments

Question 31

3 Types of Data Collection Designs are:

Answer 31

Post-only data collection design – “Have trainees attained a predetermined level?”

The pre-post design – “Have trainees changed?”

The time series design – “Do the post training results persist?”

Question 32

Compare the learning, skills, and/or performance of those who receive training to a comparable group of employees who do not.

When a significant difference is detected, the cause is the training program.

Answer 32

CAUSAL EVALUATION DESIGN