Week 11: Training Evaluation Flashcards
What is this the Definition of?:
A process to assess the value or worthiness of training
programs to employees and to organizations.
A continuum of techniques, methods, and measures.
Training Evaluation
List 4 Reasons Training Evaluation is of value to organizations:
1- Improve training
2- Determine the programs most useful to employees and determine who should be trained
3- Determine cost benefits and cost-effectiveness of a program
4- Determine if a program has achieved expected results or solved the problem it was meant to solve
Just because you can measure everything, should you?
No, not always.
Why is it important to have metrics?
Without metrics, you’re “just another person with an opinion.”
What are the 2 types of BARRIERS to Training Evaluation?
Pragmatic and Political
Give 3 Examples of “Pragmatic” barriers to Training Evaluation:
• Evaluation requires knowledge of evaluation models,
research design, measurement, questionnaire
construction, and data analysis (may be intimidating)
• Time and effort
• Financial costs
Give 3 Examples of “Political” barriers to Training Evaluation:
- Absence of accountability
- Evaluation can be threatening
- Ethical concerns; a belief that evaluation should be externally conducted (think about Makers and Checkers example)
Two types of Training Evaluation are:
1- Formative (Throughout, i.e. reading the crowd)
2- Summative (After)
This type of training evaluation happens:
- during development to modify and improve the program
- during training to improve activities
- during training to monitor learning
- during training to improve facilitation
Formative Evaluation
Tugsel realizing everyone in the class is very tired and deciding not to do the case study is an example of
Formative Evaluation - he is reading the crowd to monitor learning and improving facilitation / modifying
This type of training evaluation is generally done 6-12 months after the training to measure Business Results
Summative Evalution
Summative Evaluation determines which 2 things:
1 - Was the learning effective?
2- Were the Business Objectives met?
4 Training Evaluation Models:
1- KIRKPATRICK MODEL:
4 STEPS: 1 - did they LIKE it? (reaction) 2- did they LEARN it? (learning) 3- do they USE it? (behaviour) 4- did it improve RESULTS? (results)
4 Training Evaluation Models:
2- PHILIPS MODEL
Added Level 5 (ROI) to Kirckpatrick’s Model
4 Training Evaluation Models:
3- SUCCESS CASE METHOD
Return on Expectations - using a scorecard, collaborating with stakeholders to reach a business goal
4 Training Evaluation Models:
4- COMA MODEL don’t need to know this
A training evaluation model that has 4 categories of
measurement:
Cognitive (the trainees mastered the skills?)
Organizational (the environment supports?)
Motivation (motivation there?)
Attitudes (right attidues to apply?)
Salespeople received training on selling a new phone plan that wasn’t selling as well previously - the sales peoples sales numbers were compared before and after the training - this is an example of which level of Kirckpatrick’s Model?
- Level 4 - Results - Did it improve results?
Salespeople received training on selling a new phone plan that wasn’t selling as well previously - the sales peoples sales numbers were compared before and after the training
A) this is an example of which level of Kirckpatrick’s Model?
B) If they were to evaluate the Gain of the Investment compared to the cost of the investment to figure out how much profit the training got the company(in dollars) they would be doing what?
- A) Level 4 - Results - Did it improve results?
- B) Measuring ROI - AKA level 5 - Philips Model
Salespeople received training on selling a new phone plan that wasn’t selling as well previously - the sales peoples sales numbers were compared before and after the training
A) this is an example of which level of Kirckpatrick’s Model?
B) If they were to evaluate the Gain of the Investment compared to the cost of the investment to figure out how much profit the training got the company(in dollars) they would be doing what?
- A) Level 4 - Results - Did it improve results?
- B) Measuring ROI - AKA level 5 - Philips Model
What is the Calculation of ROI?
[(Gain - Cost) / Cost] x 100
If you wanted to measure Organizational Perceptions (how trainees perceive the organization’s culture and climate for learning and transfer) you would use which method of measurement?
Standardized Questionnaires
Examples of how you would measure behaviour ?
- Self Reports
- Supervisory Reports
- Observations (Direct and Indirect)
- Production Records
How would you measure Self Efficacy (the trainee confidence in learning and/or behaviour display on the job)
Questionnaires
How would you measure Organizational Results post training?
Organizational Records
The two methods to test / measure learning are:
Declarative: AKA M/C or T/F Tests
and
Procedural: Situational / Mastery tests
3 Learning Variables Categories
1- Knowledge Retention Immediately Post-Training
2- Knowledge Retention A While after Training
3- Behaviour / Skills demonstration OJT once trainee has returned to work
What are the 3 Basic Approaches to Measuring Behaviour?
Three basic approaches:
a) Self-reports—Trainee indicates if and/or how often newly trained behaviours are used on the job
b) Observations—others observe and record whether
and/or how often trainee used the newly trained
behaviours on the job
c) Production indicators—objective output assessed
through productivity records
If you are measuring a trainees CONFIDENCE, you are measuring their…
SELF - EFFICACY
What are are often viewed as the ultimate criteria and may include measures such as: productivity, quality, profitability, customer satisfaction, and accidents
RESULTS CRITERIA
2 types of Results criteria data
Hard data: results that are assessed objectively
Soft data: results that are assessed through perceptions and judgments
3 Types of Data Collection Designs are:
Post-only data collection design – “Have trainees attained a predetermined level?”
The pre-post design – “Have trainees changed?”
The time series design – “Do the post training results persist?”
Compare the learning, skills, and/or performance of those who receive training to a comparable group of employees who do not.
When a significant difference is detected, the cause is the training program.
CAUSAL EVALUATION DESIGN
