Chapter 4 Flashcards
What is not a key benefit of the elevation function?
Determining the cause of the performance gap and appropriate remediation.
What best describes a reliable evaluation instrument?
The extent to which an instrument is consistent enough that the subsequent measures of an item give the same approximate results.
What best describes construct validity?
The extent to which the instrument represents the content of the program.
What best describes formative evaluation?
Assessing the effectiveness of a training program while its being developed.
What best describes concurrent validity?
The extent to which an instrument agrees with the results of other instruments administered at approximately the same time to measure the same characteristics.
What best describes summative evaluation?
Assessing the effectiveness of a training program when its completed and has been test piloted.
Which of Kirkpatrick’s 4 levels of evaluation assesses the learners ability to transfer learning on the job and usually occurs a few weeks to 3 months after the training event.
Level 3
Which of Kirkpatrick’s 4 levels of evaluation assesses the learners reaction to the training program and is usually administered at the end of the training event?
Level 1
Which evaluation methods encourages the analysis of each step to ensure the quality of the evaluation?
Meta
Which evaluation method focuses on converting data into monetary values and comparing it to costs?
Phillips ROI Methodology
What is an example of a learning (Level 2) evaluation?
Paper and Pencil Test
An assessment of on the job performance is an example of which level of evaluation?
Behavior (Level 3)
What measures the central tendency reflecting the most frequently occurring score in a distribution?
Mode
What best describes a situation where the mode
Positive Skewness, refers to assembly in the distribution of sample.
What best describes a normal distribution?
The way which observations tend to gather around the mean, also known as a bell shaped curve.
What best describes dispersion?
Variation in values that could be widely scattered or tightly clustered.
What best describes an outlier?
An observation in a data set that’s further from others in the data set.
What types of data make it possible to rank order items measured by which has less or more quality represented?
Ordinal
What identifies an absolute zero point?
Interval
For which reasons do TD professionals convert raw scores to standard scores?
To reflect where they fall with respect to the mean to allow scores to be compared and interpreted.
A relational study statistic that measures the relationship between 2 variables is a…
Correlation Coefficient, describes the strength and direction of the relationship.
What is not a major problem regarding the use of evaluation data?
Organizations do not apply confidence intervals.
What are measures of central tendency?
Mean, Median, Mode
In a measurement of the distribution of a group of numbers, which represents the middle number where half of the numbers are above half are below?
Median
What is controlled by the evaluator when splitting groups into treatment group and control groups?
Independent Variable
Sampling methods does each person in the population have an equal chance of being chosen for the sample?
Random Sampling
Process of finding the sample of people for a study from the population?
Random Selection
Source of bias may be a source of measurement error when participants know they are being watched?
Observation Bias
Source of bias be a source of measurement error because participants are hesitant to choose either end of a measurement scale and instead respond in near the middle?
Bias of central tendency
Helps identify the item maintaining the status quo and helps clarify approaches needed to facilitate change?
Force Field Analysis
Best described as planning and organizing an experiment to ensure that the right type of data, and enough of it, is available to answer questions of interest as clearly and efficiently as possible?
Experimental Design
Characteristic of hard data?
Objective
NOT an example of descriptive statistic?
Analysis of variance
Method provides the most compelling evidence for the effect of online learning but is sometimes infeasible in a business environment?
Using a control group to isolate the effects of online learning.
Most impact on the extent to which a research study’s results can be generalized?
Sampling
What formula can forecast how long it will take for a training program to return the investment?
Program costs/total benefits
What analysis method includes variables such as the standard deviation of job performance and the true difference in job performance between average trained and average untrained employees?
Utility Anlysis
How can TD professionals use statistical forecasting models to improve training evaluation?
Predict future performance levels
When calculating the BE (Break-Even) of a project the resulting value is 1.0 What is the meaning of that value?
When BE is 1.0 it indicates it will take one year for the program to return the investment.
What best describes utility analysis?
Measures a programs economic contribution according to how effective the program was in identifying and modifying behaviors.
What best describes a forecasting model?
Isolates the effects of training when mathematical relationships between input and output variables are known.
Most inaccuracies in survey completion occur because of what?
Respondents misrepresentation of the directions
What is not one of the major tasks in analyzing data?
Developing spreadsheets
What synthesis model contains input-process-output items, decision points, and direction of flow to organize and synthesize information.
Flowchart
Synthesizes model used by Chris Argyris and is a way to approach ambiguous pieces of information by fitting data into 2 mutually exclusive groups or contradictory positions.
Dichotomy
What best describes an executive summary?
Brief overview of the entire report, explaining the basis for the evaluation and significant conclusions and recommendations, and is designed for those who only need the most important information.
When presentation evaluation results, it is important for the TD professional to do what?
Use a presentation style that matches the purpose of the presentation
What is NOT one of the assumptions for the Chi-square test for single variance?
The respondents opinions must be cross-tabulated
What is NOT one of the primary considerations when planning and disseminating learning impact study results?
Include testimonials in the evaluation results report always has a profound effect on how effectively messages in the learning study are received,
