Chapter 5 Evaluating Learning Impact

Question 1

Which of the following is NOT a key benefit of the evaluation function? (Choose the best response.)

It improves the design of the learning experience

It determines whether objectives were met and to what extent

It determines the cause of the performance gap and appropriate remediation

It assesses the effectiveness and appropriateness of instructional strategies

Answer 1

It determines the cause of the performance gap and appropriate remediation

Response C is correct because determining the cause of a performance gap and the appropriate remediation are tasks completed during a human performance improvement project.

Question 2

Outline Expander

Which of the following best describes a reliable evaluation instrument? (Choose the best response.)

The extent to which an instrument is consistent enough that subsequent measures of an item give the same approximate results

The extent to which the instrument represents the content of the program

The degree to which an instrument represents the construct it was meant to represent

The extent to which an instrument agrees with the results of other instruments administered at approximately the same time to measure the same characteristics

Answer 2

The extent to which an instrument is consistent enough that subsequent measures of an item give the same approximate results

Response A is correct because reliability refers to the ability of the same instrument to produce consistent results over time.

Question 3

Outline Expander

Which of the following best describes construct validity? (Choose the best response.)

The extent to which an instrument produces consistent results over time

The extent to which the instrument represents the content of the program

The degree to which an instrument represents the construct it was meant to represent

The extent to which an instrument agrees with the results of other instruments administered at approximately the same time to measure the same characteristics

Answer 3

The degree to which an instrument represents the construct it was meant to represent

Response C is correct because construct validity indicates the degree to which an instrument represents the construct—the abstract variable, such as the skill or ability—that the instrument is supposed to measure. Construct validity can be defended through expert opinion, correlations, logical deductions, and criterion group studies.

Question 4

Outline Expander

Which of the following best describes concurrent validity? (Choose the best response.)

The extent to which an instrument is consistent enough that subsequent measures of an item give the same approximate results

The extent to which the instrument represents the content of the program

The degree to which an instrument represents the construct it was meant to represent

The extent to which an instrument agrees with the results of other instruments administered at approximately the same time to measure the same characteristics

Answer 4

The extent to which an instrument agrees with the results of other instruments administered at approximately the same time to measure the same characteristics

Response D is correct because concurrent validity indicates how much an instrument agrees with the results of other instruments administered at approximately the same time to measure the same characteristics. It is the exact definition

Question 5

Which of the following best describes formative evaluation? (Choose the best response.)

Ensuring that every objective has a corresponding evaluation task

Assessing the effectiveness of a training program while it’s being developed

Response B is correct because formative evaluation happens while the training is being developed to ensure that the training program is understandable, accurate, and functional.

Assessing the effectiveness of a training program when it’s completed and has been pilot tested

Assessing training transfer

Answer 5

Assessing the effectiveness of a training program while it’s being developed

Response B is correct because formative evaluation happens while the training is being developed to ensure that the training program is understandable, accurate, and functional.

Question 6

Which of the following best describes summative evaluation? (Choose the best response.)

Ensuring that every objective has a corresponding evaluation task

Assessing the effectiveness of a training program while it’s being developed

Assessing the effectiveness of a training program when it’s completed and has been pilot tested

Assessing training transfer

Answer 6

Assessing the effectiveness of a training program when it’s completed and has been pilot tested

Response C is correct because summative evaluation assesses the effectiveness of a training program that’s already available.

Question 7

Which of Kirkpatrick’s four levels of evaluation assesses the learner’s ability to transfer learning on the job and usually occurs a few weeks to three months after the training event? (Choose the best response.)

Collapse AllExpand All

Level 1

Level 2

Level 3

Level 4

Answer 7

Level 3

Response C is correct because Level 3 (behavior) measures new or changed behavior and performance back on the job. It answers the question, “Can the learner use the information?”

Question 8

Which of Kirkpatrick’s four levels of evaluation assesses the learner’s reaction to the training program and is usually administered at the end of the training event? (Choose the best response.)

Level 1

Level 2

Level 3

Level 4

Answer 8

Level 1

Response A is correct because Level 1 (reaction) measures the satisfaction or dissatisfaction with the training at the end of the program. It answers the question, “Did the learner enjoy the training?”

Question 9

Which evaluation method focuses on converting data into monetary values and comparing it to costs? (Choose the best response.)

Collapse AllExpand All

Balanced scorecard approach

Kirkpatrick’s Level 3

Kirkpatrick’s Level 4

Phillips ROI Methodology

Answer 9

Phillips ROI Methodology

Response D is correct because the Phillips ROI Methodology focuses on benefit-cost ratios and converting Level 4 measures to monetary value and comparing that value with the cost of the training. It answers the question, “Did the training have an effect on business outcomes?”

Question 10

Which of the following evaluation methods encourages the analysis of each evaluation step to ensure the quality of the evaluation? (Choose the best response.)

Formative

Meta

Summative

ROI

Answer 10

Meta

Response B is correct because meta-evaluation is a measure of how well an evaluation performed to ensure the quality of an evaluation; to provide credibility for the evaluation, if necessary; and to improve subsequent evaluations.

Question 11

Which of the following is an example of a learning (Level 2) evaluation? (Choose the best response.)

Paper-and-pencil test

Participants’ course feedback

Electronic survey

Management’s feedback

Answer 11

Paper-and-pencil test

Response A is correct because a Level 2 evaluation generally measures the knowledge gained from the training program. It is generally done with paper and pencil, although other methods can be used, depending on the delivery method.

Question 12

An assessment of on-the-job performance is an example of which level of evaluation? (Choose the best response.)

Reaction (Level 1)

Learning (Level 2)

Behavior (Level 3)

Results (Level 4)

Answer 12

Behavior (Level 3)

Response C is correct because Level 3 evaluation measures new or changed behavior and performance back on the job.

Question 13

A TD professional wants to determine if a training course resulted in actual performance improvement on the job. Which tools should he use? (Choose the best response.)

Online or paper-and-pencil tests

Opinion surveys or evaluation forms

Pre-training or post-training surveys

Productivity reports or on-site observations

Answer 13

Productivity reports or on-site observations

Response D is correct because both productivity reports and on-site observations indicate whether performance has improved since the employee attended training.

Question 14

A TD professional is in the process of designing a training program. Her supervisor asks what she is doing to ensure her program is going to meet stakeholder needs. Which of the following is a method to ensure her success? (Choose the best response.)

Summative evaluations

Formative evaluations

Pre-Training surveys

Business impact form

Answer 14

Formative evaluations

Response B is correct because the purpose of formative evaluative is to improve the draft training program and increase the likelihood that it will achieve its objectives when it is generally available.

Question 15

A TD professional is preparing to conduct an evaluation. What is the first step in the process? (Choose the best response.)

Develop an evaluation design and strategy

Select measurement tools

Identify evaluation goals

Answer 15

Identify evaluation goals

Response C is correct because the steps in all evaluation models include: identifying evaluation goals first, developing an evaluation design and strategy, selecting and constructing measurement tools, analyzing data, and reporting data.

Question 16

A TD professional is evaluating his company’s organizational effectiveness by using an approach that uses more than just financial measures and consists of using four perspectives. Which of the following methodologies is he applying? (Choose the best response.)

Phillips ROI Methodology

Kirkpatrick’s four levels of evaluation

Balanced scorecard approach

Business analysis

Answer 16

Balanced scorecard approach

Response C is correct because the balanced scorecard approach uses four perspectives that go beyond financial measures to evaluate organizational effectiveness. The four perspectives are the customer perspective (Did the solution meet the customer’s needs or expectations?), the innovation and learning perspective (Did users gain the needed skills or knowledge?), the internal business perspective (Did the solution have an effect on the job?), and the financial perspective (Did the solution have a financial payoff?).

Question 17

Which of the following measures of central tendency reflects the most frequently occurring score in a distribution? (Choose the best response.)

Average

Mean

Median

Mode

Answer 17

Mode

Response D is correct because the mode is the value that occurs most frequently in a sample.

Collapse All

Question 18

Which of the following best describes a situation where the mode < median < mean? (Choose the best response.)

Negative skewness

Positive skewness

Outlier

Normal distribution

Answer 18

Positive skewness

Response B is correct because skewness refers to asymmetry in the distribution of sample data values. When the mean is greater than the median, which is greater than the mode, the situation is referred to as positive skewness. This situation is more common than negative skewness.

Question 19

Which of the following best describes normal distribution? (Choose the best response.)

An observation in a data set that’s further from others in the data set

The symmetry in the distribution of the same data values

The way in which observations tend to gather around the mean, also known as the bell-shaped curve

Variation in values that could be widely scattered or tightly clustered

Answer 19

The way in which observations tend to gather around the mean, also known as the bell-shaped curve

Response C is correct because normal distribution describes the way that values tend to cluster around a certain value rather than spread evenly across a whole range of values.

Question 20

Which of the following best describes dispersion? (Choose the best response.)

An observation in a data set that’s further from the others in the data set

The symmetry in the distribution of the same data values

The way in which observations tend to gather around the mean, also known as the bell-shaped curve

Variation in values that could be widely scattered or tightly clustered

Answer 20

Variation in values that could be widely scattered or tightly clustered

Response D is correct because dispersion refers to how observations in a data set are spread out around their mean. The most common measure of dispersion is the standard deviation.

Question 21

Which of the following best describes an outlier? (Choose the best response.)

An observation in a data set that’s further from the others in the data set

The symmetry in the distribution of the same data values

The way in which observations tend to gather around the mean, also known as the bell-shaped curve

Variation in values that could be widely scattered or tightly clustered

Answer 21

An observation in a data set that’s further from the others in the data set

Response A is correct because an outlier is a data point that is far away from the others in a data set

Question 22

Which of the following types of data makes it possible to rank order items measured by which has less or more of the quality represented? (Choose the best response.)

Nominal

Ordinal

Interval

Ratio

Answer 22

Ordinal

Response B is correct because ordinal data are numbers or variables that allow ranking order from high to low.

Question 23

Outline Expandering types of data identifies an absolute zero point? (Choose the best response.)

Nominal

Ordinal

Interval

Ratio

Answer 23

Ratio

Response D is correct because ratio data differs from interval data in that they have an absolute zero point. This allows for greater analytical versatility because statements such as “x is two times more than y” can be made.

Question 24

For which of the following reasons do TD professionals convert raw scores to standard scores? (Choose the best response.)

To measure the relationship between two or more variables

To reflect where they fall with respect to the mean to allow scores to be compared and interpreted

To understand the cause-and-effect connections between variables

Because they are always expressed as a number between –1.00 and +1.00

Answer 24

For which of the following reasons do TD professionals convert raw scores to standard scores? (Choose the best response.)

To reflect where they fall with respect to the mean to allow scores to be compared and interpreted

Response B is correct because converting raw scores, or the number of correct scores, to standard scores allows comparison of scores by standardizing or equating different metrics. Percentiles are an example of standard scores and indicate how a score compares with others on a scale of 0 to 100.

Question 25

A relational study statistic that measures the relationship between two variables is a ______________. (Choose the option that completes the sentence accurately.)

Correlation coefficient

Cause-and-effect connection

Normal distribution

Skewness

Answer 25

A relational study statistic that measures the relationship between two variables is a ______________. (Choose the option that completes the sentence accurately.)

Correlation coefficient

Response A is correct because a correlation coefficient describes the strength and direction of the relationship between two variables and is always expressed as a number between –1.00 and +1.00.

Question 26

Which of the following is NOT a major problem regarding the use of evaluation data? (Choose the best response.)

Organizations do not apply confidence intervals

Organizations do not use evaluation data

Data are not used to drive change

Organizations use evaluation data incorrectly

Answer 26

Which of the following is NOT a major problem regarding the use of evaluation data? (Choose the best response.)

Organizations do not apply confidence intervals

Response A is correct because this is not a major problem regarding the use of evaluation data. The other three responses are examples of inappropriate use of data.

Question 27

Which of the following are measures of central tendency? (Choose the best response.)

Mean only

Mode only

Mean and median only

Mean, median, and mode

Answer 27

Mean, median, and mode

Response D is correct because measures of central tendency are various ways of looking at averages and include all three of the following: mean, median, and mode.

Question 28

In the measurement of the distribution of a group of numbers, which represents the middle number where half of the numbers are above and half are below it? (Choose the best response.)

Mean

Mode

Median

Standard deviation

Answer 28

Median

Response C is correct because the median is the number directly in the middle of the numbers.

Question 29

A TD professional conducts a study that yields a correlation of 0.48 between whether individuals take a specific leadership course and whether they are subsequently promoted during the following calendar year. Based on this correlation, what can the TD professional conclude? (Choose the best response.)

Taking the course and receiving a promotion are strongly related

People who take the course are less likely to be promoted

Taking the course directly affects the individual’s likelihood of being promoted

No conclusions can be made because only dichotomous variables were used

Answer 29

Taking the course and receiving a promotion are strongly related

Response A is correct because a correlation of 0.48 indicates a strong positive relationship between two variables. This relationship indicates that as the training variable increases, so does the promotion variable.

Question 30

A training department is trying to determine whether there is a difference in learner performance between classroom and online learning participants. Which of the following should the department use to quantify the difference between these two groups? (Choose the best response.)

Confidence interval

Effect size

Standard deviation

Variance

Answer 30

Effect size

Response B is correct because effect size is a way to quantify the difference between two groups; effect size uses standard deviation to contextualize the difference between the two groups.

Question 31

Which of the following is controlled by the evaluator when splitting groups into treatment groups and control groups? (Choose the best response.)

Dependent variable

Independent variable

Extraneous variable

Effect sizes

Answer 31

Independent variable

Response B is correct because the evaluator controls the independent variable when separating groups into treatment and control groups. The independent variable is the influencing variable. For example, age, seniority, gender, and level of education may all be factors (independent variables) that influence a person’s performance (the dependent variable).

Question 32

In which of the following sampling methods does each person in the population have an equal chance of being chosen for the sample? (Choose the best response.)

Random selection

Random sampling

Random assignment

Stratified random sampling

Answer 32

Random sampling

Response B is correct because random sampling involves selecting people randomly from the population. For example, selecting every 10th person from an alphabetical list of names creates a systematic random sample. As a result, each person in the population has an equal chance of being selected.

Question 33

Which of the following is the process for finding the sample of people for a study from the population? (Choose the best response.)

Random selection

Random sampling

Random assignment

Stratified random sampling

Answer 33

Random selection

Response A is correct because random selection is the process of drawing people randomly from a population to participate in the study.

Question 34

Which of the following is an undesired variable that influences the relationship between variables that an evaluator is examining? (Choose the best response.)

Dependent variable

Independent variable

Extraneous variable

Effect sizes

Answer 34

Extraneous variable

Response C is correct because an extraneous variable is an undesirable variable that influences the relationship between the variables that the evaluator is examining.

Question 35

Which of the following sources of bias may be a source of measurement error when participants know they are being watched and evaluated? (Choose the best response.)

Sampling bias

Observation bias

Bias of central tendency

Emotional bias

Answer 35

Observation bias

Response B is correct because observation bias refers to the way that people alter their behavior in response to being observed. The more visible the observation process, the less reliable the data are.

Question 36

Which of the following sources of bias may be a source of measurement error because participants are hesitant to choose either end of a measurement scale and instead indicate responses near the middle? (Choose the best response.)

Sampling bias

Observation bias

Bias of central tendency

Emotional bias

Answer 36

Bias of central tendency

Response C is correct because bias of central tendency happens when participants avoid selecting either the high or the low end of a rating scale and select the middle rating instead. To avoid this, the evaluator can drop the middle value of a scale, but should first be sure that bias of central tendency is truly happening.

Question 37

Which of the following helps identify the items maintaining the status quo and helps clarify approaches needed to facilitate change? (Choose the best response.)

Likert scale

Ishikawa diagram

Hypothesis testing

Force field analysis

Answer 37

Force field analysis

Response D is correct because force field analysis, which was developed by Kurt Lewin, helps identify forces that maintain the status quo and clarify approaches needed to facilitate change. Many factors play a positive or negative role in change; the positive factors are referred to as “driving forces,” and the negative factors are referred to as “resisting forces.”

Question 38

Which of the following is best described as planning and organizing an experiment to ensure that the right type of data, and enough of it, is available to answer questions of interest as clearly and efficiently as possible? (Choose the best response.)

One-way analysis of variance

Sampling

Experimental design

Qualitative research

Answer 38

Experimental design

Response C is correct because experimental design refers to the process of identifying the specific questions that an experiment is designed to answer and organizing the experiment to get the right type of data in sufficient quantities.

Question 39

Which of the following is a characteristic of hard data? (Choose the best response.)

Anecdotal

Opinion

Attitude

Objective

Answer 39

Objective

Response D is correct because hard data are the result of quantitative methods of data collection. Hard data are objective and measurable and can be stated according to frequency, percentage, proportion, or time.

Question 40

Which of the following is NOT an example of a descriptive statistic? (Choose the best response.)

Mean

Variance

Skewness

Analysis of variance

Answer 40

Analysis of variance

Response D is correct because analysis of variance is not a descriptive statistic; it is an experimental design model.

Question 41

Which method provides the most compelling evidence for the effect of online learning but is sometimes infeasible in a business environment? (Choose the best response.)

Using a control group to isolate the effects of online learning

Performing a trend line analysis focusing on the point at which the online learning was implemented

Soliciting input on the extent to which online learning influenced a particular change

Determining how many learners completed courses of study

Answer 41

Using a control group to isolate the effects of online learning

Response A is correct because control groups do offer the most objective evidence of effectiveness, but they also consume time and resources not often available in a business setting unless the training is part of a high-priority strategic initiative.

Question 42

Which of the following has the most impact on the extent to which a research study’s results can be generalized? (Choose the best response.)

Expand All

Sampling

Instrumentation

Dissemination method

Target population size

Answer 42

Sampling

Response A is correct because sampling is used to gather a cross-section of the population with all portions of the population being represented. The researcher can generalize the results based on a representative sample.

Question 43

A TD professional is performing a needs analysis. He interviews every 13th employee from the department, selected from an alphabetical list of departmental personnel. Within the department, he separates personnel into supervisor and nonsupervisor. What sampling method is this professional using? (Choose the best response.)

Simple random sample

Convenience sample

Stratified random sample

Judgment sample

Answer 43

Stratified random sample

Response C is correct because dividing a population into groups, such as departments, and then randomly choosing from each group is an example of stratified random sampling.

Question 44

A training department in a manufacturing company is seeking to improve its training feedback process. The instructor distributes a 10-question feedback survey to all participants at the end of the class and asks them to include their contact information on the form. She collects the feedback sheets as the participants leave the training room. Which of the following changes is most likely to help improve the quality of the feedback information? (Choose the best response.)

Increase the number of feedback questions

Have a neutral person administer the survey

Include short-answer question types

Survey only a sample of the participants

Answer 44

Have a neutral person administer the survey

Response B is correct because having a neutral person administer the survey increases the objectivity of the input and decreases the likelihood of the instructor or facilitator reacting unfavorably to criticism in the feedback.

Question 45

Which of the following formulas can forecast how long it will take for a training program to return the investment? (Choose the best response.)

(Program Benefits for First Year × Life of Program in Years) – Program Costs

Program Costs ÷ Total Benefits

[(Total Benefits – Program Costs) ÷ Program Costs] × 100

(Program Costs – Total Benefits) ÷ Total Benefits

Answer 45

Program Costs ÷ Total Benefits

Response B is the break-even calculation and can be used to determine when the program will return the investment.

Question 46

Which of the following analysis methods includes variables such as the standard deviation of job performance and the true difference in job performance between average trained and average untrained employees? (Choose the best response.)

Utility analysis

BCR

BE analysis

ROI

Answer 46

Utility analysis

Response A is correct because utility analysis measures a program’s contribution according to how effective the program was in identifying and modifying behavior and, therefore, the future service contribution of employees. TD professionals can use the Brogden utility estimation equation to estimate the monetary value of a training program; Brogden’s equation includes variables such as standard deviation of job performance of the untrained group and the difference in job performance between the average trained and average untrained employee.

Question 47

How can TD professionals use statistical forecasting models to improve training evaluation? (Choose the best response.)

Simplify the experimental design

Provide assessment results sooner

Predict future performance levels

Decrease the need for data collection

Answer 47

Predict future performance levels

Response C is correct because TD professionals can use statistical forecasting models to extrapolate future performance levels with the assumption that no training is conducted. The value of the forecasted performance can then be compared with the actual value to estimate the effect of training.

Question 48

When calculating the BE (break-even) of a project, the resulting value is 1.0. What is the meaning of that value? (Choose the best response.)

When the BE is 100 percent, that means one dollar is returned in net benefits for every dollar invested. In other words, the program returned the dollar invested plus one dollar in benefit

When the BE is 1.0, that means that one dollar is returned in benefits for every dollar invested. Said another way, the program broke even

When the BE is 1.0, it indicates that it will take one year for the program to return the investment

When the BE is 1.0, it indicates that the training program was effective in modifying training behavior and the organization is receiving one dollar more in value from each employee who completed the training program

Answer 48

When the BE is 1.0, it indicates that it will take one year for the program to return the investment

Response C is correct because BE calculates how long it will take for a program to return the investment. The calculation is program costs or investment divided by total benefits or cash flow. The answer is then multiplied by 12 (for 12 months) to determine how many months it will take to break even.

Question 49

Which of the following best describes utility analysis? (Choose the best response.)

Isolates the effects of training when mathematical relationships between input and output variables are known

Measures a program’s economic contribution according to how effective the program was in identifying and modifying behaviors

Proves that a solution returned the investment or generated more financial benefit than other costs

Ensures that the data influenced by a solution were not influenced by other factors

Answer 49

Measures a program’s economic contribution according to how effective the program was in identifying and modifying behaviors

Response B is correct because utility analysis determines how effective a program was based on the level of change in employees’ behavior and the employees’ future service contribution.

Question 50

Which of the following best describes a forecasting model? (Choose the best response.)

Isolates the effects of training when mathematical relationships between input and output variables are known

Measures a program’s economic contribution according to how effective the program was in identifying and modifying behaviors

Proves that a solution returned the investment or generated more financial benefit than other costs

Ensures that the data influenced by a solution were not influenced by other factors

Answer 50

Isolates the effects of training when mathematical relationships between input and output variables are known

Response A is correct because a forecasting model predicts output based on no training, which is then compared with the actual output after the training.

Question 51

A TD professional is creating a Level 1 evaluation instrument for a training class. Which of the following does NOT need to be considered? (Choose the best response.)

Determining the purpose(s) the tool will serve, such as will it be used only for a certain type of instruction, and will it be used for TD products, services, or both

Determining the format or media that will be used to present and track results such as paper, electronic, or both

Selecting the items that are important for the business to track and that will help maintain the quality of TD products and services

Calculating the BCR to decide if the class should continue to be offered or replaced with a different class

Answer 51

Calculating the BCR to decide if the class should continue to be offered or replaced with a different class

Response D is correct because a benefit-cost ratio is calculated as part of the ROI, which is a Level 4 evaluation.

Question 52

A TD professional is demonstrating ROI values to her manager. Which of the following is a formula she can use? (Choose the best response.)

(Program Benefits for First Year × Life of Program in Years) – Program Costs

Program Costs ÷ Total Benefits

[(Total Benefits – Program Costs) ÷ Program Costs] × 100

(Program Costs – Total Benefits) ÷ Total Benefits

Answer 52

[(Total Benefits – Program Costs) ÷ Program Costs] × 100

Response C is correct because the ROI calculation is [(Total Benefits – Program Costs) ÷ Program Costs] × 100.

Question 53

Most inaccuracies in survey completion occur because of which of the following? (Choose the best response.)

Respondent’s misinterpretation of the directions

Respondent’s lack of interest

Respondent’s skill level

Respondent’s opinions

Answer 53

Respondent’s misinterpretation of the directions

Response A is correct because unclear or confusing directions cause the respondent to complete survey questions incorrectly.

Question 54

Which of the following is NOT one of the major tasks in analyzing data? (Choose the best response.)

Sorting data

Tabulating data

Developing spreadsheets

Comparing raw data with condensed data

Answer 54

Developing spreadsheets

Response C is correct because it is not one of the major tasks in analyzing data. However, it maybe a useful tool for tabulating the data.

Question 55

Which of the following synthesis models contains input-process-output items, decision points, and direction of flow to organize and synthesize information? (Choose the best response.)

Reflection

Two-axis matrix

Three-axis matrix

Flowchart

Answer 55

Flowchart

Response D is correct because flowcharts contain input-process-output items, decision points, direction of flow, documentation or preparation steps, and confluence or divergence.

Question 56

Which of the following synthesis models was used by Chris Argyris and is a way to approach ambiguous pieces of information by fitting data into two mutually exclusive groups or contradictory positions? (Choose the best response.)

Events network

Dichotomy

Argumentation

Graphic models

Answer 56

Dichotomy

Response B is correct because Chris Argyris used this model when he subsumed management practice into two categories—espoused theory and theory in use—to show contradictions between what managers said and what they did.

Question 57

Which of the following best describes an executive summary? (Choose the best response.)

It explains the methods used to collect data and typically includes samples of the data collected

It’s a brief overview of the entire report, explaining the basis for the evaluation and significant conclusions and recommendations, and is designed for those who need only the most important information

It describes any unexpected issues or environmental concerns that arose during the evaluation

It describes why the evaluation was conducted and gives a general description of the TD component, practice, product, service, or process being evaluated. The evaluation’s objectives are presented here as well as information about what was being evaluated

Answer 57

It’s a brief overview of the entire report, explaining the basis for the evaluation and significant conclusions and recommendations, and is designed for those who need only the most important information

Response B is correct because the executive summary provides a brief summary of the most important points of the entire report.

Question 58

When presenting evaluation results, it is important for the TD professional to do which of the following? (Choose the best response.)

Use a dynamic presentation style to maintain audience interest

Use a simple presentation style so that the focus is on the content

Use a presentation style that matches the purpose of the presentation

Use a graphically driven presentation style to assist visual learners

Answer 58

Use a presentation style that matches the purpose of the presentation

Response C is correct because the presentation of the findings report should match the purpose of the presentation, including required level of detail for data, findings, and backup documentation; business purpose of the evaluation; and use of results.

Question 59

Which of the following is NOT one of the assumptions for the chi-square test for a single variance? (Choose the best response.)

The sample must be selected randomly from the population

The population must be normally distributed for the variable that is being studied

The observations must be separate

The respondents’ opinions must be cross-tabulated

Answer 59

The respondents’ opinions must be cross-tabulated

Response D is the correct response because cross-tabulating respondents’ opinions is not an assumption of the chi-square test.

Question 60

Which of the following is NOT one of the primary considerations when planning and disseminating learning impact study results? (Choose the best response.)

Timeliness and quality of the communication is critical

Never anticipate that all stakeholders will receive the evaluation report results positively

Tailor the report to meet the unique needs and interests of each stakeholder group—for example, by considering the most effective media or format to provide the data wanted to that particular group

Including testimonials in the evaluation results report always has a profound effect on how effectively messages in the learning study are received

Answer 60

Including testimonials in the evaluation results report always has a profound effect on how effectively messages in the learning study are received

Response D is correct because testimonials from individuals who have little respect within the organization can have a negative effect on the message and credibility or perception of the learning impact report results.

Question 61

A TD professional is completing a presentation to justify the cost of delivering a cultural awareness training program. When preparing the presentation, which of the following should he consider? (Choose the best response.)

Methodology used in receiving formative and summative evaluation information

The synthesis model used to determine the recommendation

Business purpose of the evaluation regarding key decisions the audience needs to make

That the executive summary should be three paragraphs or shorter

Answer 61

Business purpose of the evaluation regarding key decisions the audience needs to make

Response C is the correct response because this will help match the style of data presentation to the audience, and that will assist in creating a receptive audience.

Question 62

A TD professional is looking for the best way to organize information that contains inputs, processes, and outputs. Which synthesis model best supports this strategy? (Choose the best response.)

Three-axis matrix

Flowchart

Events network

Graphic model

Answer 62

Flowchart

Response B is correct because flowcharts contain input-process-output items, decision points, direction of flow, documentation or preparation steps, and confluence and divergence. None of the other synthesis models include this type of information.

Question 63

A TD professional is writing a section of an evaluation report that outlines the components of the evaluation process and identifies the tools used. What section of the report is the TD professional writing? (Choose the best response.)

Background information

Evaluation methodology

Data collection and analysis

Evaluation results

Answer 63

Evaluation methodology

Response B is correct because the evaluation methodology component of an evaluation report describes the evaluation process, including the tools used, as well as the purpose of the evaluation and a rationale for the evaluation design.