8.1 Applying Metrics To Objectives: Selecting The Preferred Design

Question 1

What is the next step after you have generated several feasible designs?

Answer 1

With several feasible designs in hand, we now turn to choosing a “best” or preferred design and choose we must because rarely are the resources (e.g., time, money, and personnel) available to fully develop more than one design scheme, never mind all of our alternatives.

Question 2

Generate a numerical evaluation matrix for the juice container design problem.

Answer 2

Question 3

Generate a priority benchmark chart for the juice container design problem. This employs the priority checkmark method.

Answer 3

Question 4

Explain the priority checkmark method.

Answer 4

• The priority checkmark method is a simpler, qualitative version of the numerical evaluation matrix.
• We simply rank the objectives as high, medium, or low in priority.
  
  • Objectives with high priority are given three checks, those with medium priority are given two checks, while objectives with low priority are given only one check.
  • Similarly, metric results are assigned as 1 if they are awarded more than some arbitrary, but high value, such as 70 points (on a scale of 0–100), and as 0 if their award is less than the target value.
  • A design alternative that meets an objective in a “satisfactory” way is then marked with one or more checks,

Question 5

What is the process for creating a Best-of-Class Chart?

Answer 5

The process for creating a Best-of-Class Chart is as follows:

For each objective, we assign scores to each design alternative that start from 1 for the alternative that meets that objective best, increasing to 2 for second-best, and so on, until the alternative that met the objective worst is given a score equal to the number of alternatives being considered.

Question 6

Note an example of a best-of-class chart for the juice container design.

Answer 6

Question 7

What are important for the priority checkmark/benchmark method, the numerical evaluation matrix method, and the best-of-class method?

Answer 7

No matter which of the three selection methods is used:

• the priority checkmark/benchmark method
• the numerical evaluation matrix method
• the best-of-class method

….design evaluation and selection demand careful, thoughtful judgment.

Question 8

What is important when evaluating results.

Answer 8

Exercising common sense is important when evaluating results.

“There is no excuse for accepting results blindly and uncritically”.

Question 9

What are two examples of ways to exercise common sense when evaluating results?

Answer 9

If the metrics results for two alternative designs are relatively close, they should be treated as effectively equal, unless there are other unevaluated strengths or weaknesses.

Further, if we are surprised by our evaluations, we should ask whether our expectations were wrong, our measurements were consistently applied, or whether our rankings and our metrics are appropriate to the problem.