Evaluation Domain Flashcards
A business analyst is responsible for carrying out Solution Evaluation processes for each product delivery project. Solution Evaluation comprises both short-term and long-term evaluations of the product/solution. Which of the following statements is correct in regard to the long-term evaluation of a product or solution?
Long-term evaluation focuses on evaluating the business value delivered by the product or solution.
Long-term evaluation focuses on evaluating the product or solution during the usability testing.
Long-term evaluation focuses on evaluating the product or solution during the penetration testing.
Long-term evaluation focuses on evaluating the product or solution during the user acceptance testing.
Long-term evaluation focuses on evaluating the business value delivered by the product or solution.
Long-term evaluation focuses on evaluating the business value delivered by the product or solution. Product or solution evaluation during the acceptance testing is the short-term focus. The other two choices are irrelevant. [The PMI Guide to Business Analysis, page 275]
Solution evaluation activities can be carried out either qualitatively or quantitatively. The choice of the appropriate approach varies from project to project. All of the following are examples of performing qualitative or coarsely quantitative solution evaluation EXCEPT:
Analyzing the results of exploratory testing of solution functionality.
Analyzing the data created by the solution.
Evaluating the solution through collecting SMEs feedback during a workshop.
Conducting surveys and analyzing results.
Analyzing the data created by the solution.
Analyzing the results from surveys, focus groups, or the results of exploratory testing of functionality are examples of qualitative or coarsely quantitative evaluation activities. Other evaluation activities involve obtaining more precise quantitative measurements, such as directly looking at data from a solution. [The PMI Guide to Business Analysis, page 278]
Regardless of the project lifecycle, scope and magnitude, a business analyst performs a number of tasks during the Solution Evaluation stage. All of the following are tasks that are typically performed during this stage EXCEPT:
Collecting and analyzing functional and non-functional solution requirements.
Evaluating nonfunctional characteristics of the solution.
Comparing estimated and actual costs and benefits.
Evaluating the solution performance against performance standards in the service-level agreements.
Collecting and analyzing functional and non-functional solution requirements.
Collecting and analyzing functional and non-functional solution requirements are part of the Elicitation and Analysis stages. The rest of the tasks are all typically performed during the Solution Evaluation stage. [The PMI Guide to Business Analysis, page 278]
A business analyst is responsible for conducting Solution Evaluation activities for each product development or delivery project. Evaluation activities may occur at several points in time during the product lifecycle. Which of the following is not an appropriate time to conduct solution evaluation activities?
At any point when a go/no-go or release decision need to be made.
Early during the project initiation stage.
During a short-term period after a solution or segment is put into operation.
Well after a solution is put into operation to obtain a long-term perspective.
Early during the project initiation stage.
Solution Evaluation includes the processes to validate a full solution or a segment of a solution that is about to be or has already been implemented. Conducting evaluation activities early during project initiation doesn’t make sense. All other choices give valid points in time to conduct solution evaluation activities. [The PMI Guide to Business Analysis, page 279]
You have recently been assigned as the lead business analyst on an ERP system design and development project. This is a complex project and the entire system needs to be developed inhouse. Given the costs, time and resource commitment, the senior leadership is not only keen about how you initially develop the business case and conduct the feasibility study, they will also be very interested in how system evaluation activities are performed throughout the product lifecycle. Solution evaluation is usually complex and requires preparation. Which of the following statements regarding preparation for Solution Evaluation is correct?
Solution Evaluation preparation is carried out immediately prior to conducting Solution Evaluation.
Solution Evaluation preparation is carried out during project closure.
Solution Evaluation often requires early preparation.
Solution Evaluation preparation is carried out after project closure.
Solution Evaluation often requires early preparation.
Solution Evaluation often requires early preparation, so that what is needed to perform this work is in place later when evaluation is conducted. [The PMI Guide to Business Analysis, pages 277, 279]
You are the business analyst on a data center migration project that is halfway through execution. The solution requirements are classified into two categories: “must-haves”, core requirements that need to be delivered as part of the project’s first release; and “Nice-to-have”, supporting requirements that need to be delivered as part of the project’s second and final release. The project team has recently delivered the first releasee and you are now conducting Solution Evaluation activities for the project. In your opinion, the value that will be obtained by delivering “nice-to-have” supporting requirements does not justify the additional effort needed to deliver the second project release. If you can substantiate this analysis you can demonstrate:
The 80/20 rule on the project
The break-even point on the project
The net present value of the project
The point of diminishing returns
The point of diminishing returns
A point of diminishing returns is the point where additional value that could be obtained from a solution does not justify the additional effort needed to achieve that value. If you can substantiate your analysis, you can demonstrate that the “nice-to-have” supporting requirements should not be developed and the remaining funds need to be allocated to other higher-priority projects. [The PMI Guide to Business Analysis, page 279]
The senior business analyst has successfully demonstrated to you (the project sponsor) that your current system upgrade project has arrived at a point of diminishing returns. At this stage, the project has hardly delivered 70% of the solution requirements. What should you do next?
Allocate additional funds to the project to overturn the point of diminishing returns.
Allocate additional human resources to the project so that the project could complete earlier.
Terminate the project.
Ask the project team to deliver the remaining scope.
Terminate the project.
A point of diminishing returns is the point where additional value that could be obtained from a solution does not justify the additional effort needed to achieve that value. In this case, Solution Evaluation gives teams the ability to “end early” even if there is still additional functionality that could be built, allowing funds to be reallocated to work on higher-priority projects. [The PMI Guide to Business Analysis, page 279]
You are currently preparing for the Solution Evaluation activities for a new accounting system your organization is developing inhouse. All of the stated benefits of the project are intangible, and therefore not possible to measure. In this case, how would you conduct Solution Evaluation?
Define measurements that provide indirect evidence that the benefits have been achieved.
Calculate the project’s NPV.
Calculate the project’s IRR.
Do not measure the benefits during the Solution Evaluation.
Define measurements that provide indirect evidence that the benefits have been achieved.
You need to define measurements that provide indirect evidence that the benefits have been achieved. You cannot choose to ignore benefit measurements if benefits are intangible. Further, if the benefits are intangible, neither NPV or IRR can be used. [The PMI Guide to Business Analysis, page 279]
You are conducting Solution Evaluation for a recently released security management system for your organization. Some of the benchmarking information needed to evaluate the system is critical for the Solution Evaluation but is not useable by the system itself. What is the disadvantage of obtaining this additional information?
Scope creep
Waste of time
Additional cost
Diminishing returns
Additional cost
If the information is critical for Solution Evaluation, this need to be obtained. There might be some cost disadvantages there but this cannot be ignored. All other options are irrelevant. [The PMI Guide to Business Analysis, page 279]
You are leading a new workflow digitization project. The project will take about two years to deliver the final product. The product’s benefits stated in the business case can only be evaluated once the system is in production for at least a year. In this scenario, how would Solution Evaluation be performed?
The current business analyst will analyze the benefits during the project closure.
A prediction of future performance will be used as a substitute for Solution Evaluation.
Solution Evaluation need not be performed for this project.
Operational business area may lead the long-term Solution Evaluation.
Operational business area may lead the long-term Solution Evaluation.
Some aspects of a solution that reflect the benefits and value may not be measurable until well after a solution is released. In these situations, the operational business area responsible for the product may take responsibility for identifying and measuring leading indicators. [The PMI Guide to Business Analysis, page 279]
You have been asked to perform Solution Evaluation for a recently released system. Since you haven’t performed such a task before, you are looking for some guidance. Which of the following PMI Guide to Business Analysis Solution Evaluation processes will help you in determining whether the implemented solution has delivered its intended business value?
Evaluate Solution Performance.
Evaluate Acceptance Results and Address Defects.
Determine Solution Evaluation Approach.
Obtain Solution Acceptance for Release.
Evaluate Solution Performance.
Evaluate Solution Performance is the process of evaluating a solution to determine whether the implemented solution or solution component is delivering the business value as intended. This PMI Guide to Business Analysis process will help you with your current task. [The PMI Guide to Business Analysis, page 280]
You are a business analyst on a waterfall project which is currently in the Solution Evaluation stage. What is your authoritative source for the project’s expected benefits when determining whether the implemented solution has delivered its intended business value?
Business Case
Project Charter
Product Backlog
Scope Statement
Business Case
The business case describes pertinent information to determine whether the initiative is worth the required investment. It is an authoritative source where expected benefits have been stated. [The PMI Guide to Business Analysis, page 281]
You are evaluating a solution’s performance post its release. According to the business case, which was developed three years ago at the project initiation, the NPV of the project at 5% discount rate was $75,000. During the project, the discount rate changed to 7%. What will be the impact of this change?
The project’s NPV will decrease, which is an unfavorable condition.
The project’s NPV will decrease, which is a favorable condition.
The project’s NPV will increase, which is a favorable condition.
The project’s NPV will increase, which is an unfavorable condition.
The project’s NPV will decrease, which is an unfavorable condition.
The NPV expresses the value of an investment in today’s value of money. A higher discount rate reduces the present value of future returns and hence reduces net present value; which is not a favorable condition. [The PMI Guide to Business Analysis, pages 90, 283]
You are evaluating a solution’s performance post its release. According to the business case, which was developed three years ago at the project initiation, the NPV of the project at 5% discount rate was $15,000. During the project, the discount rate changed to 7% resulting in a new NPV value of $10,000. How should you interpret this result?
The project’s NPV has fallen, which is an unfavorable condition, however the project is still profitable.
The project’s NPV has fallen, which is a favorable condition, but the project has incurred a $5,000 loss.
The project’s NPV has fallen, which is a favorable condition, and the project is still profitable.
The project’s NPV has fallen, which is an unfavorable condition, and the project is no longer profitable.
The project’s NPV has fallen, which is an unfavorable condition, however the project is still profitable.
The project’s NPV has fallen, which is an unfavorable condition, however the project is still profitable since the NPV is still a positive value. [The PMI Guide to Business Analysis, pages 90, 283]
You are evaluating a product’s performance post its release. The product was intended to increase the firm’s market share. To analyze this, you have created a chart with one axis reflecting market growth from low to high, while the other reflecting the market share of the firm from low to high. What is this chart called?
MoSCoW chart
Tornado matrix
Fishbone diagram
Growth-share matrix
Growth-share matrix
A product portfolio matrix, also known as a growth-share matrix, is a market analysis quadrant diagram used by organizations to qualitatively analyze their products or product lines. One axis reflects market growth while the other reflects the market share. [The PMI Guide to Business Analysis, page 283]
